From 26ba250de8d2d9dc1d298f3eb15b62e17f819bb7 Mon Sep 17 00:00:00 2001 From: CentOS Sources Date: May 07 2019 09:40:55 +0000 Subject: import qemu-kvm-2.12.0-63.module+el8+2833+c7d6d092 --- diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..df9af11 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +SOURCES/qemu-2.12.0.tar.xz diff --git a/.qemu-kvm.metadata b/.qemu-kvm.metadata new file mode 100644 index 0000000..bda0ddd --- /dev/null +++ b/.qemu-kvm.metadata @@ -0,0 +1 @@ +5a62c911b2cebbd41decd5c77c524395212411cf SOURCES/qemu-2.12.0.tar.xz diff --git a/SOURCES/0001-Initial-redhat-build.patch b/SOURCES/0001-Initial-redhat-build.patch new file mode 100644 index 0000000..950b213 --- /dev/null +++ b/SOURCES/0001-Initial-redhat-build.patch @@ -0,0 +1,332 @@ +From 7000ac85100400a686b48562d830c7b14a439a94 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Mon, 11 Sep 2017 07:11:00 +0200 +Subject: Initial redhat build + +This patch introduces redhat build structure in redhat subdirectory. In addition, +several issues are fixed in QEMU tree: + +- Change of app name for sasl_server_init in VNC code from qemu to qemu-kvm + - As we use qemu-kvm as name in all places, this is updated to be consistent +- Man page renamed from qemu to qemu-kvm + - man page is installed using make install so we have to fix it in qemu tree +- Use "/share/qemu-kvm" as SHARE_SUFFIX + - We reconfigured our share to qemu-kvm to be consistent with used name +- Added .gitpublish configuration file + - Support for git publish has to be stored in repository root + +-- +Rebase notes for RHEL-8 (2.12.0): +- Conflict fix in .gitpublish +- Not packaging hppa-firmware.img +- Disable vxhs.o in block/Makefile.obj +- Disable ppc64 builds +- Removed acpi-dsdt.aml (upstream) +- dropped libjpeg build requirement +- Replaced buildrequirement librados2 and librbd1 by librados and librbd. +- dropping "-pie -fPIE -DPIE" as we're using --enable-pie +- Do not use tcmmaloc +- Dropping gperftools-devel dependency + +Rebase notes for RHEL-8.0 (2.11.0): +- Removed references to rhel6 rom files (virtio.rom, pcnet, rtl8139, + net2k_pci e1000, bios-256k, + pxe-e1000e.rom) +- Removed 80-kvm.rules file +- Changing ipxe linking to match upstream +- Updating kvm.modules and ksmtuned files +- cleaning %kvm_files section +- Remove CONFIG_RHV +- Remove CONFIG_LIVE_BLOCK_OPS +- Removing CONFIG_VTD +- Moving vxhs related changes to another commit +- removed live-block-migration option + +Rebase notes (2.11.0): +- Removed --with-pixman configure option (upstream) +- Enabling multipath for qemu-pr-helper +- Removed qemu-kvm-tools +- Removed references to "-rhev" and "-ma" +- Removing kvm-unit-tests +- Enabling qemu-geust-agent + +Rebase notes (2.10.0): +- live_block_migration option added upstream +- moved qmp-spec.txt file (upstream) +- added s390-netboot.img (added upstream) +- removed qemu_vga.ndrv (added upstream) +- switch to (rhevh-)rhel-7.5-candidate +- switched differentiation defaults +- moved binary files to separate commit + +Rebase notes (2.9.0): +- documentation files handling changes (upstrem) +- removed --enable-colo option and --disable-archipelago (upstream) +- bump BuildRequires versions +- new mandatory argument for tracetool.py (upstream) +- updated RHEL 6 roms +- switch from sha1sum to sha256sum +- Moved adding rhel6-e1000.rom from machine types commit +- Moved adding pxe-e1000e.rom from device disable commit +- Use rdma-core instead of librdmacm +- Add upstream tarballs tar.xz to .gitignore +- Updated git-backport-diff script + +Rebase notes (2.8.0): +- removed vhdx option (upstream) +- qemu-tech.html merged to qemu-doc.html (upstream) +- removed skiboot.lid firmware +- Changed tracetool.py parameters +- Added support for easy z-stream switch + +Rebase notes (2.7.0): +- removed kvm_stat +- added efi-e1000e.rom +- added efi-vmxnet.rom +- added linuxboot_dma.bin +- trace-events renamed to trace-events-all +- reverted dependency to seccomp on aarch64 +- Add ipxe-qemu-roms ad build dependency to pass tests + +Rebase notes (2.6.0): +- removed q35-acpi-dsdt.aml +- add enable-gcrypt option + +Rebase notes (2.5.0): +- New seccomp hadling in configure +- New condition format in migration/migration.c +- libcacard extracted +- vnc fixes +- libsecomp for aarch64 requirements changed downstream + +Rebase notes (2.4.0): +- remove --enable-ws-vnc +- use error_setg instead of error_set in migration/migration.c +- remove target-x86_64.conf +- create /etc/qemu-kvm on copying of bridge.conf +- disabled opengl +- rebased to version 2.3.0-30.el7 + +- Merged patches (Rebase 2.12.0) +- spec: Change License line +- spec: Do not depend on ipxe for Power architectures +- configuration: Use gcrypt instead of nettle +- spec: Use hardening flags for ksmctl build + +Merged patches (rebase 2.11.0) +- ce6e8e5b8a redhat/qemu-kvm.spec.template: Enable seccomp on s390x, too +- 8629f208c6 redhat: Remove qemu.binfmt from the downstream repository +- b889ce1c40 Disable build of qemu-kvm-ma for x86_64 +- 4506913c42 redhat: add CONFIG_RHV flag +- 21ecaec46f s390x: vm.allocate_pgste sysctl is no longer needed +- 78a1864d99 Update build_configure for 2.10.0 options +- decf881320 redhat: Provide s390x specific /etc/modprobe.d/kvm.conf +- e0cd3138cc redhat/qemu-kvm.spec: Use the freshly built s390-ccw.img firmware image +- 7af6b9a4fa redhat: install generic kvm.conf except for s390 and x86 architectures +- c4290f50bb redhat: fix rh-srpm target +- 8943f52e8b Package qemu-block-drivers manpage +- 88b41044d6 update spec to build and install qemu-pr-helper + +Merged patches (rebase 2.10.0) +- feefd46 qemu-kvm.spec: Enable s390x build +- 985051e Removing texi2html from build requirements +- 7c64a2a Update ignore files for redhat usage +- 8f9a95a Disable replication feature +- 1b7bbc5 block/vxhs: modularize VXHS via g_module +- 7511527 Remove the dependencies to seavgabios-bin and ipxe-roms-qemu on s390x +- aa0891c Downstream: Don't disable SMT on POWER9 hosts +- a13a0e9 Update configuration for qemu 2.9 +- bbf46dd disable pulseaudio and alsa +- 9124839 redhat/Makefile: honor BREW_FLAGS like the kernel +- 53c03bd copy SLIT test reference blobs into tests directory +- c4c77e6 Differentiation support +- f1ec0e8 configure: allow to disable VT-d emulation +- b972023 Disable VT-d for rhel builds +- 29a0414 RHEL Diff.: Add option in configure to disable live block ops +- 1f33b29 RHEL Diff.: Unregister live block operations +- c590551 RHEL Diff.: Disable live block operations in HMP monitor +- c7e208f RHEL Diff.: Add rpm spec options for live block ops +- 733af5c pegas: add rpm spec options for vhost-user +- ff16138 Add support for local build +- fb426d4 qemu-kvm.spec: Configure vm.allocate_pgste for s390x + +Merged patches (rebase 2.9.0) +- 9c7ab94 Enable seccomp for ppc64/ppc64le architecture +- f6d7e9d Update qemu-kvm package Summary and Description +- a9e55b6 Disable usbredir and libcacard for unsupported architectures +- 0218220 Update configuration for 2.8.0 release + +Merged patches (rebase 2.7.0) +- 2be6077 Fix SLOF dependency +- dc58590 spec: Remove dependency to ipxe-roms-qemu for aarch64 +- 357ef43 spec: link sgabios.bin only for x86_64 +- 08d82cc spec: Update rules before triggering for kvm device +- 8980a76 spec: Do not package ivshmem-server and ivshmem-client +- 027067c spec: add a sample kvm.conf to enable Nested Virtualization +- ba2ba30 Adjust locked memory limits to allow unprivileged VMs on Power +- e9740b0 Increase locked memory limit for all users, not just kvm group +- 8c301be add vgabios-virtio.bin symlink +- 4d03723 usb: enable streams support +- 2a9363e Add install dependency required for usb streams +- 9a54442 Add dump-guest-memory.py to all archs +- 73fffc9 add e1000e ipxe rom symlink +- aaaa2a9 Add 'luks' to block driver whitelist +- c78c3a8 redhat: switch from gcrypt to nettle for crypto +- bb51a69 redhat: include username and date in RPM N-E-V-R for scratch builds + +Merged patches (rebase 2.4.0) +- 9201274 spec: Remove obsolete differentiation code +- a938a8c spec: Use external configuration script +- 5ca8d0e spec: Use configure options to prevent default resolution +- 5dca391 spec: Ship complete QMP documentation files +- 7899edd aarch64: allow --enable-seccomp +- a56fb9c aarch64: redhat spec: enable seccomp +- a9571e6 rhel: Update package version for SLOF dependency +- 25c70c4 configure: Add support for tcmalloc +- db72485 Change fsreeze-hook default location +- 14b8a9e redhat: add kvm-unit-tests tarball to environment +- 5ee4238 spec: Build tscdeadline_latency.flat from kvm-unit-tests +- 6ba800b Downstream-only: Start kvm-setup service before libvirtd service +- 59b43d6 Do not stop qemu-guest-agent service on target switch +- 4d851fa provide vhost module config file with max_mem_regions set to 509 +- 0b18027 spec: Require proper version of SLOF +- 3c436c7 Fix rh-brew-aarch64, rh-brew-ppc rh-brew-ga-ppc target + +(cherry picked from commit ba7591ec4a0906121d15ffbf740580bd79ec5814) +--- + .gitpublish | 58 +-- + Makefile | 2 +- + block/Makefile.objs | 2 +- + os-posix.c | 2 +- + redhat/.gitignore | 5 + + redhat/85-kvm.preset | 5 + + redhat/95-kvm-memlock.conf | 10 + + redhat/99-qemu-guest-agent.rules | 2 + + redhat/Makefile | 88 ++++ + redhat/Makefile.common | 48 +++ + redhat/Makefile.local | 76 ++++ + redhat/bridge.conf | 1 + + redhat/build_configure.sh | 145 +++++++ + redhat/ksm.service | 13 + + redhat/ksm.sysconfig | 4 + + redhat/ksmctl.c | 77 ++++ + redhat/ksmtuned | 139 +++++++ + redhat/ksmtuned.conf | 21 + + redhat/ksmtuned.service | 12 + + redhat/kvm-s390x.conf | 7 + + redhat/kvm-setup | 40 ++ + redhat/kvm-setup.service | 14 + + redhat/kvm-x86.conf | 12 + + redhat/kvm.conf | 3 + + redhat/kvm.modules | 18 + + redhat/qemu-ga.sysconfig | 19 + + redhat/qemu-guest-agent.service | 20 + + redhat/qemu-kvm.spec.template | 794 +++++++++++++++++++++++++++++++++++++ + redhat/qemu-pr-helper.service | 15 + + redhat/qemu-pr-helper.socket | 9 + + redhat/rpmbuild/BUILD/.gitignore | 2 + + redhat/rpmbuild/RPMS/.gitignore | 2 + + redhat/rpmbuild/SOURCES/.gitignore | 2 + + redhat/rpmbuild/SPECS/.gitignore | 2 + + redhat/rpmbuild/SRPMS/.gitignore | 2 + + redhat/scripts/frh.py | 26 ++ + redhat/scripts/git-backport-diff | 327 +++++++++++++++ + redhat/scripts/git-compile-check | 215 ++++++++++ + redhat/scripts/process-patches.sh | 78 ++++ + redhat/scripts/tarball_checksum.sh | 3 + + redhat/vhost.conf | 3 + + ui/vnc.c | 2 +- + 42 files changed, 2270 insertions(+), 55 deletions(-) + create mode 100644 redhat/.gitignore + create mode 100644 redhat/85-kvm.preset + create mode 100644 redhat/95-kvm-memlock.conf + create mode 100644 redhat/99-qemu-guest-agent.rules + create mode 100644 redhat/Makefile + create mode 100644 redhat/Makefile.common + create mode 100644 redhat/Makefile.local + create mode 100644 redhat/bridge.conf + create mode 100755 redhat/build_configure.sh + create mode 100644 redhat/ksm.service + create mode 100644 redhat/ksm.sysconfig + create mode 100644 redhat/ksmctl.c + create mode 100644 redhat/ksmtuned + create mode 100644 redhat/ksmtuned.conf + create mode 100644 redhat/ksmtuned.service + create mode 100644 redhat/kvm-s390x.conf + create mode 100644 redhat/kvm-setup + create mode 100644 redhat/kvm-setup.service + create mode 100644 redhat/kvm-x86.conf + create mode 100644 redhat/kvm.conf + create mode 100644 redhat/kvm.modules + create mode 100644 redhat/qemu-ga.sysconfig + create mode 100644 redhat/qemu-guest-agent.service + create mode 100644 redhat/qemu-kvm.spec.template + create mode 100644 redhat/qemu-pr-helper.service + create mode 100644 redhat/qemu-pr-helper.socket + create mode 100644 redhat/rpmbuild/BUILD/.gitignore + create mode 100644 redhat/rpmbuild/RPMS/.gitignore + create mode 100644 redhat/rpmbuild/SOURCES/.gitignore + create mode 100644 redhat/rpmbuild/SPECS/.gitignore + create mode 100644 redhat/rpmbuild/SRPMS/.gitignore + create mode 100755 redhat/scripts/frh.py + create mode 100755 redhat/scripts/git-backport-diff + create mode 100755 redhat/scripts/git-compile-check + create mode 100755 redhat/scripts/process-patches.sh + create mode 100755 redhat/scripts/tarball_checksum.sh + create mode 100644 redhat/vhost.conf + +diff --git a/Makefile b/Makefile +index d71dd5b..89ba4c5 100644 +--- a/Makefile ++++ b/Makefile +@@ -804,7 +804,7 @@ install-doc: $(DOCS) + $(INSTALL_DATA) docs/interop/qemu-qmp-ref.txt "$(DESTDIR)$(qemu_docdir)" + ifdef CONFIG_POSIX + $(INSTALL_DIR) "$(DESTDIR)$(mandir)/man1" +- $(INSTALL_DATA) qemu.1 "$(DESTDIR)$(mandir)/man1" ++ $(INSTALL_DATA) qemu.1 "$(DESTDIR)$(mandir)/man1/qemu-kvm.1" + $(INSTALL_DIR) "$(DESTDIR)$(mandir)/man7" + $(INSTALL_DATA) docs/interop/qemu-qmp-ref.7 "$(DESTDIR)$(mandir)/man7" + $(INSTALL_DATA) docs/qemu-block-drivers.7 "$(DESTDIR)$(mandir)/man7" +diff --git a/block/Makefile.objs b/block/Makefile.objs +index d644bac..c0693fc 100644 +--- a/block/Makefile.objs ++++ b/block/Makefile.objs +@@ -20,7 +20,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o + block-obj-$(CONFIG_CURL) += curl.o + block-obj-$(CONFIG_RBD) += rbd.o + block-obj-$(CONFIG_GLUSTERFS) += gluster.o +-block-obj-$(CONFIG_VXHS) += vxhs.o ++#block-obj-$(CONFIG_VXHS) += vxhs.o + block-obj-$(CONFIG_LIBSSH2) += ssh.o + block-obj-y += accounting.o dirty-bitmap.o + block-obj-y += write-threshold.o +diff --git a/os-posix.c b/os-posix.c +index b9c2343..05de8ee 100644 +--- a/os-posix.c ++++ b/os-posix.c +@@ -75,7 +75,7 @@ void os_setup_signal_handling(void) + /* Find a likely location for support files using the location of the binary. + For installed binaries this will be "$bindir/../share/qemu". When + running from the build tree this will be "$bindir/../pc-bios". */ +-#define SHARE_SUFFIX "/share/qemu" ++#define SHARE_SUFFIX "/share/qemu-kvm" + #define BUILD_SUFFIX "/pc-bios" + char *os_find_datadir(void) + { +diff --git a/ui/vnc.c b/ui/vnc.c +index e164eb7..0c3011b 100644 +--- a/ui/vnc.c ++++ b/ui/vnc.c +@@ -4045,7 +4045,7 @@ void vnc_display_open(const char *id, Error **errp) + trace_vnc_auth_init(vd, 1, vd->ws_auth, vd->ws_subauth); + + #ifdef CONFIG_VNC_SASL +- if ((saslErr = sasl_server_init(NULL, "qemu")) != SASL_OK) { ++ if ((saslErr = sasl_server_init(NULL, "qemu-kvm")) != SASL_OK) { + error_setg(errp, "Failed to initialize SASL auth: %s", + sasl_errstring(saslErr, NULL, NULL)); + goto fail; +-- +1.8.3.1 + diff --git a/SOURCES/0002-Enable-disable-devices-for-RHEL-7.patch b/SOURCES/0002-Enable-disable-devices-for-RHEL-7.patch new file mode 100644 index 0000000..64a847f --- /dev/null +++ b/SOURCES/0002-Enable-disable-devices-for-RHEL-7.patch @@ -0,0 +1,1786 @@ +From 85de0b0c587b62ad59db0f1ca4ff5d76d0c88634 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Mon, 11 Jan 2016 11:53:33 +0100 +Subject: Enable/disable devices for RHEL 7 + +This commit adds all changes related to changes in supported devices +up to qemu-kvm-rhev-2.1.2-16.el7. + +Signed-off-by: Miroslav Rezanina + +Rebase notes for RHEL8 - 2.12: +Disable AMD_IOMMU +New handling of VHOST_USER_* device disabling +Use cpu_type instead of cpu_model +Disabled CONFIG_P4XXX for power64 +Disabled sam460ex board tests + +Rebase notes for RHEL-8.0: +- Use upstream's romfile for e1000e.c +- Removed CONFIG_VTD references in hw/i386/Makefile.objs +- Comment tests out instead of dropping them +- Dropped duplicated CONFIG_I2C + +Rebase notes (2.11.0): +- Switched order with machine type commit + +Rebase notes (2.10.0): +- replace cannot_instantiate_with_device_add_yet with user_creatable (upstream e90f2a) +- Comment out default configs instead of removing the options +- Reenable test_unaligned_write_same +- Remove default-configs changes for 32bit architectures +- Removed unnecessary usage of user_creatable +- Documented arm cpu changes + +Rebase notes (2.9.0): +- enabled CONFIG_ARM_V7M for aarch64 +- need to fix cpu mask +- disabled some irrelevant ppc64 tests +- Adding pxe-e1000e.rom to Inital redhat commit +- Context changed in vfio_probe_igd_bar4_quirk (rc3) +- Remove fdc test +- Enabled null-co driver +- Disabled megasas and bios-tables test +- Disabled netfilter tests for ppc64 +- Disabled ivshmem test for ppc64 +- Disabled numa test for aarhc64 +- Add CONFIG_USB_XHCI_NEC option for ppc64 +- Enable vhost-user-scsi for ppc64 and aarch64 +- Disable ipmi_bt, ipmi_local, impi_extern for ppc64 + +Rebase notes (2.8.0): +- Removed CONFIG_PIIX_PCI (upstream) +- Disabled POWERNV +- Disabled additional occurencies of prom-env-test +- Disabled additional occurencies of usb-hcd-ohci-test and usb-hcd-uhci-test +- Disabled unsupported machine types in boot-serieal-test +- Disabled 2.7 machine type compatibility test in test-x86-cpuid-compat +- Disabled pnv-xscom-test +- Disabled RHEl 6 machine types for qom-test (failing acpi setting) +- Added diffutils BuildRequires and diff usage for make check + +Rebase notes (2.6.0): +- disabled prom-env-test + +Rebase notes (2.4.0): +- fixed types +- include CONFIG_PLATFORM_BUS for aarch64 +- disable failing virtio-scsi-test + +Rebase notes (2.3.0): +- Added USB=y in 2.3-rc2 (used instead of downstream version fora aarch64) + +Merged patches (2.11.): +- e6c4563d68 hw: Remove the redundant user_creatable = false from SYS_BUS_DEVICEs +- 35474b23b1 hw/pci-host/q35: Remove redundant downstream user_creatable = false +- 374294de46 hw/dma/i8257: Remove redundant downstream user_creatable = false +- 17abc4f5bd RHEL: Disable vfio-ccw and x-terminal270 devices +- b303e792c3 Disable vhost-user-scsi and vhost-user-scsi-pci +- 28f294c023 Disable sm501 and sysbus-sm501 devices +- d5959fcefc s390x/cpumodel: Disable unsupported CPU models + +Merged patches (2.10.0): +- e9b413e Add PCIe bridge devices for AArch64 +- a522114 s390x/virtio-ccw: Disable crypto device in downstream RHEL builds +- 15dbf98 Disable unimplemented device +- 5c0ea49 Disable serial-isa for ppc64 +- 728e7e8 Disable rs6000-mc device +- 2a11896 ppc64le: Remove isabus-bridge device +- 5c4df94 Reenable Educational device +- a936463 aarch64: Enable usb-xhci +- 984f5cd Enable USB_CONFIG for aarch64 +- f13b783 AArch64: Add pci-testdev +- 81867af Disable virtio-pci for s390x builds +- bf5f636 target/ppc: Show POWER9 in -cpu help + +Merged patches (2.9.0): +- 9320fc1 Fix unuseds/Fedora build +- cb16934 config: Remove EHCI from ppc64 builds +- 626fe4d Disable qemu,register device +- 783a0b2 Disable vfio-pci-igd-lpc-bridge device +- bf7c127 Disable new virtio crypto devices +- a4d3c4e Disable amd iommu devices +- 5396ebc Disable loader device +- 1957779 Disable or-irq device +- 25ee621 Hide new floppy device +- 2bd29ed Disable devices for for AArch64 QEMU + +Merged patches (2.7.0): +- e2944a5 RHEL: Disable unsupported PowerPC CPU models + - have to add additional "if 0" sections to hw/ppc/spapr_cpu_core.c +- 81b2836 Remove unsupported VFIO devices from QEMU +- 1248029 Disable spapr-rng +- 351e1fb Disable Windows enlightnementas +- 5760290 Disable mptsas1068 device +- 0b74460 Disable sd-card +- 2eaf71c Disable rocker device +- e0ed699 Disable new ipmi devices + - disable ipmi tests in bios-tables-test +- 30e3bee Disable hyperv-testdev +- 8a9aadf Disable allwiner_ahci device +- a41119d Disable igd-passthrough-i440FX +- e305bb4 Disable vfio-platform device +- a127042 rhel: Revert unwanted inconsequential changes to ivshmem +- ce1419c rhel: Disable ivshmem-plain migration, ivshmem-doorbell, ivshmem +- 3f9349b q35: disable s3/s4 by default +- 2158ca1 i8257: Set no-user flag +- b2a3bb9 e1000e: add boot rom +- have to add pxe-e1000e.rom to source files + +Merged patches (2.6.0): +- ce3206a qemu-iotests: Fix broken test cases + - Reduced to disabling test 071 only +- bb34585 qemu-iotests: Disable 099 (requires blkverify) +- 81be408 build: reenable local builds to pass --enable-debug (downstream only) + +Merged patches (2.4.0): +- fa4fd10 AArch64: Enable ACPI +- 1219d52 ivshmem: RHEL-only: remove unsupported code +- 5f6d954 ivshmem: RHEL-only: explicitly remove dead code +- b88bbf0 Revert "rhel: Drop "ivshmem" device" +- 8f0aadf Split serial-isa into its own config option +- 01bff0f rhel: Disable "info irq" and "info pic" for Power +- b915077 RHEL: Disable remaining unsupported devices for ppc +- 64cbdc5 Mark onboard devices as cannot_instantiate_with_device_add_yet +- 4792566 Disable sdhci device +- bda8169 Disable Educational device +- a17a8fb rhel: Revert unwanted cannot_instantiate_with_device_add_yet changes +- 91c76c5 Remove intel-iommu device +- ec1615d Disable additional e1000 models + +(cherry picked from commit 0e72e616b2d80e47c0eb6c5976276e9f8d920e92) +--- + default-configs/aarch64-softmmu.mak | 35 +++++++++--- + default-configs/pci.mak | 38 ++++++------- + default-configs/ppc64-softmmu.mak | 26 ++++++--- + default-configs/s390x-softmmu.mak | 12 +++-- + default-configs/sound.mak | 8 +-- + default-configs/usb.mak | 14 ++--- + default-configs/x86_64-softmmu.mak | 30 ++++++----- + hw/acpi/ich9.c | 4 +- + hw/block/fdc.c | 1 + + hw/char/serial-pci.c | 4 ++ + hw/core/Makefile.objs | 7 +-- + hw/display/cirrus_vga.c | 2 + + hw/i386/pc.c | 2 + + hw/ide/piix.c | 5 +- + hw/ide/via.c | 2 + + hw/input/pckbd.c | 2 + + hw/misc/Makefile.objs | 2 +- + hw/misc/ivshmem.c | 11 ++++ + hw/net/e1000.c | 2 + + hw/pci-host/piix.c | 4 ++ + hw/ppc/Makefile.objs | 2 +- + hw/ppc/spapr.c | 3 +- + hw/ppc/spapr_cpu_core.c | 2 + + hw/s390x/virtio-ccw.c | 8 +++ + hw/usb/ccid-card-emulated.c | 2 + + hw/vfio/Makefile.objs | 3 -- + hw/vfio/pci-quirks.c | 5 ++ + hw/virtio/Makefile.objs | 5 +- + hw/virtio/virtio-pci.c | 8 +-- + qemu-options.hx | 5 -- + stubs/Makefile.objs | 1 + + stubs/ide-isa.c | 13 +++++ + target/arm/cpu.c | 4 +- + target/i386/cpu.c | 37 ++++++++++--- + target/ppc/cpu-models.c | 17 +++++- + target/s390x/cpu_models.c | 3 ++ + target/s390x/kvm.c | 8 +++ + tests/Makefile.include | 104 ++++++++++++++++++------------------ + tests/bios-tables-test.c | 4 ++ + tests/boot-order-test.c | 7 +++ + tests/boot-serial-test.c | 10 ++-- + tests/e1000-test.c | 2 + + tests/endianness-test.c | 2 + + tests/ivshmem-test.c | 10 +++- + tests/qemu-iotests/051 | 12 ++--- + tests/qemu-iotests/group | 4 +- + tests/qom-test.c | 4 +- + tests/test-hmp.c | 2 +- + tests/test-x86-cpuid-compat.c | 2 + + tests/usb-hcd-xhci-test.c | 5 +- + vl.c | 2 +- + 51 files changed, 341 insertions(+), 166 deletions(-) + create mode 100644 stubs/ide-isa.c + +diff --git a/default-configs/aarch64-softmmu.mak b/default-configs/aarch64-softmmu.mak +index 9ddccf8..001eb8e 100644 +--- a/default-configs/aarch64-softmmu.mak ++++ b/default-configs/aarch64-softmmu.mak +@@ -1,10 +1,29 @@ + # Default configuration for aarch64-softmmu + +-# We support all the 32 bit boards so need all their config +-include arm-softmmu.mak +- +-CONFIG_AUX=y +-CONFIG_DDC=y +-CONFIG_DPCD=y +-CONFIG_XLNX_ZYNQMP=y +-CONFIG_XLNX_ZYNQMP_ARM=y ++# CONFIG_AUX=y ++# CONFIG_DDC=y ++# CONFIG_DPCD=y ++# CONFIG_XLNX_ZYNQMP=y ++# CONFIG_XLNX_ZYNQMP_ARM=y ++CONFIG_PCI=y ++CONFIG_PCI_TESTDEV=y ++CONFIG_VIRTIO_PCI=y ++CONFIG_VIRTIO=y ++CONFIG_ARM_GIC=y ++CONFIG_ARM_GIC_KVM=$(CONFIG_KVM) ++CONFIG_PL011=y ++CONFIG_PL031=y ++CONFIG_PFLASH_CFI01=y ++CONFIG_PCI_GENERIC=y ++CONFIG_ACPI=y ++CONFIG_PLATFORM_BUS=y ++CONFIG_SMBIOS=y ++CONFIG_PL061=y ++CONFIG_GPIO_KEY=y ++CONFIG_ARM_V7M=y ++CONFIG_PCIE_PORT=y ++CONFIG_XIO3130=y ++CONFIG_IOH3420=y ++CONFIG_USB_XHCI=y ++CONFIG_USB=y ++CONFIG_I2C=y +diff --git a/default-configs/pci.mak b/default-configs/pci.mak +index 35e7596..4c8c296 100644 +--- a/default-configs/pci.mak ++++ b/default-configs/pci.mak +@@ -4,21 +4,21 @@ CONFIG_ISA_BUS=y + CONFIG_VIRTIO_PCI=y + CONFIG_VIRTIO=y + CONFIG_USB_UHCI=y +-CONFIG_USB_OHCI=y ++#CONFIG_USB_OHCI=y + CONFIG_USB_EHCI=y + CONFIG_USB_XHCI=y + CONFIG_USB_XHCI_NEC=y +-CONFIG_NE2000_PCI=y +-CONFIG_EEPRO100_PCI=y +-CONFIG_PCNET_PCI=y +-CONFIG_PCNET_COMMON=y ++#CONFIG_NE2000_PCI=y ++#CONFIG_EEPRO100_PCI=y ++#CONFIG_PCNET_PCI=y ++#CONFIG_PCNET_COMMON=y + CONFIG_AC97=y + CONFIG_HDA=y +-CONFIG_ES1370=y +-CONFIG_LSI_SCSI_PCI=y +-CONFIG_VMW_PVSCSI_SCSI_PCI=y +-CONFIG_MEGASAS_SCSI_PCI=y +-CONFIG_MPTSAS_SCSI_PCI=y ++#CONFIG_ES1370=y ++#CONFIG_LSI_SCSI_PCI=y ++#CONFIG_VMW_PVSCSI_SCSI_PCI=y ++#CONFIG_MEGASAS_SCSI_PCI=y ++#CONFIG_MPTSAS_SCSI_PCI=y + CONFIG_RTL8139_PCI=y + CONFIG_E1000_PCI=y + CONFIG_E1000E_PCI=y +@@ -26,24 +26,24 @@ CONFIG_IDE_CORE=y + CONFIG_IDE_QDEV=y + CONFIG_IDE_PCI=y + CONFIG_AHCI=y +-CONFIG_ESP=y +-CONFIG_ESP_PCI=y ++#CONFIG_ESP=y ++#CONFIG_ESP_PCI=y + CONFIG_SERIAL=y + CONFIG_SERIAL_ISA=y + CONFIG_SERIAL_PCI=y + CONFIG_CAN_BUS=y + CONFIG_CAN_SJA1000=y + CONFIG_CAN_PCI=y +-CONFIG_IPACK=y ++#CONFIG_IPACK=y + CONFIG_WDT_IB6300ESB=y + CONFIG_PCI_TESTDEV=y +-CONFIG_NVME_PCI=y +-CONFIG_SD=y +-CONFIG_SDHCI=y ++#CONFIG_NVME_PCI=y ++#CONFIG_SD=y ++#CONFIG_SDHCI=y + CONFIG_EDU=y + CONFIG_VGA=y + CONFIG_VGA_PCI=y + CONFIG_IVSHMEM_DEVICE=$(CONFIG_IVSHMEM) +-CONFIG_ROCKER=y +-CONFIG_VHOST_USER_SCSI=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) +-CONFIG_VHOST_USER_BLK=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) ++#CONFIG_ROCKER=y ++#CONFIG_VHOST_USER_SCSI=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) ++#CONFIG_VHOST_USER_BLK=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) +diff --git a/default-configs/ppc64-softmmu.mak b/default-configs/ppc64-softmmu.mak +index b94af6c..0ee8f6c 100644 +--- a/default-configs/ppc64-softmmu.mak ++++ b/default-configs/ppc64-softmmu.mak +@@ -1,14 +1,28 @@ + # Default configuration for ppc64-softmmu + +-# Include all 32-bit boards +-include ppc-softmmu.mak ++include sound.mak ++include usb.mak ++ ++## PCI configuration - cut down from the defaults in pci.mak ++CONFIG_PCI=y ++CONFIG_VIRTIO_PCI=y ++CONFIG_VIRTIO=y ++CONFIG_USB_XHCI=y ++CONFIG_USB_XHCI_NEC=y ++CONFIG_WDT_IB6300ESB=y ++CONFIG_PCI_TESTDEV=y ++CONFIG_USB_OHCI=y ++CONFIG_VGA=y ++CONFIG_VGA_PCI=y ++CONFIG_SERIAL=y ++CONFIG_I2C=y + + # For PowerNV +-CONFIG_POWERNV=y ++#CONFIG_POWERNV=y + CONFIG_IPMI=y +-CONFIG_IPMI_LOCAL=y +-CONFIG_IPMI_EXTERN=y +-CONFIG_ISA_IPMI_BT=y ++#CONFIG_IPMI_LOCAL=y ++#CONFIG_IPMI_EXTERN=y ++#CONFIG_ISA_IPMI_BT=y + + # For pSeries + CONFIG_PSERIES=y +diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak +index 2f4bfe7..649bf2c 100644 +--- a/default-configs/s390x-softmmu.mak ++++ b/default-configs/s390x-softmmu.mak +@@ -1,11 +1,13 @@ + CONFIG_PCI=y +-CONFIG_VIRTIO_PCI=$(CONFIG_PCI) +-CONFIG_VHOST_USER_SCSI=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) +-CONFIG_VHOST_USER_BLK=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) ++#CONFIG_VIRTIO_PCI=$(CONFIG_PCI) ++#CONFIG_VHOST_USER_SCSI=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) ++#CONFIG_VHOST_USER_BLK=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) + CONFIG_VIRTIO=y + CONFIG_SCLPCONSOLE=y +-CONFIG_TERMINAL3270=y ++# Disabled for Red Hat Enterprise Linux: ++# CONFIG_TERMINAL3270=y + CONFIG_S390_FLIC=y + CONFIG_S390_FLIC_KVM=$(CONFIG_KVM) +-CONFIG_VFIO_CCW=$(CONFIG_LINUX) ++# Disabled for Red Hat Enterprise Linux: ++# CONFIG_VFIO_CCW=$(CONFIG_LINUX) + CONFIG_WDT_DIAG288=y +diff --git a/default-configs/sound.mak b/default-configs/sound.mak +index 4f22c34..1bead9b 100644 +--- a/default-configs/sound.mak ++++ b/default-configs/sound.mak +@@ -1,4 +1,4 @@ +-CONFIG_SB16=y +-CONFIG_ADLIB=y +-CONFIG_GUS=y +-CONFIG_CS4231A=y ++#CONFIG_SB16=y ++#CONFIG_ADLIB=y ++#CONFIG_GUS=y ++#CONFIG_CS4231A=y +diff --git a/default-configs/usb.mak b/default-configs/usb.mak +index f4b8568..a256f84 100644 +--- a/default-configs/usb.mak ++++ b/default-configs/usb.mak +@@ -1,10 +1,10 @@ + CONFIG_USB=y +-CONFIG_USB_TABLET_WACOM=y ++#CONFIG_USB_TABLET_WACOM=y + CONFIG_USB_STORAGE_BOT=y +-CONFIG_USB_STORAGE_UAS=y +-CONFIG_USB_STORAGE_MTP=y ++#CONFIG_USB_STORAGE_UAS=y ++#CONFIG_USB_STORAGE_MTP=y + CONFIG_USB_SMARTCARD=y +-CONFIG_USB_AUDIO=y +-CONFIG_USB_SERIAL=y +-CONFIG_USB_NETWORK=y +-CONFIG_USB_BLUETOOTH=y ++#CONFIG_USB_AUDIO=y ++#CONFIG_USB_SERIAL=y ++#CONFIG_USB_NETWORK=y ++#CONFIG_USB_BLUETOOTH=y +diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak +index 0390b43..2675606 100644 +--- a/default-configs/x86_64-softmmu.mak ++++ b/default-configs/x86_64-softmmu.mak +@@ -4,20 +4,21 @@ include pci.mak + include sound.mak + include usb.mak + CONFIG_QXL=$(CONFIG_SPICE) +-CONFIG_VGA_ISA=y ++#CONFIG_VGA_ISA=y ++CONFIG_VGA_PCI=y + CONFIG_VGA_CIRRUS=y +-CONFIG_VMWARE_VGA=y +-CONFIG_VMXNET3_PCI=y ++#CONFIG_VMWARE_VGA=y ++#CONFIG_VMXNET3_PCI=y + CONFIG_VIRTIO_VGA=y + CONFIG_VMMOUSE=y + CONFIG_IPMI=y +-CONFIG_IPMI_LOCAL=y +-CONFIG_IPMI_EXTERN=y +-CONFIG_ISA_IPMI_KCS=y +-CONFIG_ISA_IPMI_BT=y ++#CONFIG_IPMI_LOCAL=y ++#CONFIG_IPMI_EXTERN=y ++#CONFIG_ISA_IPMI_KCS=y ++#CONFIG_ISA_IPMI_BT=y + CONFIG_SERIAL=y + CONFIG_SERIAL_ISA=y +-CONFIG_PARALLEL=y ++#CONFIG_PARALLEL=y + CONFIG_I8254=y + CONFIG_PCSPK=y + CONFIG_PCKBD=y +@@ -29,11 +30,11 @@ CONFIG_ACPI_MEMORY_HOTPLUG=y + CONFIG_ACPI_CPU_HOTPLUG=y + CONFIG_APM=y + CONFIG_I8257=y +-CONFIG_IDE_ISA=y ++#CONFIG_IDE_ISA=y + CONFIG_IDE_PIIX=y +-CONFIG_NE2000_ISA=y +-CONFIG_HPET=y +-CONFIG_APPLESMC=y ++#CONFIG_NE2000_ISA=y ++#CONFIG_HPET=y ++#CONFIG_APPLESMC=y + CONFIG_I8259=y + CONFIG_PFLASH_CFI01=y + CONFIG_TPM_TIS=$(CONFIG_TPM) +@@ -41,6 +42,7 @@ CONFIG_TPM_CRB=$(CONFIG_TPM) + CONFIG_MC146818RTC=y + CONFIG_PCI_PIIX=y + CONFIG_WDT_IB700=y ++CONFIG_ISA_BUS=y + CONFIG_ISA_DEBUG=y + CONFIG_ISA_TESTDEV=y + CONFIG_VMPORT=y +@@ -58,11 +60,11 @@ CONFIG_XIO3130=y + CONFIG_IOH3420=y + CONFIG_I82801B11=y + CONFIG_SMBIOS=y +-CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM) ++#CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM) + CONFIG_PXB=y + CONFIG_ACPI_VMGENID=y + CONFIG_FW_CFG_DMA=y + CONFIG_I2C=y + CONFIG_SEV=$(CONFIG_KVM) + CONFIG_VTD=y +-CONFIG_AMD_IOMMU=y ++#CONFIG_AMD_IOMMU=y +diff --git a/hw/acpi/ich9.c b/hw/acpi/ich9.c +index c5d8646..a4e87b8 100644 +--- a/hw/acpi/ich9.c ++++ b/hw/acpi/ich9.c +@@ -446,8 +446,8 @@ void ich9_pm_add_properties(Object *obj, ICH9LPCPMRegs *pm, Error **errp) + static const uint32_t gpe0_len = ICH9_PMIO_GPE0_LEN; + pm->acpi_memory_hotplug.is_enabled = true; + pm->cpu_hotplug_legacy = true; +- pm->disable_s3 = 0; +- pm->disable_s4 = 0; ++ pm->disable_s3 = 1; ++ pm->disable_s4 = 1; + pm->s4_val = 2; + + object_property_add_uint32_ptr(obj, ACPI_PM_PROP_PM_IO_BASE, +diff --git a/hw/block/fdc.c b/hw/block/fdc.c +index cd29e27..3964096 100644 +--- a/hw/block/fdc.c ++++ b/hw/block/fdc.c +@@ -605,6 +605,7 @@ static void floppy_drive_class_init(ObjectClass *klass, void *data) + k->bus_type = TYPE_FLOPPY_BUS; + k->props = floppy_drive_properties; + k->desc = "virtual floppy drive"; ++ k->user_creatable = false; /* RH state preserve */ + } + + static const TypeInfo floppy_drive_info = { +diff --git a/hw/char/serial-pci.c b/hw/char/serial-pci.c +index cb0d04c..d426982 100644 +--- a/hw/char/serial-pci.c ++++ b/hw/char/serial-pci.c +@@ -228,6 +228,8 @@ static void multi_2x_serial_pci_class_initfn(ObjectClass *klass, void *data) + dc->vmsd = &vmstate_pci_multi_serial; + dc->props = multi_2x_serial_pci_properties; + set_bit(DEVICE_CATEGORY_INPUT, dc->categories); ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static void multi_4x_serial_pci_class_initfn(ObjectClass *klass, void *data) +@@ -243,6 +245,8 @@ static void multi_4x_serial_pci_class_initfn(ObjectClass *klass, void *data) + dc->vmsd = &vmstate_pci_multi_serial; + dc->props = multi_4x_serial_pci_properties; + set_bit(DEVICE_CATEGORY_INPUT, dc->categories); ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo serial_pci_info = { +diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs +index eb88ca9..2b4491f 100644 +--- a/hw/core/Makefile.objs ++++ b/hw/core/Makefile.objs +@@ -16,10 +16,11 @@ common-obj-$(CONFIG_SOFTMMU) += machine.o + common-obj-$(CONFIG_SOFTMMU) += loader.o + common-obj-$(CONFIG_FITLOADER) += loader-fit.o + common-obj-$(CONFIG_SOFTMMU) += qdev-properties-system.o +-common-obj-$(CONFIG_SOFTMMU) += register.o +-common-obj-$(CONFIG_SOFTMMU) += or-irq.o ++# Disabled in Red Hat Enterprise Linux ++# common-obj-$(CONFIG_SOFTMMU) += register.o ++# obj-$(CONFIG_SOFTMMU) += generic-loader.o ++# common-obj-$(CONFIG_SOFTMMU) += or-irq.o + common-obj-$(CONFIG_SOFTMMU) += split-irq.o + common-obj-$(CONFIG_PLATFORM_BUS) += platform-bus.o + +-obj-$(CONFIG_SOFTMMU) += generic-loader.o + obj-$(CONFIG_SOFTMMU) += null-machine.o +diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c +index 138ae96..d116651 100644 +--- a/hw/display/cirrus_vga.c ++++ b/hw/display/cirrus_vga.c +@@ -3074,6 +3074,8 @@ static void isa_cirrus_vga_class_init(ObjectClass *klass, void *data) + dc->realize = isa_cirrus_vga_realizefn; + dc->props = isa_cirrus_vga_properties; + set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories); ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo isa_cirrus_vga_info = { +diff --git a/hw/i386/pc.c b/hw/i386/pc.c +index d36bac8..fdad4bb 100644 +--- a/hw/i386/pc.c ++++ b/hw/i386/pc.c +@@ -1525,7 +1525,9 @@ static void pc_superio_init(ISABus *isa_bus, bool create_fdctrl, bool no_vmport) + ISADevice *i8042, *port92, *vmmouse; + + serial_hds_isa_init(isa_bus, 0, MAX_SERIAL_PORTS); ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + parallel_hds_isa_init(isa_bus, MAX_PARALLEL_PORTS); ++#endif + + for (i = 0; i < MAX_FD; i++) { + fd[i] = drive_get(IF_FLOPPY, 0, i); +diff --git a/hw/ide/piix.c b/hw/ide/piix.c +index a3afe1f..6de12ca 100644 +--- a/hw/ide/piix.c ++++ b/hw/ide/piix.c +@@ -253,7 +253,8 @@ static void piix3_ide_class_init(ObjectClass *klass, void *data) + k->device_id = PCI_DEVICE_ID_INTEL_82371SB_1; + k->class_id = PCI_CLASS_STORAGE_IDE; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); +- dc->hotpluggable = false; ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo piix3_ide_info = { +@@ -280,6 +281,8 @@ static void piix4_ide_class_init(ObjectClass *klass, void *data) + k->class_id = PCI_CLASS_STORAGE_IDE; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + dc->hotpluggable = false; ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo piix4_ide_info = { +diff --git a/hw/ide/via.c b/hw/ide/via.c +index 117ac4d..b1bafe6 100644 +--- a/hw/ide/via.c ++++ b/hw/ide/via.c +@@ -217,6 +217,8 @@ static void via_ide_class_init(ObjectClass *klass, void *data) + k->revision = 0x06; + k->class_id = PCI_CLASS_STORAGE_IDE; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo via_ide_info = { +diff --git a/hw/input/pckbd.c b/hw/input/pckbd.c +index f17f18e..66adb83 100644 +--- a/hw/input/pckbd.c ++++ b/hw/input/pckbd.c +@@ -570,6 +570,8 @@ static void i8042_class_initfn(ObjectClass *klass, void *data) + + dc->realize = i8042_realizefn; + dc->vmsd = &vmstate_kbd_isa; ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo i8042_info = { +diff --git a/hw/misc/Makefile.objs b/hw/misc/Makefile.objs +index 00e834d..d7430cf 100644 +--- a/hw/misc/Makefile.objs ++++ b/hw/misc/Makefile.objs +@@ -8,7 +8,7 @@ common-obj-$(CONFIG_ISA_TESTDEV) += pc-testdev.o + common-obj-$(CONFIG_PCI_TESTDEV) += pci-testdev.o + common-obj-$(CONFIG_EDU) += edu.o + +-common-obj-y += unimp.o ++#common-obj-y += unimp.o + common-obj-$(CONFIG_FW_CFG_DMA) += vmcoreinfo.o + + # ARM devices +diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c +index 16f0370..bfbfc0e 100644 +--- a/hw/misc/ivshmem.c ++++ b/hw/misc/ivshmem.c +@@ -892,6 +892,13 @@ static void ivshmem_common_realize(PCIDevice *dev, Error **errp) + return; + } + ++ /* Migration disabled for Red Hat Enterprise Linux: */ ++ if (s->master == ON_OFF_AUTO_ON) { ++ error_setg(errp, "master=on is not supported"); ++ return; ++ } ++ s->master = ON_OFF_AUTO_OFF; ++ + pci_conf = dev->config; + pci_conf[PCI_COMMAND] = PCI_COMMAND_IO | PCI_COMMAND_MEMORY; + +@@ -1183,6 +1190,8 @@ static void ivshmem_doorbell_class_init(ObjectClass *klass, void *data) + k->realize = ivshmem_doorbell_realize; + dc->props = ivshmem_doorbell_properties; + dc->vmsd = &ivshmem_doorbell_vmsd; ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo ivshmem_doorbell_info = { +@@ -1352,6 +1361,8 @@ static void ivshmem_class_init(ObjectClass *klass, void *data) + dc->desc = "Inter-VM shared memory (legacy)"; + dc->props = ivshmem_properties; + dc->vmsd = &ivshmem_vmsd; ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo ivshmem_info = { +diff --git a/hw/net/e1000.c b/hw/net/e1000.c +index 13a9494..742cd0a 100644 +--- a/hw/net/e1000.c ++++ b/hw/net/e1000.c +@@ -1768,6 +1768,7 @@ static const E1000Info e1000_devices[] = { + .revision = 0x03, + .phy_id2 = E1000_PHY_ID2_8254xx_DEFAULT, + }, ++#if 0 /* Disabled for Red Hat Enterprise Linux 7 */ + { + .name = "e1000-82544gc", + .device_id = E1000_DEV_ID_82544GC_COPPER, +@@ -1780,6 +1781,7 @@ static const E1000Info e1000_devices[] = { + .revision = 0x03, + .phy_id2 = E1000_PHY_ID2_8254xx_DEFAULT, + }, ++#endif + }; + + static void e1000_register_types(void) +diff --git a/hw/pci-host/piix.c b/hw/pci-host/piix.c +index 0e60834..3ce4b14 100644 +--- a/hw/pci-host/piix.c ++++ b/hw/pci-host/piix.c +@@ -787,6 +787,7 @@ static const TypeInfo i440fx_info = { + }, + }; + ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + /* IGD Passthrough Host Bridge. */ + typedef struct { + uint8_t offset; +@@ -870,6 +871,7 @@ static const TypeInfo igd_passthrough_i440fx_info = { + .instance_size = sizeof(PCII440FXState), + .class_init = igd_passthrough_i440fx_class_init, + }; ++#endif + + static const char *i440fx_pcihost_root_bus_path(PCIHostState *host_bridge, + PCIBus *rootbus) +@@ -915,7 +917,9 @@ static const TypeInfo i440fx_pcihost_info = { + static void i440fx_register_types(void) + { + type_register_static(&i440fx_info); ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + type_register_static(&igd_passthrough_i440fx_info); ++#endif + type_register_static(&piix3_pci_type_info); + type_register_static(&piix3_info); + type_register_static(&piix3_xen_info); +diff --git a/hw/ppc/Makefile.objs b/hw/ppc/Makefile.objs +index 86d82a6..a46a989 100644 +--- a/hw/ppc/Makefile.objs ++++ b/hw/ppc/Makefile.objs +@@ -3,7 +3,7 @@ obj-y += ppc.o ppc_booke.o fdt.o + # IBM pSeries (sPAPR) + obj-$(CONFIG_PSERIES) += spapr.o spapr_caps.o spapr_vio.o spapr_events.o + obj-$(CONFIG_PSERIES) += spapr_hcall.o spapr_iommu.o spapr_rtas.o +-obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o spapr_rng.o ++obj-$(CONFIG_PSERIES) += spapr_pci.o spapr_rtc.o spapr_drc.o + obj-$(CONFIG_PSERIES) += spapr_cpu_core.o spapr_ovec.o + # IBM PowerNV + obj-$(CONFIG_POWERNV) += pnv.o pnv_xscom.o pnv_core.o pnv_lpc.o pnv_psi.o pnv_occ.o pnv_bmc.o +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index a81570e..6a92b20 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -1153,6 +1153,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr, + /* /vdevice */ + spapr_dt_vdevice(spapr->vio_bus, fdt); + ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + if (object_resolve_path_type("", TYPE_SPAPR_RNG, NULL)) { + ret = spapr_rng_populate_dt(fdt); + if (ret < 0) { +@@ -1160,7 +1161,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr, + exit(1); + } + } +- ++#endif + QLIST_FOREACH(phb, &spapr->phbs, list) { + ret = spapr_populate_pci_dt(phb, PHANDLE_XICP, fdt); + if (ret < 0) { +diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c +index 94afeb3..1eda854 100644 +--- a/hw/ppc/spapr_cpu_core.c ++++ b/hw/ppc/spapr_cpu_core.c +@@ -240,10 +240,12 @@ static const TypeInfo spapr_cpu_core_type_infos[] = { + .instance_size = sizeof(sPAPRCPUCore), + .class_size = sizeof(sPAPRCPUCoreClass), + }, ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + DEFINE_SPAPR_CPU_CORE_TYPE("970_v2.2"), + DEFINE_SPAPR_CPU_CORE_TYPE("970mp_v1.0"), + DEFINE_SPAPR_CPU_CORE_TYPE("970mp_v1.1"), + DEFINE_SPAPR_CPU_CORE_TYPE("power5+_v2.1"), ++#endif + DEFINE_SPAPR_CPU_CORE_TYPE("power7_v2.3"), + DEFINE_SPAPR_CPU_CORE_TYPE("power7+_v2.1"), + DEFINE_SPAPR_CPU_CORE_TYPE("power8_v2.0"), +diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c +index e51fbef..8720e46 100644 +--- a/hw/s390x/virtio-ccw.c ++++ b/hw/s390x/virtio-ccw.c +@@ -927,6 +927,8 @@ static void virtio_ccw_rng_realize(VirtioCcwDevice *ccw_dev, Error **errp) + NULL); + } + ++#if 0 /* Disabled in Red Hat Enterprise Linux */ ++ + static void virtio_ccw_crypto_realize(VirtioCcwDevice *ccw_dev, Error **errp) + { + VirtIOCryptoCcw *dev = VIRTIO_CRYPTO_CCW(ccw_dev); +@@ -944,6 +946,7 @@ static void virtio_ccw_crypto_realize(VirtioCcwDevice *ccw_dev, Error **errp) + OBJECT(dev->vdev.conf.cryptodev), "cryptodev", + NULL); + } ++#endif + + static void virtio_ccw_gpu_realize(VirtioCcwDevice *ccw_dev, Error **errp) + { +@@ -1534,6 +1537,8 @@ static const TypeInfo virtio_ccw_rng = { + .class_init = virtio_ccw_rng_class_init, + }; + ++#if 0 /* Disabled in Red Hat Enterprise Linux */ ++ + static Property virtio_ccw_crypto_properties[] = { + DEFINE_PROP_BIT("ioeventfd", VirtioCcwDevice, flags, + VIRTIO_CCW_FLAG_USE_IOEVENTFD_BIT, true), +@@ -1571,6 +1576,7 @@ static const TypeInfo virtio_ccw_crypto = { + .instance_init = virtio_ccw_crypto_instance_init, + .class_init = virtio_ccw_crypto_class_init, + }; ++#endif + + static Property virtio_ccw_gpu_properties[] = { + DEFINE_PROP_BIT("ioeventfd", VirtioCcwDevice, flags, +@@ -1895,7 +1901,9 @@ static void virtio_ccw_register(void) + #ifdef CONFIG_VHOST_VSOCK + type_register_static(&vhost_vsock_ccw_info); + #endif ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + type_register_static(&virtio_ccw_crypto); ++#endif + type_register_static(&virtio_ccw_gpu); + type_register_static(&virtio_ccw_input); + type_register_static(&virtio_ccw_input_hid); +diff --git a/hw/usb/ccid-card-emulated.c b/hw/usb/ccid-card-emulated.c +index ea42e46..f2d8b0d 100644 +--- a/hw/usb/ccid-card-emulated.c ++++ b/hw/usb/ccid-card-emulated.c +@@ -588,6 +588,8 @@ static void emulated_class_initfn(ObjectClass *klass, void *data) + set_bit(DEVICE_CATEGORY_INPUT, dc->categories); + dc->desc = "emulated smartcard"; + dc->props = emulated_card_properties; ++ /* Disabled for Red Hat Enterprise Linux: */ ++ dc->user_creatable = false; + } + + static const TypeInfo emulated_card_info = { +diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs +index a2e7a0a..b25ca64 100644 +--- a/hw/vfio/Makefile.objs ++++ b/hw/vfio/Makefile.objs +@@ -2,8 +2,5 @@ ifeq ($(CONFIG_LINUX), y) + obj-$(CONFIG_SOFTMMU) += common.o + obj-$(CONFIG_PCI) += pci.o pci-quirks.o display.o + obj-$(CONFIG_VFIO_CCW) += ccw.o +-obj-$(CONFIG_SOFTMMU) += platform.o +-obj-$(CONFIG_VFIO_XGMAC) += calxeda-xgmac.o +-obj-$(CONFIG_VFIO_AMD_XGBE) += amd-xgbe.o + obj-$(CONFIG_SOFTMMU) += spapr.o + endif +diff --git a/hw/vfio/pci-quirks.c b/hw/vfio/pci-quirks.c +index e5779a7..cb2f79c 100644 +--- a/hw/vfio/pci-quirks.c ++++ b/hw/vfio/pci-quirks.c +@@ -1193,6 +1193,8 @@ static void vfio_pci_igd_lpc_bridge_class_init(ObjectClass *klass, void *data) + set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); + dc->desc = "VFIO dummy ISA/LPC bridge for IGD assignment"; + dc->hotpluggable = false; ++ /* Disabled in Red Hat Enterprise Linux */ ++ dc->user_creatable = false; + k->realize = vfio_pci_igd_lpc_bridge_realize; + k->class_id = PCI_CLASS_BRIDGE_ISA; + } +@@ -1386,6 +1388,9 @@ static void vfio_probe_igd_bar4_quirk(VFIOPCIDevice *vdev, int nr) + 0, PCI_DEVFN(0x2, 0))) { + return; + } ++ ++ /* Disabled in Red Hat Enterprise Linux */ ++ return; + + /* + * We need to create an LPC/ISA bridge at PCI bus address 00:1f.0 that we +diff --git a/hw/virtio/Makefile.objs b/hw/virtio/Makefile.objs +index 765d363..a5a0936 100644 +--- a/hw/virtio/Makefile.objs ++++ b/hw/virtio/Makefile.objs +@@ -7,8 +7,9 @@ common-obj-y += virtio-mmio.o + obj-y += virtio.o virtio-balloon.o + obj-$(CONFIG_LINUX) += vhost.o vhost-backend.o vhost-user.o + obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock.o +-obj-y += virtio-crypto.o +-obj-$(CONFIG_VIRTIO_PCI) += virtio-crypto-pci.o ++# Disabled in Red Hat Enterprise Linux ++#obj-y += virtio-crypto.o ++#obj-$(CONFIG_VIRTIO_PCI) += virtio-crypto-pci.o + endif + + common-obj-$(call lnot,$(CONFIG_LINUX)) += vhost-stub.o +diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c +index 1e8ab7b..92bdc9e 100644 +--- a/hw/virtio/virtio-pci.c ++++ b/hw/virtio/virtio-pci.c +@@ -1983,7 +1983,7 @@ static const TypeInfo virtio_blk_pci_info = { + .class_init = virtio_blk_pci_class_init, + }; + +-#if defined(CONFIG_VHOST_USER) && defined(CONFIG_LINUX) ++#if defined(CONFIG_VHOST_USER_BLK) + /* vhost-user-blk */ + + static Property vhost_user_blk_pci_properties[] = { +@@ -2163,7 +2163,7 @@ static const TypeInfo vhost_scsi_pci_info = { + }; + #endif + +-#if defined(CONFIG_VHOST_USER) && defined(CONFIG_LINUX) ++#if defined(CONFIG_VHOST_USER_BLK) + /* vhost-user-scsi-pci */ + static Property vhost_user_scsi_pci_properties[] = { + DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, +@@ -2686,7 +2686,7 @@ static void virtio_pci_register_types(void) + type_register_static(&virtio_9p_pci_info); + #endif + type_register_static(&virtio_blk_pci_info); +-#if defined(CONFIG_VHOST_USER) && defined(CONFIG_LINUX) ++#if defined(CONFIG_VHOST_USER_BLK) + type_register_static(&vhost_user_blk_pci_info); + #endif + type_register_static(&virtio_scsi_pci_info); +@@ -2696,7 +2696,7 @@ static void virtio_pci_register_types(void) + #ifdef CONFIG_VHOST_SCSI + type_register_static(&vhost_scsi_pci_info); + #endif +-#if defined(CONFIG_VHOST_USER) && defined(CONFIG_LINUX) ++#if defined(CONFIG_VHOST_USER_SCSI) + type_register_static(&vhost_user_scsi_pci_info); + #endif + #ifdef CONFIG_VHOST_VSOCK +diff --git a/qemu-options.hx b/qemu-options.hx +index ca4e412..2042dba 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -1811,11 +1811,6 @@ ETEXI + + DEF("no-hpet", 0, QEMU_OPTION_no_hpet, + "-no-hpet disable HPET\n", QEMU_ARCH_I386) +-STEXI +-@item -no-hpet +-@findex -no-hpet +-Disable HPET support. +-ETEXI + + DEF("acpitable", HAS_ARG, QEMU_OPTION_acpitable, + "-acpitable [sig=str][,rev=n][,oem_id=str][,oem_table_id=str][,oem_rev=n][,asl_compiler_id=str][,asl_compiler_rev=n][,{data|file}=file1[:file2]...]\n" +diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs +index 2d59d84..dfdfca7 100644 +--- a/stubs/Makefile.objs ++++ b/stubs/Makefile.objs +@@ -43,3 +43,4 @@ stub-obj-y += xen-common.o + stub-obj-y += xen-hvm.o + stub-obj-y += pci-host-piix.o + stub-obj-y += ram-block.o ++stub-obj-y += ide-isa.o +diff --git a/stubs/ide-isa.c b/stubs/ide-isa.c +new file mode 100644 +index 0000000..9fd50ef +--- /dev/null ++++ b/stubs/ide-isa.c +@@ -0,0 +1,13 @@ ++#include "qemu/osdep.h" ++#include "hw/ide.h" ++#include ++ ++ISADevice *isa_ide_init(ISABus *bus, int iobase, int iobase2, int isairq, ++ DriveInfo *hd0, DriveInfo *hd1) ++{ ++ /* ++ * In theory the real isa_ide_init() function can return NULL, but no ++ * caller actually checks for that. Make sure we go out with a clear bang. ++ */ ++ abort(); ++} +diff --git a/target/arm/cpu.c b/target/arm/cpu.c +index 022d8c5..4255e9c 100644 +--- a/target/arm/cpu.c ++++ b/target/arm/cpu.c +@@ -1953,7 +1953,9 @@ static void arm_cpu_register_types(void) + type_register_static(&idau_interface_type_info); + + while (info->name) { +- cpu_register(info); ++ /* RHEL specific: Filter out unsupported cpu models */ ++ if (!strcmp(info->name, "cortex-a15")) ++ cpu_register(info); + info++; + } + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index a20fe26..f483a71 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -825,14 +825,14 @@ static X86CPUDefinition builtin_x86_defs[] = { + .family = 6, + .model = 6, + .stepping = 3, +- .features[FEAT_1_EDX] = +- PPRO_FEATURES | +- CPUID_MTRR | CPUID_CLFLUSH | CPUID_MCA | +- CPUID_PSE36, +- .features[FEAT_1_ECX] = +- CPUID_EXT_SSE3 | CPUID_EXT_CX16, +- .features[FEAT_8000_0001_EDX] = +- CPUID_EXT2_LM | CPUID_EXT2_SYSCALL | CPUID_EXT2_NX, ++ .features[FEAT_1_EDX] = CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | ++ CPUID_MMX | CPUID_CLFLUSH | CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | ++ CPUID_MCA | CPUID_PGE | CPUID_MTRR | CPUID_SEP | CPUID_APIC | ++ CPUID_CX8 | CPUID_MCE | CPUID_PAE | CPUID_MSR | CPUID_TSC | ++ CPUID_PSE | CPUID_DE | CPUID_FP87, ++ .features[FEAT_1_ECX] = CPUID_EXT_CX16 | CPUID_EXT_SSE3, ++ .features[FEAT_8000_0001_EDX] = CPUID_EXT2_LM | CPUID_EXT2_NX | ++ CPUID_EXT2_SYSCALL, + .features[FEAT_8000_0001_ECX] = + CPUID_EXT3_LAHF_LM | CPUID_EXT3_SVM, + .xlevel = 0x8000000A, +@@ -1062,6 +1062,25 @@ static X86CPUDefinition builtin_x86_defs[] = { + .model_id = "Intel(R) Atom(TM) CPU N270 @ 1.60GHz", + }, + { ++ .name = "cpu64-rhel6", ++ .level = 4, ++ .vendor = CPUID_VENDOR_AMD, ++ .family = 6, ++ .model = 13, ++ .stepping = 3, ++ .features[FEAT_1_EDX] = CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | ++ CPUID_MMX | CPUID_CLFLUSH | CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | ++ CPUID_MCA | CPUID_PGE | CPUID_MTRR | CPUID_SEP | CPUID_APIC | ++ CPUID_CX8 | CPUID_MCE | CPUID_PAE | CPUID_MSR | CPUID_TSC | ++ CPUID_PSE | CPUID_DE | CPUID_FP87, ++ .features[FEAT_1_ECX] = CPUID_EXT_CX16 | CPUID_EXT_SSE3, ++ .features[FEAT_8000_0001_EDX] = CPUID_EXT2_LM | CPUID_EXT2_NX | CPUID_EXT2_SYSCALL, ++ .features[FEAT_8000_0001_ECX] = CPUID_EXT3_SSE4A | CPUID_EXT3_ABM | ++ CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM, ++ .xlevel = 0x8000000A, ++ .model_id = "QEMU Virtual CPU version (cpu64-rhel6)", ++ }, ++ { + .name = "Conroe", + .level = 10, + .vendor = CPUID_VENDOR_INTEL, +@@ -4764,11 +4783,13 @@ static Property x86_cpu_properties[] = { + DEFINE_PROP_BOOL("hv-vapic", X86CPU, hyperv_vapic, false), + DEFINE_PROP_BOOL("hv-time", X86CPU, hyperv_time, false), + DEFINE_PROP_BOOL("hv-crash", X86CPU, hyperv_crash, false), ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + DEFINE_PROP_BOOL("hv-reset", X86CPU, hyperv_reset, false), + DEFINE_PROP_BOOL("hv-vpindex", X86CPU, hyperv_vpindex, false), + DEFINE_PROP_BOOL("hv-runtime", X86CPU, hyperv_runtime, false), + DEFINE_PROP_BOOL("hv-synic", X86CPU, hyperv_synic, false), + DEFINE_PROP_BOOL("hv-stimer", X86CPU, hyperv_stimer, false), ++#endif + DEFINE_PROP_BOOL("hv-frequencies", X86CPU, hyperv_frequencies, false), + DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true), + DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false), +diff --git a/target/ppc/cpu-models.c b/target/ppc/cpu-models.c +index 6c9bfde..77cb298 100644 +--- a/target/ppc/cpu-models.c ++++ b/target/ppc/cpu-models.c +@@ -65,6 +65,7 @@ + #define POWERPC_DEF(_name, _pvr, _type, _desc) \ + POWERPC_DEF_SVR(_name, _desc, _pvr, POWERPC_SVR_NONE, _type) + ++#if 0 /* Embedded and 32-bit CPUs disabled for Red Hat Enterprise Linux */ + /* Embedded PowerPC */ + /* PowerPC 401 family */ + POWERPC_DEF("401", CPU_POWERPC_401, 401, +@@ -739,10 +740,13 @@ + "PowerPC 7447A v1.2 (G4)") + POWERPC_DEF("7457a_v1.2", CPU_POWERPC_74x7A_v12, 7455, + "PowerPC 7457A v1.2 (G4)") ++#endif + /* 64 bits PowerPC */ + #if defined (TARGET_PPC64) ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + POWERPC_DEF("power5+_v2.1", CPU_POWERPC_POWER5P_v21, POWER5P, + "POWER5+ v2.1") ++#endif + POWERPC_DEF("power7_v2.3", CPU_POWERPC_POWER7_v23, POWER7, + "POWER7 v2.3") + POWERPC_DEF("power7+_v2.1", CPU_POWERPC_POWER7P_v21, POWER7, +@@ -753,14 +757,17 @@ + "POWER8 v2.0") + POWERPC_DEF("power8nvl_v1.0", CPU_POWERPC_POWER8NVL_v10, POWER8, + "POWER8NVL v1.0") ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + POWERPC_DEF("970_v2.2", CPU_POWERPC_970_v22, 970, + "PowerPC 970 v2.2") ++#endif + + POWERPC_DEF("power9_v1.0", CPU_POWERPC_POWER9_DD1, POWER9, + "POWER9 v1.0") + POWERPC_DEF("power9_v2.0", CPU_POWERPC_POWER9_DD20, POWER9, + "POWER9 v2.0") + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + POWERPC_DEF("970fx_v1.0", CPU_POWERPC_970FX_v10, 970, + "PowerPC 970FX v1.0 (G5)") + POWERPC_DEF("970fx_v2.0", CPU_POWERPC_970FX_v20, 970, +@@ -775,12 +782,14 @@ + "PowerPC 970MP v1.0") + POWERPC_DEF("970mp_v1.1", CPU_POWERPC_970MP_v11, 970, + "PowerPC 970MP v1.1") ++#endif + #endif /* defined (TARGET_PPC64) */ + + /***************************************************************************/ + /* PowerPC CPU aliases */ + + PowerPCCPUAlias ppc_cpu_aliases[] = { ++#if 0 /* Embedded and 32-bit CPUs disabled for Red Hat Enterprise Linux */ + { "403", "403gc" }, + { "405", "405d4" }, + { "405cr", "405crc" }, +@@ -939,20 +948,25 @@ PowerPCCPUAlias ppc_cpu_aliases[] = { + { "7447a", "7447a_v1.2" }, + { "7457a", "7457a_v1.2" }, + { "apollo7pm", "7457a_v1.0" }, ++#endif + #if defined(TARGET_PPC64) ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + { "power5+", "power5+_v2.1" }, + { "power5gs", "power5+_v2.1" }, ++#endif + { "power7", "power7_v2.3" }, + { "power7+", "power7+_v2.1" }, + { "power8e", "power8e_v2.1" }, + { "power8", "power8_v2.0" }, + { "power8nvl", "power8nvl_v1.0" }, + { "power9", "power9_v2.0" }, ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + { "970", "970_v2.2" }, + { "970fx", "970fx_v3.1" }, + { "970mp", "970mp_v1.1" }, + #endif +- ++#endif ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + /* Generic PowerPCs */ + #if defined(TARGET_PPC64) + { "ppc64", "970fx_v3.1" }, +@@ -960,5 +974,6 @@ PowerPCCPUAlias ppc_cpu_aliases[] = { + { "ppc32", "604" }, + { "ppc", "604" }, + { "default", "604" }, ++#endif + { NULL, NULL } + }; +diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c +index 2741b68..c4016e0 100644 +--- a/target/s390x/cpu_models.c ++++ b/target/s390x/cpu_models.c +@@ -372,6 +372,9 @@ static void check_unavailable_features(const S390CPUModel *max_model, + (max_model->def->gen == model->def->gen && + max_model->def->ec_ga < model->def->ec_ga)) { + list_add_feat("type", unavailable); ++ } else if (model->def->gen < 11 && kvm_enabled()) { ++ /* Older CPU models are not supported on Red Hat Enterprise Linux */ ++ list_add_feat("type", unavailable); + } + + /* detect missing features if any to properly report them */ +diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c +index fb59d92..fbccceb 100644 +--- a/target/s390x/kvm.c ++++ b/target/s390x/kvm.c +@@ -2285,6 +2285,14 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, Error **errp) + error_setg(errp, "KVM doesn't support CPU models"); + return; + } ++ ++ /* Older CPU models are not supported on Red Hat Enterprise Linux */ ++ if (model->def->gen < 11) { ++ error_setg(errp, "KVM: Unsupported CPU type specified: %s", ++ MACHINE(qdev_get_machine())->cpu_type); ++ return; ++ } ++ + prop.cpuid = s390_cpuid_from_cpu_model(model); + prop.ibc = s390_ibc_from_cpu_model(model); + /* configure cpu features indicated via STFL(e) */ +diff --git a/tests/Makefile.include b/tests/Makefile.include +index 3b9a5e3..3fd4706 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -181,8 +181,8 @@ check-qtest-generic-y += tests/device-introspect-test$(EXESUF) + gcov-files-generic-y = qdev-monitor.c qmp.c + + gcov-files-ipack-y += hw/ipack/ipack.c +-check-qtest-ipack-y += tests/ipoctal232-test$(EXESUF) +-gcov-files-ipack-y += hw/char/ipoctal232.c ++#check-qtest-ipack-y += tests/ipoctal232-test$(EXESUF) ++#gcov-files-ipack-y += hw/char/ipoctal232.c + + check-qtest-virtioserial-y += tests/virtio-console-test$(EXESUF) + gcov-files-virtioserial-y += hw/char/virtio-console.c +@@ -214,23 +214,23 @@ check-qtest-pci-y += tests/e1000e-test$(EXESUF) + gcov-files-pci-y += hw/net/e1000e.c hw/net/e1000e_core.c + check-qtest-pci-y += tests/rtl8139-test$(EXESUF) + gcov-files-pci-y += hw/net/rtl8139.c +-check-qtest-pci-y += tests/pcnet-test$(EXESUF) +-gcov-files-pci-y += hw/net/pcnet.c +-gcov-files-pci-y += hw/net/pcnet-pci.c +-check-qtest-pci-y += tests/eepro100-test$(EXESUF) +-gcov-files-pci-y += hw/net/eepro100.c +-check-qtest-pci-y += tests/ne2000-test$(EXESUF) +-gcov-files-pci-y += hw/net/ne2000.c +-check-qtest-pci-y += tests/nvme-test$(EXESUF) +-gcov-files-pci-y += hw/block/nvme.c ++#check-qtest-pci-y += tests/pcnet-test$(EXESUF) ++#gcov-files-pci-y += hw/net/pcnet.c ++#gcov-files-pci-y += hw/net/pcnet-pci.c ++#check-qtest-pci-y += tests/eepro100-test$(EXESUF) ++#gcov-files-pci-y += hw/net/eepro100.c ++#check-qtest-pci-y += tests/ne2000-test$(EXESUF) ++#gcov-files-pci-y += hw/net/ne2000.c ++#check-qtest-pci-y += tests/nvme-test$(EXESUF) ++#gcov-files-pci-y += hw/block/nvme.c + check-qtest-pci-y += tests/ac97-test$(EXESUF) + gcov-files-pci-y += hw/audio/ac97.c +-check-qtest-pci-y += tests/es1370-test$(EXESUF) +-gcov-files-pci-y += hw/audio/es1370.c ++#check-qtest-pci-y += tests/es1370-test$(EXESUF) ++#gcov-files-pci-y += hw/audio/es1370.c + check-qtest-pci-y += $(check-qtest-virtio-y) + gcov-files-pci-y += $(gcov-files-virtio-y) hw/virtio/virtio-pci.c +-check-qtest-pci-y += tests/tpci200-test$(EXESUF) +-gcov-files-pci-y += hw/ipack/tpci200.c ++#check-qtest-pci-y += tests/tpci200-test$(EXESUF) ++#gcov-files-pci-y += hw/ipack/tpci200.c + check-qtest-pci-y += $(check-qtest-ipack-y) + gcov-files-pci-y += $(gcov-files-ipack-y) + check-qtest-pci-y += tests/display-vga-test$(EXESUF) +@@ -244,23 +244,23 @@ check-qtest-pci-y += tests/intel-hda-test$(EXESUF) + gcov-files-pci-y += hw/audio/intel-hda.c hw/audio/hda-codec.c + check-qtest-pci-$(CONFIG_IVSHMEM) += tests/ivshmem-test$(EXESUF) + gcov-files-pci-y += hw/misc/ivshmem.c +-check-qtest-pci-y += tests/megasas-test$(EXESUF) +-gcov-files-pci-y += hw/scsi/megasas.c ++#check-qtest-pci-y += tests/megasas-test$(EXESUF) ++#gcov-files-pci-y += hw/scsi/megasas.c + + check-qtest-i386-y = tests/endianness-test$(EXESUF) +-check-qtest-i386-y += tests/fdc-test$(EXESUF) +-gcov-files-i386-y = hw/block/fdc.c ++#check-qtest-i386-y += tests/fdc-test$(EXESUF) ++#gcov-files-i386-y = hw/block/fdc.c + check-qtest-i386-y += tests/ide-test$(EXESUF) + check-qtest-i386-y += tests/ahci-test$(EXESUF) + check-qtest-i386-y += tests/hd-geo-test$(EXESUF) + gcov-files-i386-y += hw/block/hd-geometry.c + check-qtest-i386-y += tests/boot-order-test$(EXESUF) +-check-qtest-i386-y += tests/bios-tables-test$(EXESUF) ++#check-qtest-i386-y += tests/bios-tables-test$(EXESUF) + check-qtest-i386-y += tests/boot-serial-test$(EXESUF) + check-qtest-i386-$(CONFIG_SLIRP) += tests/pxe-test$(EXESUF) + check-qtest-i386-y += tests/rtc-test$(EXESUF) +-check-qtest-i386-y += tests/ipmi-kcs-test$(EXESUF) +-check-qtest-i386-y += tests/ipmi-bt-test$(EXESUF) ++#check-qtest-i386-y += tests/ipmi-kcs-test$(EXESUF) ++#check-qtest-i386-y += tests/ipmi-bt-test$(EXESUF) + check-qtest-i386-y += tests/i440fx-test$(EXESUF) + check-qtest-i386-y += tests/fw_cfg-test$(EXESUF) + check-qtest-i386-y += tests/drive_del-test$(EXESUF) +@@ -269,8 +269,8 @@ check-qtest-i386-y += tests/tco-test$(EXESUF) + gcov-files-i386-y += hw/watchdog/watchdog.c hw/watchdog/wdt_ib700.c + check-qtest-i386-y += $(check-qtest-pci-y) + gcov-files-i386-y += $(gcov-files-pci-y) +-check-qtest-i386-y += tests/vmxnet3-test$(EXESUF) +-gcov-files-i386-y += hw/net/vmxnet3.c ++#check-qtest-i386-y += tests/vmxnet3-test$(EXESUF) ++#gcov-files-i386-y += hw/net/vmxnet3.c + gcov-files-i386-y += hw/net/net_rx_pkt.c + gcov-files-i386-y += hw/net/net_tx_pkt.c + check-qtest-i386-y += tests/pvpanic-test$(EXESUF) +@@ -279,8 +279,8 @@ check-qtest-i386-y += tests/i82801b11-test$(EXESUF) + gcov-files-i386-y += hw/pci-bridge/i82801b11.c + check-qtest-i386-y += tests/ioh3420-test$(EXESUF) + gcov-files-i386-y += hw/pci-bridge/ioh3420.c +-check-qtest-i386-y += tests/usb-hcd-ohci-test$(EXESUF) +-gcov-files-i386-y += hw/usb/hcd-ohci.c ++#check-qtest-i386-y += tests/usb-hcd-ohci-test$(EXESUF) ++#gcov-files-i386-y += hw/usb/hcd-ohci.c + check-qtest-i386-y += tests/usb-hcd-uhci-test$(EXESUF) + gcov-files-i386-y += hw/usb/hcd-uhci.c + check-qtest-i386-y += tests/usb-hcd-ehci-test$(EXESUF) +@@ -306,7 +306,7 @@ check-qtest-i386-y += tests/migration-test$(EXESUF) + check-qtest-i386-y += tests/test-x86-cpuid-compat$(EXESUF) + check-qtest-i386-y += tests/numa-test$(EXESUF) + check-qtest-x86_64-y += $(check-qtest-i386-y) +-check-qtest-x86_64-y += tests/sdhci-test$(EXESUF) ++#check-qtest-x86_64-y += tests/sdhci-test$(EXESUF) + gcov-files-i386-y += i386-softmmu/hw/timer/mc146818rtc.c + gcov-files-x86_64-y = $(subst i386-softmmu/,x86_64-softmmu/,$(gcov-files-i386-y)) + +@@ -328,7 +328,7 @@ check-qtest-moxie-y = tests/boot-serial-test$(EXESUF) + + check-qtest-ppc-y = tests/endianness-test$(EXESUF) + check-qtest-ppc-y += tests/boot-order-test$(EXESUF) +-check-qtest-ppc-y += tests/prom-env-test$(EXESUF) ++#check-qtest-ppc-y += tests/prom-env-test$(EXESUF) + check-qtest-ppc-y += tests/drive_del-test$(EXESUF) + check-qtest-ppc-y += tests/boot-serial-test$(EXESUF) + check-qtest-ppc-y += tests/m48t59-test$(EXESUF) +@@ -342,19 +342,19 @@ check-qtest-ppc64-y += tests/pnv-xscom-test$(EXESUF) + check-qtest-ppc64-y += tests/migration-test$(EXESUF) + check-qtest-ppc64-y += tests/rtas-test$(EXESUF) + check-qtest-ppc64-$(CONFIG_SLIRP) += tests/pxe-test$(EXESUF) +-check-qtest-ppc64-y += tests/usb-hcd-ohci-test$(EXESUF) ++#check-qtest-ppc64-y += tests/usb-hcd-ohci-test$(EXESUF) + gcov-files-ppc64-y += hw/usb/hcd-ohci.c +-check-qtest-ppc64-y += tests/usb-hcd-uhci-test$(EXESUF) ++#check-qtest-ppc64-y += tests/usb-hcd-uhci-test$(EXESUF) + gcov-files-ppc64-y += hw/usb/hcd-uhci.c + check-qtest-ppc64-y += tests/usb-hcd-xhci-test$(EXESUF) + gcov-files-ppc64-y += hw/usb/hcd-xhci.c + check-qtest-ppc64-y += $(check-qtest-virtio-y) +-check-qtest-ppc64-$(CONFIG_SLIRP) += tests/test-netfilter$(EXESUF) +-check-qtest-ppc64-$(CONFIG_POSIX) += tests/test-filter-mirror$(EXESUF) +-check-qtest-ppc64-$(CONFIG_POSIX) += tests/test-filter-redirector$(EXESUF) ++#check-qtest-ppc64-$(CONFIG_SLIRP) += tests/test-netfilter$(EXESUF) ++#check-qtest-ppc64-$(CONFIG_POSIX) += tests/test-filter-mirror$(EXESUF) ++#check-qtest-ppc64-$(CONFIG_POSIX) += tests/test-filter-redirector$(EXESUF) + check-qtest-ppc64-y += tests/display-vga-test$(EXESUF) + check-qtest-ppc64-y += tests/numa-test$(EXESUF) +-check-qtest-ppc64-$(CONFIG_IVSHMEM) += tests/ivshmem-test$(EXESUF) ++#check-qtest-ppc64-$(CONFIG_IVSHMEM) += tests/ivshmem-test$(EXESUF) + check-qtest-ppc64-y += tests/cpu-plug-test$(EXESUF) + + check-qtest-sh4-y = tests/endianness-test$(EXESUF) +@@ -379,10 +379,10 @@ gcov-files-arm-y += arm-softmmu/hw/block/virtio-blk.c + check-qtest-arm-y += tests/test-arm-mptimer$(EXESUF) + gcov-files-arm-y += hw/timer/arm_mptimer.c + check-qtest-arm-y += tests/boot-serial-test$(EXESUF) +-check-qtest-arm-y += tests/sdhci-test$(EXESUF) ++#check-qtest-arm-y += tests/sdhci-test$(EXESUF) + +-check-qtest-aarch64-y = tests/numa-test$(EXESUF) +-check-qtest-aarch64-y += tests/sdhci-test$(EXESUF) ++#check-qtest-aarch64-y = tests/numa-test$(EXESUF) ++#check-qtest-aarch64-y += tests/sdhci-test$(EXESUF) + check-qtest-aarch64-y += tests/boot-serial-test$(EXESUF) + + check-qtest-microblazeel-y = $(check-qtest-microblaze-y) +@@ -757,18 +757,18 @@ tests/rtc-test$(EXESUF): tests/rtc-test.o + tests/m48t59-test$(EXESUF): tests/m48t59-test.o + tests/endianness-test$(EXESUF): tests/endianness-test.o + tests/spapr-phb-test$(EXESUF): tests/spapr-phb-test.o $(libqos-obj-y) +-tests/prom-env-test$(EXESUF): tests/prom-env-test.o $(libqos-obj-y) ++#tests/prom-env-test$(EXESUF): tests/prom-env-test.o $(libqos-obj-y) + tests/rtas-test$(EXESUF): tests/rtas-test.o $(libqos-spapr-obj-y) +-tests/fdc-test$(EXESUF): tests/fdc-test.o ++#tests/fdc-test$(EXESUF): tests/fdc-test.o + tests/ide-test$(EXESUF): tests/ide-test.o $(libqos-pc-obj-y) + tests/ahci-test$(EXESUF): tests/ahci-test.o $(libqos-pc-obj-y) +-tests/ipmi-kcs-test$(EXESUF): tests/ipmi-kcs-test.o +-tests/ipmi-bt-test$(EXESUF): tests/ipmi-bt-test.o ++#tests/ipmi-kcs-test$(EXESUF): tests/ipmi-kcs-test.o ++#tests/ipmi-bt-test$(EXESUF): tests/ipmi-bt-test.o + tests/hd-geo-test$(EXESUF): tests/hd-geo-test.o + tests/boot-order-test$(EXESUF): tests/boot-order-test.o $(libqos-obj-y) + tests/boot-serial-test$(EXESUF): tests/boot-serial-test.o $(libqos-obj-y) +-tests/bios-tables-test$(EXESUF): tests/bios-tables-test.o \ +- tests/boot-sector.o tests/acpi-utils.o $(libqos-obj-y) ++#tests/bios-tables-test$(EXESUF): tests/bios-tables-test.o \ ++# tests/boot-sector.o tests/acpi-utils.o $(libqos-obj-y) + tests/pxe-test$(EXESUF): tests/pxe-test.o tests/boot-sector.o $(libqos-obj-y) + tests/tmp105-test$(EXESUF): tests/tmp105-test.o $(libqos-omap-obj-y) + tests/ds1338-test$(EXESUF): tests/ds1338-test.o $(libqos-imx-obj-y) +@@ -779,11 +779,11 @@ tests/fw_cfg-test$(EXESUF): tests/fw_cfg-test.o $(libqos-pc-obj-y) + tests/e1000-test$(EXESUF): tests/e1000-test.o + tests/e1000e-test$(EXESUF): tests/e1000e-test.o $(libqos-pc-obj-y) + tests/rtl8139-test$(EXESUF): tests/rtl8139-test.o $(libqos-pc-obj-y) +-tests/pcnet-test$(EXESUF): tests/pcnet-test.o +-tests/pnv-xscom-test$(EXESUF): tests/pnv-xscom-test.o ++#tests/pcnet-test$(EXESUF): tests/pcnet-test.o ++#tests/pnv-xscom-test$(EXESUF): tests/pnv-xscom-test.o + tests/eepro100-test$(EXESUF): tests/eepro100-test.o +-tests/vmxnet3-test$(EXESUF): tests/vmxnet3-test.o +-tests/ne2000-test$(EXESUF): tests/ne2000-test.o ++#tests/vmxnet3-test$(EXESUF): tests/vmxnet3-test.o ++#tests/ne2000-test$(EXESUF): tests/ne2000-test.o + tests/wdt_ib700-test$(EXESUF): tests/wdt_ib700-test.o + tests/tco-test$(EXESUF): tests/tco-test.o $(libqos-pc-obj-y) + tests/virtio-balloon-test$(EXESUF): tests/virtio-balloon-test.o $(libqos-virtio-obj-y) +@@ -802,14 +802,14 @@ tests/test-hmp$(EXESUF): tests/test-hmp.o + tests/machine-none-test$(EXESUF): tests/machine-none-test.o + tests/drive_del-test$(EXESUF): tests/drive_del-test.o $(libqos-virtio-obj-y) + tests/qdev-monitor-test$(EXESUF): tests/qdev-monitor-test.o $(libqos-pc-obj-y) +-tests/nvme-test$(EXESUF): tests/nvme-test.o ++#tests/nvme-test$(EXESUF): tests/nvme-test.o + tests/pvpanic-test$(EXESUF): tests/pvpanic-test.o + tests/i82801b11-test$(EXESUF): tests/i82801b11-test.o + tests/ac97-test$(EXESUF): tests/ac97-test.o +-tests/es1370-test$(EXESUF): tests/es1370-test.o ++#tests/es1370-test$(EXESUF): tests/es1370-test.o + tests/intel-hda-test$(EXESUF): tests/intel-hda-test.o + tests/ioh3420-test$(EXESUF): tests/ioh3420-test.o +-tests/usb-hcd-ohci-test$(EXESUF): tests/usb-hcd-ohci-test.o $(libqos-usb-obj-y) ++#tests/usb-hcd-ohci-test$(EXESUF): tests/usb-hcd-ohci-test.o $(libqos-usb-obj-y) + tests/usb-hcd-uhci-test$(EXESUF): tests/usb-hcd-uhci-test.o $(libqos-usb-obj-y) + tests/usb-hcd-ehci-test$(EXESUF): tests/usb-hcd-ehci-test.o $(libqos-usb-obj-y) + tests/usb-hcd-xhci-test$(EXESUF): tests/usb-hcd-xhci-test.o $(libqos-usb-obj-y) +@@ -827,14 +827,14 @@ tests/test-filter-mirror$(EXESUF): tests/test-filter-mirror.o $(qtest-obj-y) + tests/test-filter-redirector$(EXESUF): tests/test-filter-redirector.o $(qtest-obj-y) + tests/test-x86-cpuid-compat$(EXESUF): tests/test-x86-cpuid-compat.o $(qtest-obj-y) + tests/ivshmem-test$(EXESUF): tests/ivshmem-test.o contrib/ivshmem-server/ivshmem-server.o $(libqos-pc-obj-y) $(libqos-spapr-obj-y) +-tests/megasas-test$(EXESUF): tests/megasas-test.o $(libqos-spapr-obj-y) $(libqos-pc-obj-y) ++#tests/megasas-test$(EXESUF): tests/megasas-test.o $(libqos-spapr-obj-y) $(libqos-pc-obj-y) + tests/vhost-user-bridge$(EXESUF): tests/vhost-user-bridge.o $(test-util-obj-y) libvhost-user.a + tests/test-uuid$(EXESUF): tests/test-uuid.o $(test-util-obj-y) + tests/test-arm-mptimer$(EXESUF): tests/test-arm-mptimer.o + tests/test-qapi-util$(EXESUF): tests/test-qapi-util.o $(test-util-obj-y) + tests/numa-test$(EXESUF): tests/numa-test.o + tests/vmgenid-test$(EXESUF): tests/vmgenid-test.o tests/boot-sector.o tests/acpi-utils.o +-tests/sdhci-test$(EXESUF): tests/sdhci-test.o $(libqos-pc-obj-y) ++#tests/sdhci-test$(EXESUF): tests/sdhci-test.o $(libqos-pc-obj-y) + + tests/migration/stress$(EXESUF): tests/migration/stress.o + $(call quiet-command, $(LINKPROG) -static -O3 $(PTHREAD_LIB) -o $@ $< ,"LINK","$(TARGET_DIR)$@") +diff --git a/tests/bios-tables-test.c b/tests/bios-tables-test.c +index bf3e193..715f90c 100644 +--- a/tests/bios-tables-test.c ++++ b/tests/bios-tables-test.c +@@ -736,6 +736,7 @@ static void test_acpi_q35_tcg_cphp(void) + free_test_data(&data); + } + ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + static uint8_t ipmi_required_struct_types[] = { + 0, 1, 3, 4, 16, 17, 19, 32, 38, 127 + }; +@@ -772,6 +773,7 @@ static void test_acpi_piix4_tcg_ipmi(void) + &data); + free_test_data(&data); + } ++#endif + + static void test_acpi_q35_tcg_memhp(void) + { +@@ -875,8 +877,10 @@ int main(int argc, char *argv[]) + qtest_add_func("acpi/piix4/bridge", test_acpi_piix4_tcg_bridge); + qtest_add_func("acpi/q35", test_acpi_q35_tcg); + qtest_add_func("acpi/q35/bridge", test_acpi_q35_tcg_bridge); ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + qtest_add_func("acpi/piix4/ipmi", test_acpi_piix4_tcg_ipmi); + qtest_add_func("acpi/q35/ipmi", test_acpi_q35_tcg_ipmi); ++#endif + qtest_add_func("acpi/piix4/cpuhp", test_acpi_piix4_tcg_cphp); + qtest_add_func("acpi/q35/cpuhp", test_acpi_q35_tcg_cphp); + qtest_add_func("acpi/piix4/memhp", test_acpi_piix4_tcg_memhp); +diff --git a/tests/boot-order-test.c b/tests/boot-order-test.c +index e70f5de..1be0731 100644 +--- a/tests/boot-order-test.c ++++ b/tests/boot-order-test.c +@@ -106,6 +106,7 @@ static void test_pc_boot_order(void) + test_boot_orders(NULL, read_boot_order_pc, test_cases_pc); + } + ++#if 0 /* Disabled for RHEL, since CONFIG_MAC and CONFIG_PREP are not enabled */ + static uint8_t read_m48t59(uint64_t addr, uint16_t reg) + { + writeb(addr, reg & 0xff); +@@ -136,6 +137,7 @@ static uint64_t read_boot_order_pmac(void) + + return qfw_cfg_get_u16(fw_cfg, FW_CFG_BOOT_DEVICE); + } ++#endif /* Disabled for RHEL, since CONFIG_MAC and CONFIG_PREP are not enabled */ + + static const boot_order_test test_cases_fw_cfg[] = { + { "", 'c', 'c' }, +@@ -145,6 +147,7 @@ static const boot_order_test test_cases_fw_cfg[] = { + {} + }; + ++#if 0 /* Disabled for RHEL, since CONFIG_MAC and CONFIG_PREP are not enabled */ + static void test_pmac_oldworld_boot_order(void) + { + test_boot_orders("g3beige", read_boot_order_pmac, test_cases_fw_cfg); +@@ -153,7 +156,9 @@ static void test_pmac_oldworld_boot_order(void) + static void test_pmac_newworld_boot_order(void) + { + test_boot_orders("mac99", read_boot_order_pmac, test_cases_fw_cfg); ++ + } ++#endif /* Disabled for RHEL, since CONFIG_MAC and CONFIG_PREP are not enabled */ + + static uint64_t read_boot_order_sun4m(void) + { +@@ -188,11 +193,13 @@ int main(int argc, char *argv[]) + if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) { + qtest_add_func("boot-order/pc", test_pc_boot_order); + } else if (strcmp(arch, "ppc") == 0 || strcmp(arch, "ppc64") == 0) { ++#if 0 /* Disabled for RHEL, since CONFIG_MAC and CONFIG_PREP are not enabled */ + qtest_add_func("boot-order/prep", test_prep_boot_order); + qtest_add_func("boot-order/pmac_oldworld", + test_pmac_oldworld_boot_order); + qtest_add_func("boot-order/pmac_newworld", + test_pmac_newworld_boot_order); ++#endif /* Disabled for RHEL, since CONFIG_MAC and CONFIG_PREP are not enabled */ + } else if (strcmp(arch, "sparc") == 0) { + qtest_add_func("boot-order/sun4m", test_sun4m_boot_order); + } else if (strcmp(arch, "sparc64") == 0) { +diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c +index 011525d..dc682c1 100644 +--- a/tests/boot-serial-test.c ++++ b/tests/boot-serial-test.c +@@ -73,7 +73,7 @@ typedef struct testdef { + } testdef_t; + + static testdef_t tests[] = { +- { "alpha", "clipper", "", "PCI:" }, ++/* { "alpha", "clipper", "", "PCI:" }, + { "ppc", "ppce500", "", "U-Boot" }, + { "ppc", "prep", "-m 96", "Memory size: 96 MB" }, + { "ppc", "40p", "-boot d", "Booting from device d" }, +@@ -83,14 +83,14 @@ static testdef_t tests[] = { + { "ppc64", "ppce500", "", "U-Boot" }, + { "ppc64", "prep", "-boot e", "Booting from device e" }, + { "ppc64", "40p", "-m 192", "Memory size: 192 MB" }, +- { "ppc64", "mac99", "", "PowerPC,970FX" }, ++ { "ppc64", "mac99", "", "PowerPC,970FX" },*/ + { "ppc64", "pseries", "", "Open Firmware" }, +- { "ppc64", "powernv", "-cpu POWER8", "OPAL" }, +- { "ppc64", "sam460ex", "-device e1000", "8086 100e" }, ++/* { "ppc64", "powernv", "-cpu POWER8", "OPAL" }, ++ { "ppc64", "sam460ex", "-device e1000", "8086 100e" },*/ + { "i386", "isapc", "-cpu qemu32 -device sga", "SGABIOS" }, + { "i386", "pc", "-device sga", "SGABIOS" }, + { "i386", "q35", "-device sga", "SGABIOS" }, +- { "x86_64", "isapc", "-cpu qemu32 -device sga", "SGABIOS" }, ++ { "x86_64", "pc", "-cpu qemu32 -device sga", "SGABIOS" }, + { "x86_64", "q35", "-device sga", "SGABIOS" }, + { "sparc", "LX", "", "TMS390S10" }, + { "sparc", "SS-4", "", "MB86904" }, +diff --git a/tests/e1000-test.c b/tests/e1000-test.c +index 0c5fcdc..0504d33 100644 +--- a/tests/e1000-test.c ++++ b/tests/e1000-test.c +@@ -29,8 +29,10 @@ static void test_device(gconstpointer data) + static const char *models[] = { + "e1000", + "e1000-82540em", ++#if 0 /* Disabled in Red Hat Enterprise Linux 7 */ + "e1000-82544gc", + "e1000-82545em", ++#endif + }; + + int main(int argc, char **argv) +diff --git a/tests/endianness-test.c b/tests/endianness-test.c +index 546e096..1bd87db 100644 +--- a/tests/endianness-test.c ++++ b/tests/endianness-test.c +@@ -34,6 +34,7 @@ static const TestCase test_cases[] = { + { "mips64", "mips", 0x14000000, .bswap = true }, + { "mips64", "malta", 0x10000000, .bswap = true }, + { "mips64el", "fulong2e", 0x1fd00000 }, ++#if 0 /* Disabled for RHEL, since ISA is not enabled */ + { "ppc", "g3beige", 0xfe000000, .bswap = true, .superio = "i82378" }, + { "ppc", "prep", 0x80000000, .bswap = true }, + { "ppc", "bamboo", 0xe8000000, .bswap = true, .superio = "i82378" }, +@@ -41,6 +42,7 @@ static const TestCase test_cases[] = { + { "ppc64", "pseries", (1ULL << 45), .bswap = true, .superio = "i82378" }, + { "ppc64", "pseries-2.7", 0x10080000000ULL, + .bswap = true, .superio = "i82378" }, ++#endif /* Disabled for RHEL, since ISA is not enabled */ + { "sh4", "r2d", 0xfe240000, .superio = "i82378" }, + { "sh4eb", "r2d", 0xfe240000, .bswap = true, .superio = "i82378" }, + { "sparc64", "sun4u", 0x1fe02000000LL, .bswap = true }, +diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c +index 8af16ee..df2eb50 100644 +--- a/tests/ivshmem-test.c ++++ b/tests/ivshmem-test.c +@@ -257,6 +257,7 @@ static void test_ivshmem_pair(void) + g_free(data); + } + ++#if 0 /* Disabled for Red Hat Enterprise Linux: */ + typedef struct ServerThread { + GThread *thread; + IvshmemServer *server; +@@ -414,9 +415,11 @@ static void test_ivshmem_server_irq(void) + { + test_ivshmem_server(false); + } ++#endif + + #define PCI_SLOT_HP 0x06 + ++#if 0 /* Test uses legacy ivshmem, which is disabled for Red Hat Enterprise Linux: */ + static void test_ivshmem_hotplug(void) + { + const char *arch = qtest_get_arch(); +@@ -434,6 +437,7 @@ static void test_ivshmem_hotplug(void) + qtest_end(); + g_free(opts); + } ++#endif + + static void test_ivshmem_memdev(void) + { +@@ -501,7 +505,7 @@ static gchar *mktempshm(int size, int *fd) + int main(int argc, char **argv) + { + int ret, fd; +- const char *arch = qtest_get_arch(); ++/* const char *arch = qtest_get_arch(); */ + gchar dir[] = "/tmp/ivshmem-test.XXXXXX"; + + #if !GLIB_CHECK_VERSION(2, 31, 0) +@@ -528,14 +532,18 @@ int main(int argc, char **argv) + tmpserver = g_strconcat(tmpdir, "/server", NULL); + + qtest_add_func("/ivshmem/single", test_ivshmem_single); ++#if 0 /* Test uses legacy ivshmem, which is disabled for Red Hat Enterprise Linux: */ + qtest_add_func("/ivshmem/hotplug", test_ivshmem_hotplug); ++#endif + qtest_add_func("/ivshmem/memdev", test_ivshmem_memdev); + if (g_test_slow()) { + qtest_add_func("/ivshmem/pair", test_ivshmem_pair); ++#if 0 /* Disabled for Red Hat Enterprise Linux: */ + if (strcmp(arch, "ppc64") != 0) { + qtest_add_func("/ivshmem/server-msi", test_ivshmem_server_msi); + qtest_add_func("/ivshmem/server-irq", test_ivshmem_server_irq); + } ++#endif + } + + ret = g_test_run(); +diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 +index f617e25..69d34eb 100755 +--- a/tests/qemu-iotests/051 ++++ b/tests/qemu-iotests/051 +@@ -168,11 +168,11 @@ run_qemu -drive if=virtio + case "$QEMU_DEFAULT_MACHINE" in + pc) + run_qemu -drive if=none,id=disk -device ide-cd,drive=disk +- run_qemu -drive if=none,id=disk -device lsi53c895a -device scsi-cd,drive=disk ++# run_qemu -drive if=none,id=disk -device lsi53c895a -device scsi-cd,drive=disk + run_qemu -drive if=none,id=disk -device ide-drive,drive=disk + run_qemu -drive if=none,id=disk -device ide-hd,drive=disk +- run_qemu -drive if=none,id=disk -device lsi53c895a -device scsi-disk,drive=disk +- run_qemu -drive if=none,id=disk -device lsi53c895a -device scsi-hd,drive=disk ++# run_qemu -drive if=none,id=disk -device lsi53c895a -device scsi-disk,drive=disk ++# run_qemu -drive if=none,id=disk -device lsi53c895a -device scsi-hd,drive=disk + ;; + *) + ;; +@@ -197,11 +197,11 @@ run_qemu -drive file="$TEST_IMG",if=virtio,readonly=on + case "$QEMU_DEFAULT_MACHINE" in + pc) + run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device ide-cd,drive=disk +- run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device lsi53c895a -device scsi-cd,drive=disk ++# run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device lsi53c895a -device scsi-cd,drive=disk + run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device ide-drive,drive=disk + run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device ide-hd,drive=disk +- run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device lsi53c895a -device scsi-disk,drive=disk +- run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device lsi53c895a -device scsi-hd,drive=disk ++# run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device lsi53c895a -device scsi-disk,drive=disk ++# run_qemu -drive file="$TEST_IMG",if=none,id=disk,readonly=on -device lsi53c895a -device scsi-hd,drive=disk + ;; + *) + ;; +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 52a80f3..99777ec 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -77,7 +77,7 @@ + 068 rw auto quick + 069 rw auto quick + 070 rw auto quick +-071 rw auto quick ++# 071 rw auto quick -- requires whitelisted blkverify + 072 rw auto quick + 073 rw auto quick + 074 rw auto quick +@@ -105,7 +105,7 @@ + 096 rw auto quick + 097 rw auto backing + 098 rw auto backing quick +-099 rw auto quick ++# 099 rw auto quick -- requires whitelisted blkverify + # 100 was removed, do not reuse + 101 rw auto quick + 102 rw auto quick +diff --git a/tests/qom-test.c b/tests/qom-test.c +index a34ff6b..db0d3ab 100644 +--- a/tests/qom-test.c ++++ b/tests/qom-test.c +@@ -16,7 +16,9 @@ + #include "libqtest.h" + + static const char *blacklist_x86[] = { +- "xenfv", "xenpv", NULL ++ "xenfv", "xenpv", "isapc", ++ "rhel6.6.0", "rhel6.5.0", "rhel6.4.0", "rhel6.3.0", ++ "rhel6.2.0", "rhel6.1.0", "rhel6.0.0", NULL + }; + + static const struct { +diff --git a/tests/test-hmp.c b/tests/test-hmp.c +index 5352c9c..674dd64 100644 +--- a/tests/test-hmp.c ++++ b/tests/test-hmp.c +@@ -139,7 +139,7 @@ static void add_machine_test_case(const char *mname) + char *path; + + /* Ignore blacklisted machines that have known problems */ +- if (!strcmp("xenfv", mname) || !strcmp("xenpv", mname)) { ++ if (!strcmp("xenfv", mname) || !strcmp("xenpv", mname) || !strcmp("isapc", mname)) { + return; + } + +diff --git a/tests/test-x86-cpuid-compat.c b/tests/test-x86-cpuid-compat.c +index 02e4184..9e4a508 100644 +--- a/tests/test-x86-cpuid-compat.c ++++ b/tests/test-x86-cpuid-compat.c +@@ -306,6 +306,7 @@ int main(int argc, char **argv) + "-cpu 486,xlevel2=0xC0000002,+xstore", + "xlevel2", 0xC0000002); + ++#if 0 /* Disabled in Red Hat Enterprise Linux */ + /* Check compatibility of old machine-types that didn't + * auto-increase level/xlevel/xlevel2: */ + +@@ -356,6 +357,7 @@ int main(int argc, char **argv) + add_cpuid_test("x86/cpuid/xlevel-compat/pc-i440fx-2.4/npt-on", + "-machine pc-i440fx-2.4 -cpu SandyBridge,+npt", + "xlevel", 0x80000008); ++#endif + + /* Test feature parsing */ + add_feature_test("x86/cpuid/features/plus", +diff --git a/tests/usb-hcd-xhci-test.c b/tests/usb-hcd-xhci-test.c +index 9c14e30..192b7f7 100644 +--- a/tests/usb-hcd-xhci-test.c ++++ b/tests/usb-hcd-xhci-test.c +@@ -21,6 +21,7 @@ static void test_xhci_hotplug(void) + usb_test_hotplug("xhci", 1, NULL); + } + ++#if 0 /* Disabled for Red Hat Enterprise Linux 7 */ + static void test_usb_uas_hotplug(void) + { + qtest_qmp_device_add("usb-uas", "uas", NULL); +@@ -34,6 +35,7 @@ static void test_usb_uas_hotplug(void) + qtest_qmp_device_del("scsihd"); + qtest_qmp_device_del("uas"); + } ++#endif + + int main(int argc, char **argv) + { +@@ -43,8 +45,9 @@ int main(int argc, char **argv) + + qtest_add_func("/xhci/pci/init", test_xhci_init); + qtest_add_func("/xhci/pci/hotplug", test_xhci_hotplug); ++#if 0 /* Disabled for Red Hat Enterprise Linux 7 */ + qtest_add_func("/xhci/pci/hotplug/usb-uas", test_usb_uas_hotplug); +- ++#endif + qtest_start("-device nec-usb-xhci,id=xhci" + " -drive id=drive0,if=none,file=null-co://,format=raw"); + ret = g_test_run(); +diff --git a/vl.c b/vl.c +index fce1fd1..03950fc 100644 +--- a/vl.c ++++ b/vl.c +@@ -165,7 +165,7 @@ unsigned int max_cpus; + int smp_cores = 1; + int smp_threads = 1; + int acpi_enabled = 1; +-int no_hpet = 0; ++int no_hpet = 1; /* Always disabled for Red Hat Enterprise Linux */ + int fd_bootchk = 1; + static int no_reboot; + int no_shutdown = 0; +-- +1.8.3.1 + diff --git a/SOURCES/0003-Add-RHEL-7-machine-types.patch b/SOURCES/0003-Add-RHEL-7-machine-types.patch new file mode 100644 index 0000000..a2163b4 --- /dev/null +++ b/SOURCES/0003-Add-RHEL-7-machine-types.patch @@ -0,0 +1,3596 @@ +From 6f781f8512bbc6ae19184c8e26c96c48ffa31d93 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Sun, 14 Dec 2014 18:32:18 +0100 +Subject: Add RHEL 7 machine types + +This commit adds all changes related to machine types applied since +qemu-kvm-rhev-2.1.2-16.el7. + +Signed-off-by: Miroslav Rezanina +Signed-off-by: Danilo C. L. de Paula + +-- + +Rebase notes for RHEL-8 (2.12.0): +- replace has_dynamic_sysbus with list of dynamic devices (upstream) +- Changed force_rev1_fadt handling +- Added default nic to machine options for pc_piix and pc_q35 +- Disabled m48t59-test for ppc64 +- Disabled unsupported machine types in boot-serial-test +- Disabled pseries-rhel7.2.0 machine type for cpu-plug-test + +Rebase notes for RHEL-8 (2.11.0): +- Removed references to rhel6 rom files (virtio.rom, pcnet, rtl3193, + net2k_pci e1000) +- Removed CONFIG_RHV usage from hw/ppc/spapr.c +- Removed x-write-pointer-available property from compat.h + +Rebase notes (2.11.0): +- Return type for VMStateDescription.pre_save changed (upstream) +- Use default_cpu_type instead of tcg_default_cpu for sPAPRMachineClass +- Use virt_get_default_cpu_node_id in arm rhel machine type +- Set default_cpu_type to cortex-a57 +- Disable options for arm boards instead of object file remove +- Changed order with device disable commit - removed unnecessary changes + +Rebase notes (2.10.0): +- Commented out new unused functions in hw/arm/virt.c +- savevm_skip_section_footers changed to COMPAT variable (upstream) +- global_state_set_optional changed to COMPAT variable (upstream) +- savevm_skip_configuration changed to COMPAT variable (upstream) +- move shadow_bios() to separate module with stub version available +- set possible_cpu_arch_ids and cpu_index_to_instance_props fields for arm machine type +- Re-enable older machine types for ppc64 +- Commented 7.2 rebase error for compat qemu-extended-regs + +Rebase notes (2.9.0): +- new header file for arm (upstream) +- query_hotpluggable_cpus renamed to has_hotpluggable_cpus (upstream) +- replace MAX_CPUMASK_BITS with max_cpus +- Adding rhel6-e1000.rom moved to Initial redhat commit +- Fixed conflict on cirrus_vga.c + +Rebase notes (2.8.0): +- new "m->max_cpus = 288" in pc_q35_machine_options hw/i386/pc_q35.c + +Rebase notes (2.7.0): +- Additional fsl-imx6.o sabrelito.o files in arm hw dir + +Rebase notes (2.6.0): +- Changes in handling of some compat properties +- Fixes in x86_64 copmat models +- Added required devices for aarch64 +- Fixes for ppc machine types + +Rebase notes (2.5.0): +- changed cpu defaults structure +- chnaged cpu compat properties handling +- added fix for arm machine type + +Rebase notes (2.4.0) +- Moved needed attribute (due to 5cd8cadae8db905afcbf877cae568c27d1d55a8a) +- Fixes to machine types changes + +Merged patches (2.12.0): +- 402de2c redhat: Define the pseries-rhel7.5-sxxm machine type +- a232ba5 redhat: Define the pseries-rhel7.4-sxxm machine type +- 54f90c2 redhat: Define the pseries-rhel7.3-sxxm machine type +- 3fe3c44 Disable GeForce quirks in vfio-pci for RHEL machines +- 4780659 target-i386: cpu: add new CPU models for indirect branch predictor restrictions +- d486f00 hw/ppc/spapr_caps: Rework spapr_caps to use uint8 internal representation +- cc06449 spapr: Handle Decimal Floating Point (DFP) as an optional capability +- 64ef8ed spapr: Handle VMX/VSX presence as an spapr capability flag +- acad5a1 spapr: Treat Hardware Transactional Memory (HTM) as an optional capability +- b1163e4 serial: always transmit send/receive buffers on migration + +Merged patches (2.11.0): +- 58702e8dbb redhat: fix HW_COMPAT_RHEL7_3 +- ea157600da redhat: define HW_COMPAT_RHEL7_4 +- 86ef5cd189 redhat: define pseries-rhel7.5.0 machine type +- d8bf28e9ec qemu-kvm-ma: define only pseries-rhel7.5.0 machine type for -ma +- 9b07271835 Create x86 7.5.0 machine types +- e63d707db9 acpi: Force rev1 FADT on old q35 machine types +- c091cd7a1b pc: make pc_rom RO only on new machine types +- 082bd3ba3d arm/virt: Add RHEL 7.5 machine typei +- 8663586f2b qemu-kvm-rhev: only allows pseries-rhel7.5.0 machine type with POWER9 guest +- 229441f111 machine compat: pci_bridge/shpc always enable +- 917c9e7df7 pcie_root_port: Fix x-migrate-msix compat +- b0c0614041 q35: Fix mismerge +- 888e98c6d6 hw/pci-host: Fix x86 Host Bridges 64bit PCI hole (partial) +- b2f9f4fcaa i386/cpu/hyperv: support over 64 vcpus for windows guests (partial) +- cc306393d7 migcompat/e1000e: Work around 7.3 msi/intr_state field +- c4753f76a3 migcompat/rtl8139: Work around version bump + +Merged patches (2.10.0) +- 5090ba1 Fix qemu-kvm does not quit when booting guest w/ 241 vcpus and "-no-kvm" +- 3c52050 x86: bump maximum vcpu count of pc-q35-rhel7.4.0 to 288 +- 0ed92ca x86: Work around SMI breakages +- f7b5a7f spapr: update SPAPR_COMPAT_RHEL7_3 +- 04faf4d migration: update HW_COMPAT_RHEL7_3 +- 9982768 x86: bump maximum vcpu count of pc-q35-rhel7.4.0 to 384 +- f5095ba re-enable DMA for 7.3 machine type +- d6ec65d x86 machine compat: 2.9 stragglers +- eda659a disable linuxboot_dma.bin option rom for 7.3 machine types +- 8c122a5 Revert "hw/pci: disable pci-bridge's shpc by default" (partial) +- 627adfa virtio_net: Bypass backends for MTU feature negotiation (partial) +- b430787 pc: Use "min[x]level" on compat_props on RHEL machine-types +- 8c3f45a AArch64: remove mach-virt-7.3 machine type +- d5df1ef Downstream: Update pseries machine types for RHEL-ALT-7.4 + +Merged patches (2.9.0) +- 8475d69 hw/arm/virt: Disable virtio-net-pci option ROM file loading +- 73fe1f6 Workaround rhel6 ctrl_guest_offloads machine type mismatch +- 21d32ca pc_piix: fix compat props typo for RHEL6 machine types +- 55a5002 compat: define HW_COMPAT_RHEL7_3 +- 1b8e927 spapr: define pseries-rhel7.4.0 machine type +- cdb76ec hw/arm/virt: remove aarch64 rhel machine type +- 7dfa88b hw/arm/virt: create virt-rhel7.3.0 machine type +- 6894f91 hw/arm/virt: create virt-rhel7.4.0 machine type +- a9d2d39 x86: Split out options for the head rhel7 machine types +- fdafbdc x86: Create PC_RHEL7_3_COMPAT definition +- 3427c72 x86: Define pc-i440fx-rhel7.4.0 +- aea20ab x86: Define pc-q35-rhel7.4.0 +- 0185c0f x86: Remove downstream opteron rdtscp override +- 6b51073 fix abort in acpi_setup() since 2.8 with rhel6 machine types +- 954fc0d intel-hda: fix rhel6 compat property +- 1b57274 kvmclock: reduce kvmclock difference on migration (rhel only part) + +Merged patches (2.8.0) +- a1da2f0 virtio-pci: reduce modern_mem_bar size (rhel only part) + +Merged patches (2.7.0): +- fe9d1cf pc: Use right HW_COMPAT_* macros at PC_RHEL7* compat macros +- 3938189 compat: Add missing "any_layout" in HW_COMPAT_RHEL7_1 +- 6dffc9d spapr: update RHEL-7.2 machine type +- c5d5910 migration: fix HW_COMPAT_RHEL7_2 +- 2da9bb8 pc: New (default) pc-i440fx-rhel7.3.0 machine-type +- 0520d7e 7.3 mismerge fix: Fix ich9-intel-hda compatibility +- 89528b3 PC migration compat: Section footers/global state +- 2231e35 fw_cfg for 7.2 compatibility +- b8a3ade pc: Create new pc-q35-rhel7.3.0 machine-type +- 340929b q35: Remove 7.0, 7.1, 7.2 machine types +- bb7fc95 machine types: fix pc_machine_*_options chain +- d9fa9aa Fix rhel6 rom file +- dc39363 fix vga type for older machines +- 255a2d1 7.2 machine type compatibility +- 16c3d25 target-i386: Remove SSE4a from qemu64 CPU model (rhel only part) +- 76a1796 target-i386: Remove ABM from qemu64 CPU model (rhel only part) +- a9f8773 pc: Recover PC_RHEL7_1_COMPAT from RHEL-7.2 code +- 7a6ed67 pc: Include missing PC_COMPAT_2_3 entries in PC_RHEL7_2_COMPAT +- 07428f6 Revert "static checker: e1000-82540em got aliased to e1000" +- 446cf1f Revert "e1000: use alias for default model" +- 615096e 7.x compat: e1000-82540em +- 0855905 hw/arm/virt: kill 7.2 machine type +- 18bbea2 usbredir: turn off streams for rhel7.2 & older +- 910cf4a target-i386: Fill high bits of mtrr mask (rhel only part) +- 0e8ab1b target-i386: Enable host-phys-bits on RHEL +- 8c5f8a5 pc: Fix rhel6.3.0 compat_props setting +- 8f869f1 pc: use new CPU hotplug interface since 2.7 machine type (rhel only part) +- d9d646f machine: add properties to compat_props incrementaly (rhel only part) +- acb18fd apic: Use apic_id as apic's migration instance_id (rhel only part) +- c7e37d4 apic: fix broken migration for kvm-apic (rhel only part) +- eca64aee hw/virtio-pci: fix virtio behaviour +- c56b8F6e pc-rhel-7.2: pcie: fix link active status bit migration +- 5522aa3 q35-rhel: allow dynamic sysbus + +Merged patches (2.6.0): +- f915d7f arm: virt: Add an abstract RHEL ARM virt machine type +- deffcc0 arm: virt: Add RHEL 7.3.0 virt machine type +- 04ca07d arm: virt: Consolidate the naming of RHEL virt machine types +- 2856ce2 Define HW_COMPAT_RHEL7_2 +- 1869242 spapr: move pseries-2.5 machine to RHEL disabled machine zone +- cc59ce7 spapr: add RHEL-7.3 machine type +- 98549c5 pc: Fix property names on CPU compat code +- caa47bb Fix ich9-intel-hda compatibility + +Merged patches (2.3.0): +- bb4e53c2 pc: add rhel6.6.0 machine type +- 129a2b3 Downstream-only: Restore "pseries" machine alias + +Merged patches (2.4.0): +- 8e8107c numa: Don't allow memdev= on RHEL-6 machine-types +- 8b220c0 pc_sysfw: prevent pflash and/or mis-sized firmware for rhel6.x.0 machtypes +- 9dba3a5 Add pc-i440fx-rhel7.2.0 machine type +- 1c88ffa Add pc-q35-rhel7.2.0 machine type +- 6f74d0c Downstream-only: Add rhel7.2.0 machine type +- a7d6105 Add flag for pre-2.2 migration compatibility +- 17f9a18 Serial: Migration compatibility pre 2.2/7.2 +- 3799a57 Migration compat for mc146818rtc/irq_reinject_on_ack_count subsection +- 5668cc1 Fix reported machine type +- 2417534 386: drop FDC in pc-q35-rhel7.2.0 if neither it nor fl. drives are anted +- f42eee5 global_state: Make section optional +- 8640f84 migration: Add configuration section +- 48c857b pc: memhotplug: fix incorrectly set reserved-memory-end +- f33f0b6 pc: memhotplug: keep reserved-memory-end broken on rhel71 and earlier machines + +(cherry picked from commit 44f7e7595c416686a00015e317e74183037a8136) + +RH-Author: Paolo Bonzini +Message-id: <20180111135644.16253-1-pbonzini@redhat.com> +Patchwork-id: 78551 +O-Subject: [RHEL7.5 qemu-kvm-rhev PATCH v2] serial: always transmit send/receive buffers on migration +Bugzilla: 1459945 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Miroslav Rezanina + +When subsections were added in qemu 2.3, they were all disabled +for machine types that require pre-2.3 compatibility. The commit +message says "disabling these subsections on older machine types should +leave it no worse than existing qemu", but actually migrating from +new QEMU to new QEMU (but with old machine type) will detect an +inconsistent migration state and fail the migration: + + qemu-system-x86_64: inconsistent state in serial device (tsr not empty, tsr_retry=0 + qemu-system-x86_64: Failed to load serial:state + qemu-system-x86_64: error while loading state for instance 0x0 of device 'serial' + +In fact, this shows that migration from new source to old destination +might have also eaten the data in the FIFO or transmit/receive buffers. + +It's actually pretty easy to trigger the failure by connecting a console +to a hung-up reader (not a *disconnected* reader!). The fix is to +handle the subsections the same as we did in the qemu-kvm BZ1452067. +The data registers are migrated, which may indeed cause some more migrations +to fail to old qemu-kvm-rhev, but it will fix migration to new qemu-kvm-rhev. +Some subsections are still keyed on migrate_pre_2_2; from the commit message +of downstream commit 7d2e8f9662feb64c0b15b6fd53e06e3c56921f27: + + thr_ipending can be reconstructed fairly + reliably by serial_post_load. The others are features that are + unlikely to be used in RHEL, respectively receive timeout (Linux + does not even have the UART_IIR_CTI symbol in the driver) and + physical serial ports connected to a modem + +I consider this okay because nobody has yet complained about it for +qemu-kvm. It's also safer because the failure avoids serial data loss +on migration. This is consistent with the intended use of subsections. + +Signed-off-by: Paolo Bonzini +Signed-off-by: Miroslav Rezanina +(cherry picked from commit 10564d0bb819113f925e32b989b24fb26dca45ef) +(cherry picked from commit 9e1104c955c05c7b850cda3a02f69bf3e931c765) +--- + hw/acpi/ich9.c | 16 + + hw/acpi/piix4.c | 6 +- + hw/arm/virt.c | 121 +++++- + hw/char/serial.c | 16 + + hw/display/cirrus_vga.c | 4 +- + hw/display/vga-isa.c | 2 +- + hw/i386/Makefile.objs | 1 + + hw/i386/acpi-build.c | 3 + + hw/i386/pc.c | 7 +- + hw/i386/pc_piix.c | 872 +++++++++++++++++++++++++++++++++++++- + hw/i386/pc_q35.c | 74 +++- + hw/i386/pc_sysfw.c | 16 + + hw/i386/shadow-bios.c | 64 +++ + hw/net/e1000.c | 18 +- + hw/net/e1000e.c | 21 + + hw/net/rtl8139.c | 4 +- + hw/ppc/spapr.c | 208 +++++++++ + hw/ppc/spapr_cpu_core.c | 13 + + hw/s390x/s390-virtio-ccw.c | 17 +- + hw/smbios/smbios.c | 1 + + hw/timer/i8254_common.c | 2 +- + hw/timer/mc146818rtc.c | 6 + + hw/usb/hcd-uhci.c | 15 +- + hw/usb/hcd-xhci.c | 20 + + hw/usb/hcd-xhci.h | 2 + + hw/virtio/virtio.c | 22 +- + include/hw/acpi/ich9.h | 3 + + include/hw/arm/virt.h | 22 + + include/hw/compat.h | 191 +++++++++ + include/hw/i386/pc.h | 554 ++++++++++++++++++++++++ + include/hw/ppc/spapr.h | 1 + + include/hw/usb.h | 7 + + include/hw/virtio/virtio.h | 1 + + include/sysemu/sysemu.h | 2 + + migration/migration.c | 2 + + migration/migration.h | 5 + + migration/savevm.c | 9 + + numa.c | 13 + + qdev-monitor.c | 1 - + scripts/vmstate-static-checker.py | 1 - + stubs/Makefile.objs | 1 + + stubs/shadow-bios.c | 7 + + target/i386/cpu.c | 9 +- + target/i386/machine.c | 21 + + target/ppc/compat.c | 11 + + target/ppc/cpu.h | 1 + + tests/Makefile.include | 8 +- + tests/cpu-plug-test.c | 3 +- + 48 files changed, 2397 insertions(+), 27 deletions(-) + create mode 100644 hw/i386/shadow-bios.c + create mode 100644 stubs/shadow-bios.c + +diff --git a/hw/acpi/ich9.c b/hw/acpi/ich9.c +index a4e87b8..23a7baa 100644 +--- a/hw/acpi/ich9.c ++++ b/hw/acpi/ich9.c +@@ -441,6 +441,18 @@ static void ich9_pm_set_enable_tco(Object *obj, bool value, Error **errp) + s->pm.enable_tco = value; + } + ++static bool ich9_pm_get_force_rev1_fadt(Object *obj, Error **errp) ++{ ++ ICH9LPCState *s = ICH9_LPC_DEVICE(obj); ++ return s->pm.force_rev1_fadt; ++} ++ ++static void ich9_pm_set_force_rev1_fadt(Object *obj, bool value, Error **errp) ++{ ++ ICH9LPCState *s = ICH9_LPC_DEVICE(obj); ++ s->pm.force_rev1_fadt = value; ++} ++ + void ich9_pm_add_properties(Object *obj, ICH9LPCPMRegs *pm, Error **errp) + { + static const uint32_t gpe0_len = ICH9_PMIO_GPE0_LEN; +@@ -465,6 +477,10 @@ void ich9_pm_add_properties(Object *obj, ICH9LPCPMRegs *pm, Error **errp) + ich9_pm_get_cpu_hotplug_legacy, + ich9_pm_set_cpu_hotplug_legacy, + NULL); ++ object_property_add_bool(obj, "__com.redhat_force-rev1-fadt", ++ ich9_pm_get_force_rev1_fadt, ++ ich9_pm_set_force_rev1_fadt, ++ NULL); + object_property_add(obj, ACPI_PM_PROP_S3_DISABLED, "uint8", + ich9_pm_get_disable_s3, + ich9_pm_set_disable_s3, +diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c +index 8b70345..d706360 100644 +--- a/hw/acpi/piix4.c ++++ b/hw/acpi/piix4.c +@@ -311,7 +311,7 @@ static const VMStateDescription vmstate_cpuhp_state = { + static const VMStateDescription vmstate_acpi = { + .name = "piix4_pm", + .version_id = 3, +- .minimum_version_id = 3, ++ .minimum_version_id = 2, + .minimum_version_id_old = 1, + .load_state_old = acpi_load_old, + .post_load = vmstate_acpi_post_load, +@@ -671,8 +671,8 @@ static void piix4_send_gpe(AcpiDeviceIf *adev, AcpiEventStatusBits ev) + + static Property piix4_pm_properties[] = { + DEFINE_PROP_UINT32("smb_io_base", PIIX4PMState, smb_io_base, 0), +- DEFINE_PROP_UINT8(ACPI_PM_PROP_S3_DISABLED, PIIX4PMState, disable_s3, 0), +- DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_DISABLED, PIIX4PMState, disable_s4, 0), ++ DEFINE_PROP_UINT8(ACPI_PM_PROP_S3_DISABLED, PIIX4PMState, disable_s3, 1), ++ DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_DISABLED, PIIX4PMState, disable_s4, 1), + DEFINE_PROP_UINT8(ACPI_PM_PROP_S4_VAL, PIIX4PMState, s4_val, 2), + DEFINE_PROP_BOOL("acpi-pci-hotplug-with-bridge-support", PIIX4PMState, + use_acpi_pci_hotplug, true), +diff --git a/hw/arm/virt.c b/hw/arm/virt.c +index 94dcb12..806eb1e 100644 +--- a/hw/arm/virt.c ++++ b/hw/arm/virt.c +@@ -59,6 +59,7 @@ + #include "qapi/visitor.h" + #include "standard-headers/linux/input.h" + ++#if 0 /* disabled Red Hat Enterprise Linux */ + #define DEFINE_VIRT_MACHINE_LATEST(major, minor, latest) \ + static void virt_##major##_##minor##_class_init(ObjectClass *oc, \ + void *data) \ +@@ -86,7 +87,36 @@ + DEFINE_VIRT_MACHINE_LATEST(major, minor, true) + #define DEFINE_VIRT_MACHINE(major, minor) \ + DEFINE_VIRT_MACHINE_LATEST(major, minor, false) +- ++#endif /* disabled for RHEL */ ++ ++#define DEFINE_RHEL_MACHINE_LATEST(m, n, s, latest) \ ++ static void rhel##m##n##s##_virt_class_init(ObjectClass *oc, \ ++ void *data) \ ++ { \ ++ MachineClass *mc = MACHINE_CLASS(oc); \ ++ rhel##m##n##s##_virt_options(mc); \ ++ mc->desc = "RHEL " # m "." # n "." # s " ARM Virtual Machine"; \ ++ if (latest) { \ ++ mc->alias = "virt"; \ ++ mc->is_default = 1; \ ++ } \ ++ } \ ++ static const TypeInfo rhel##m##n##s##_machvirt_info = { \ ++ .name = MACHINE_TYPE_NAME("virt-rhel" # m "." # n "." # s), \ ++ .parent = TYPE_RHEL_MACHINE, \ ++ .instance_init = rhel##m##n##s##_virt_instance_init, \ ++ .class_init = rhel##m##n##s##_virt_class_init, \ ++ }; \ ++ static void rhel##m##n##s##_machvirt_init(void) \ ++ { \ ++ type_register_static(&rhel##m##n##s##_machvirt_info); \ ++ } \ ++ type_init(rhel##m##n##s##_machvirt_init); ++ ++#define DEFINE_RHEL_MACHINE_AS_LATEST(major, minor, subminor) \ ++ DEFINE_RHEL_MACHINE_LATEST(major, minor, subminor, true) ++#define DEFINE_RHEL_MACHINE(major, minor, subminor) \ ++ DEFINE_RHEL_MACHINE_LATEST(major, minor, subminor, false) + + /* Number of external interrupt lines to configure the GIC with */ + #define NUM_IRQS 256 +@@ -1416,6 +1446,7 @@ static void machvirt_init(MachineState *machine) + create_platform_bus(vms, pic); + } + ++#if 0 /* disabled for RHEL */ + static bool virt_get_secure(Object *obj, Error **errp) + { + VirtMachineState *vms = VIRT_MACHINE(obj); +@@ -1444,6 +1475,7 @@ static void virt_set_virt(Object *obj, bool value, Error **errp) + vms->virt = value; + } + ++#endif /* disabled for RHEL */ + static bool virt_get_highmem(Object *obj, Error **errp) + { + VirtMachineState *vms = VIRT_MACHINE(obj); +@@ -1536,6 +1568,7 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms) + return ms->possible_cpus; + } + ++#if 0 /* disabled for RHEL */ + static void virt_machine_class_init(ObjectClass *oc, void *data) + { + MachineClass *mc = MACHINE_CLASS(oc); +@@ -1748,3 +1781,89 @@ static void virt_machine_2_6_options(MachineClass *mc) + vmc->no_pmu = true; + } + DEFINE_VIRT_MACHINE(2, 6) ++#endif /* disabled for RHEL */ ++ ++static void rhel_machine_class_init(ObjectClass *oc, void *data) ++{ ++ MachineClass *mc = MACHINE_CLASS(oc); ++ ++ mc->family = "virt-rhel-Z"; ++ mc->init = machvirt_init; ++ /* Start max_cpus at the maximum QEMU supports. We'll further restrict ++ * it later in machvirt_init, where we have more information about the ++ * configuration of the particular instance. ++ */ ++ mc->max_cpus = 255; ++ mc->block_default_type = IF_VIRTIO; ++ mc->no_cdrom = 1; ++ mc->pci_allow_0_address = true; ++ /* We know we will never create a pre-ARMv7 CPU which needs 1K pages */ ++ mc->minimum_page_bits = 12; ++ mc->possible_cpu_arch_ids = virt_possible_cpu_arch_ids; ++ mc->cpu_index_to_instance_props = virt_cpu_index_to_props; ++ mc->default_cpu_type = ARM_CPU_TYPE_NAME("cortex-a57"); ++ mc->get_default_cpu_node_id = virt_get_default_cpu_node_id; ++} ++ ++static const TypeInfo rhel_machine_info = { ++ .name = TYPE_RHEL_MACHINE, ++ .parent = TYPE_MACHINE, ++ .abstract = true, ++ .instance_size = sizeof(VirtMachineState), ++ .class_size = sizeof(VirtMachineClass), ++ .class_init = rhel_machine_class_init, ++}; ++ ++static void rhel_machine_init(void) ++{ ++ type_register_static(&rhel_machine_info); ++} ++type_init(rhel_machine_init); ++ ++static void rhel750_virt_instance_init(Object *obj) ++{ ++ VirtMachineState *vms = VIRT_MACHINE(obj); ++ VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms); ++ ++ /* EL3 is disabled by default and non-configurable for RHEL */ ++ vms->secure = false; ++ /* EL2 is disabled by default and non-configurable for RHEL */ ++ vms->virt = false; ++ /* High memory is enabled by default for RHEL */ ++ vms->highmem = true; ++ object_property_add_bool(obj, "highmem", virt_get_highmem, ++ virt_set_highmem, NULL); ++ object_property_set_description(obj, "highmem", ++ "Set on/off to enable/disable using " ++ "physical address space above 32 bits", ++ NULL); ++ /* Default GIC type is still v2, but became configurable for RHEL */ ++ vms->gic_version = 2; ++ object_property_add_str(obj, "gic-version", virt_get_gic_version, ++ virt_set_gic_version, NULL); ++ object_property_set_description(obj, "gic-version", ++ "Set GIC version. " ++ "Valid values are 2, 3 and host", NULL); ++ ++ if (vmc->no_its) { ++ vms->its = false; ++ } else { ++ /* Default allows ITS instantiation */ ++ vms->its = true; ++ object_property_add_bool(obj, "its", virt_get_its, ++ virt_set_its, NULL); ++ object_property_set_description(obj, "its", ++ "Set on/off to enable/disable " ++ "ITS instantiation", ++ NULL); ++ } ++ ++ vms->memmap=a15memmap; ++ vms->irqmap=a15irqmap; ++} ++ ++static void rhel750_virt_options(MachineClass *mc) ++{ ++ SET_MACHINE_COMPAT(mc, ARM_RHEL_COMPAT); ++} ++DEFINE_RHEL_MACHINE_AS_LATEST(7, 5, 0) +diff --git a/hw/char/serial.c b/hw/char/serial.c +index eb72191..d6d9b18 100644 +--- a/hw/char/serial.c ++++ b/hw/char/serial.c +@@ -30,6 +30,7 @@ + #include "qemu/timer.h" + #include "exec/address-spaces.h" + #include "qemu/error-report.h" ++#include "migration/migration.h" + + //#define DEBUG_SERIAL + +@@ -691,6 +692,9 @@ static int serial_post_load(void *opaque, int version_id) + static bool serial_thr_ipending_needed(void *opaque) + { + SerialState *s = opaque; ++ if (migrate_pre_2_2) { ++ return false; ++ } + + if (s->ier & UART_IER_THRI) { + bool expected_value = ((s->iir & UART_IIR_ID) == UART_IIR_THRI); +@@ -772,6 +776,10 @@ static const VMStateDescription vmstate_serial_xmit_fifo = { + static bool serial_fifo_timeout_timer_needed(void *opaque) + { + SerialState *s = (SerialState *)opaque; ++ if (migrate_pre_2_2) { ++ return false; ++ } ++ + return timer_pending(s->fifo_timeout_timer); + } + +@@ -789,6 +797,10 @@ static const VMStateDescription vmstate_serial_fifo_timeout_timer = { + static bool serial_timeout_ipending_needed(void *opaque) + { + SerialState *s = (SerialState *)opaque; ++ if (migrate_pre_2_2) { ++ return false; ++ } ++ + return s->timeout_ipending != 0; + } + +@@ -806,6 +818,10 @@ static const VMStateDescription vmstate_serial_timeout_ipending = { + static bool serial_poll_needed(void *opaque) + { + SerialState *s = (SerialState *)opaque; ++ if (migrate_pre_2_2) { ++ return false; ++ } ++ + return s->poll_msl >= 0; + } + +diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c +index d116651..feacb45 100644 +--- a/hw/display/cirrus_vga.c ++++ b/hw/display/cirrus_vga.c +@@ -3060,7 +3060,7 @@ static void isa_cirrus_vga_realizefn(DeviceState *dev, Error **errp) + + static Property isa_cirrus_vga_properties[] = { + DEFINE_PROP_UINT32("vgamem_mb", struct ISACirrusVGAState, +- cirrus_vga.vga.vram_size_mb, 4), ++ cirrus_vga.vga.vram_size_mb, 16), + DEFINE_PROP_BOOL("blitter", struct ISACirrusVGAState, + cirrus_vga.enable_blitter, true), + DEFINE_PROP_END_OF_LIST(), +@@ -3133,7 +3133,7 @@ static void pci_cirrus_vga_realize(PCIDevice *dev, Error **errp) + + static Property pci_vga_cirrus_properties[] = { + DEFINE_PROP_UINT32("vgamem_mb", struct PCICirrusVGAState, +- cirrus_vga.vga.vram_size_mb, 4), ++ cirrus_vga.vga.vram_size_mb, 16), + DEFINE_PROP_BOOL("blitter", struct PCICirrusVGAState, + cirrus_vga.enable_blitter, true), + DEFINE_PROP_END_OF_LIST(), +diff --git a/hw/display/vga-isa.c b/hw/display/vga-isa.c +index 469834a..eb44d21 100644 +--- a/hw/display/vga-isa.c ++++ b/hw/display/vga-isa.c +@@ -79,7 +79,7 @@ static void vga_isa_realizefn(DeviceState *dev, Error **errp) + } + + static Property vga_isa_properties[] = { +- DEFINE_PROP_UINT32("vgamem_mb", ISAVGAState, state.vram_size_mb, 8), ++ DEFINE_PROP_UINT32("vgamem_mb", ISAVGAState, state.vram_size_mb, 16), + DEFINE_PROP_END_OF_LIST(), + }; + +diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs +index fa87a14..8c25538 100644 +--- a/hw/i386/Makefile.objs ++++ b/hw/i386/Makefile.objs +@@ -10,3 +10,4 @@ obj-$(CONFIG_VMMOUSE) += vmmouse.o + + obj-y += kvmvapic.o + obj-y += acpi-build.o ++obj-y += shadow-bios.o +diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c +index 3cf2a16..976d151 100644 +--- a/hw/i386/acpi-build.c ++++ b/hw/i386/acpi-build.c +@@ -183,6 +183,9 @@ static void acpi_get_pm_info(AcpiPmInfo *pm) + pm->fadt.reset_reg = r; + pm->fadt.reset_val = 0xf; + pm->fadt.flags |= 1 << ACPI_FADT_F_RESET_REG_SUP; ++ if (object_property_get_bool(lpc, ++ "__com.redhat_force-rev1-fadt", NULL)) ++ pm->fadt.rev = 1; + pm->cpu_hp_io_base = ICH9_CPU_HOTPLUG_IO_BASE; + } + assert(obj); +diff --git a/hw/i386/pc.c b/hw/i386/pc.c +index fdad4bb..6f686c7 100644 +--- a/hw/i386/pc.c ++++ b/hw/i386/pc.c +@@ -1417,7 +1417,8 @@ void pc_memory_init(PCMachineState *pcms, + option_rom_mr = g_malloc(sizeof(*option_rom_mr)); + memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE, + &error_fatal); +- if (pcmc->pci_enabled) { ++ /* RH difference: See bz 1489800, explicitly make ROM ro */ ++ if (pcmc->pc_rom_ro) { + memory_region_set_readonly(option_rom_mr, true); + } + memory_region_add_subregion_overlap(rom_memory, +@@ -2361,6 +2362,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) + pcmc->acpi_data_size = 0x20000 + 0x8000; + pcmc->save_tsc_khz = true; + pcmc->linuxboot_dma_enabled = true; ++ pcmc->pc_rom_ro = true; + mc->get_hotplug_handler = pc_get_hotpug_handler; + mc->cpu_index_to_instance_props = pc_cpu_index_to_props; + mc->get_default_cpu_node_id = pc_get_default_cpu_node_id; +@@ -2370,7 +2372,8 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) + mc->default_boot_order = "cad"; + mc->hot_add_cpu = pc_hot_add_cpu; + mc->block_default_type = IF_IDE; +- mc->max_cpus = 255; ++ /* 240: max CPU count for RHEL */ ++ mc->max_cpus = 240; + mc->reset = pc_machine_reset; + hc->pre_plug = pc_machine_device_pre_plug_cb; + hc->plug = pc_machine_device_plug_cb; +diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c +index 729a050..cc72512 100644 +--- a/hw/i386/pc_piix.c ++++ b/hw/i386/pc_piix.c +@@ -48,6 +48,7 @@ + #include "cpu.h" + #include "qapi/error.h" + #include "qemu/error-report.h" ++#include "migration/migration.h" + #ifdef CONFIG_XEN + #include + #include "hw/xen/xen_pt.h" +@@ -168,8 +169,8 @@ static void pc_init1(MachineState *machine, + if (pcmc->smbios_defaults) { + MachineClass *mc = MACHINE_GET_CLASS(machine); + /* These values are guest ABI, do not change */ +- smbios_set_defaults("QEMU", "Standard PC (i440FX + PIIX, 1996)", +- mc->name, pcmc->smbios_legacy_mode, ++ smbios_set_defaults("Red Hat", "KVM", ++ mc->desc, pcmc->smbios_legacy_mode, + pcmc->smbios_uuid_encoded, + SMBIOS_ENTRY_POINT_21); + } +@@ -307,6 +308,7 @@ static void pc_init1(MachineState *machine, + * HW_COMPAT_*, PC_COMPAT_*, or * pc_*_machine_options(). + */ + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + static void pc_compat_2_3(MachineState *machine) + { + PCMachineState *pcms = PC_MACHINE(machine); +@@ -1135,3 +1137,869 @@ static void xenfv_machine_options(MachineClass *m) + DEFINE_PC_MACHINE(xenfv, "xenfv", pc_xen_hvm_init, + xenfv_machine_options); + #endif ++machine_init(pc_machine_init); ++ ++#endif /* Disabled for Red Hat Enterprise Linux */ ++ ++/* Red Hat Enterprise Linux machine types */ ++ ++/* Options for the latest rhel7 machine type */ ++static void pc_machine_rhel7_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ m->family = "pc_piix_Y"; ++ m->default_machine_opts = "firmware=bios-256k.bin"; ++ pcmc->default_nic_model = "e1000"; ++ m->default_display = "std"; ++ SET_MACHINE_COMPAT(m, PC_RHEL_COMPAT); ++ m->alias = "pc"; ++ m->is_default = 1; ++} ++ ++static void pc_init_rhel750(MachineState *machine) ++{ ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel750_options(MachineClass *m) ++{ ++ pc_machine_rhel7_options(m); ++ m->desc = "RHEL 7.5.0 PC (i440FX + PIIX, 1996)"; ++} ++ ++DEFINE_PC_MACHINE(rhel750, "pc-i440fx-rhel7.5.0", pc_init_rhel750, ++ pc_machine_rhel750_options); ++ ++static void pc_init_rhel740(MachineState *machine) ++{ ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel740_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pc_machine_rhel750_options(m); ++ m->alias = NULL; ++ m->is_default = 0; ++ m->desc = "RHEL 7.4.0 PC (i440FX + PIIX, 1996)"; ++ m->numa_auto_assign_ram = numa_legacy_auto_assign_ram; ++ pcmc->pc_rom_ro = false; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_4_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel740, "pc-i440fx-rhel7.4.0", pc_init_rhel740, ++ pc_machine_rhel740_options); ++ ++static void pc_init_rhel730(MachineState *machine) ++{ ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel730_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pc_machine_rhel740_options(m); ++ m->desc = "RHEL 7.3.0 PC (i440FX + PIIX, 1996)"; ++ pcmc->linuxboot_dma_enabled = false; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_3_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel730, "pc-i440fx-rhel7.3.0", pc_init_rhel730, ++ pc_machine_rhel730_options); ++ ++ ++static void pc_init_rhel720(MachineState *machine) ++{ ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel720_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pc_machine_rhel730_options(m); ++ m->desc = "RHEL 7.2.0 PC (i440FX + PIIX, 1996)"; ++ /* From pc_i440fx_2_5_machine_options */ ++ pcmc->save_tsc_khz = false; ++ m->legacy_fw_cfg_order = 1; ++ /* Note: broken_reserved_end was already in 7.2 */ ++ /* From pc_i440fx_2_6_machine_options */ ++ pcmc->legacy_cpu_hotplug = true; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_2_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel720, "pc-i440fx-rhel7.2.0", pc_init_rhel720, ++ pc_machine_rhel720_options); ++ ++static void pc_compat_rhel710(MachineState *machine) ++{ ++ PCMachineState *pcms = PC_MACHINE(machine); ++ PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms); ++ ++ /* From pc_compat_2_2 */ ++ pcmc->rsdp_in_ram = false; ++ machine->suppress_vmdesc = true; ++ ++ /* From pc_compat_2_1 */ ++ pcmc->smbios_uuid_encoded = false; ++ x86_cpu_change_kvm_default("svm", NULL); ++ pcmc->enforce_aligned_dimm = false; ++ ++ /* Disable all the extra subsections that were added in 2.2 */ ++ migrate_pre_2_2 = true; ++ ++ /* From pc_i440fx_2_4_machine_options */ ++ pcmc->broken_reserved_end = true; ++} ++ ++static void pc_init_rhel710(MachineState *machine) ++{ ++ pc_compat_rhel710(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel710_options(MachineClass *m) ++{ ++ pc_machine_rhel720_options(m); ++ m->family = "pc_piix_Y"; ++ m->desc = "RHEL 7.1.0 PC (i440FX + PIIX, 1996)"; ++ m->default_display = "cirrus"; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_1_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel710, "pc-i440fx-rhel7.1.0", pc_init_rhel710, ++ pc_machine_rhel710_options); ++ ++static void pc_compat_rhel700(MachineState *machine) ++{ ++ PCMachineState *pcms = PC_MACHINE(machine); ++ PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms); ++ ++ pc_compat_rhel710(machine); ++ ++ /* Upstream enables it for everyone, we're a little more selective */ ++ x86_cpu_change_kvm_default("x2apic", NULL); ++ x86_cpu_change_kvm_default("svm", NULL); ++ pcmc->legacy_acpi_table_size = 6418; /* see pc_compat_2_0() */ ++ pcmc->smbios_legacy_mode = true; ++ pcmc->has_reserved_memory = false; ++ migrate_cve_2014_5263_xhci_fields = true; ++} ++ ++static void pc_init_rhel700(MachineState *machine) ++{ ++ pc_compat_rhel700(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel700_options(MachineClass *m) ++{ ++ pc_machine_rhel710_options(m); ++ m->family = "pc_piix_Y"; ++ m->desc = "RHEL 7.0.0 PC (i440FX + PIIX, 1996)"; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_0_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel700, "pc-i440fx-rhel7.0.0", pc_init_rhel700, ++ pc_machine_rhel700_options); ++ ++#define PC_RHEL6_6_COMPAT \ ++ {\ ++ .driver = "scsi-hd",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "scsi-cd",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "scsi-disk",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "ide-hd",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "ide-cd",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "ide-drive",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "virtio-blk-pci",\ ++ .property = "discard_granularity",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "virtio-serial-pci",\ ++ .property = "vectors",\ ++ /* DEV_NVECTORS_UNSPECIFIED as a uint32_t string */\ ++ .value = stringify(0xFFFFFFFF),\ ++ },{\ ++ .driver = "486-" TYPE_X86_CPU,\ ++ .property = "model",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "usb-tablet",\ ++ .property = "usb_version",\ ++ .value = stringify(1),\ ++ },{\ ++ .driver = "virtio-net-pci",\ ++ .property = "mq",\ ++ .value = "off",\ ++ },{\ ++ .driver = "VGA",\ ++ .property = "mmio",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-blk-pci",\ ++ .property = "config-wce",\ ++ .value = "off",\ ++ },{\ ++ .driver = TYPE_ISA_FDC,\ ++ .property = "check_media_rate",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-balloon-pci",\ ++ .property = "class",\ ++ .value = stringify(PCI_CLASS_MEMORY_RAM),\ ++ },{\ ++ .driver = TYPE_PCI_DEVICE,\ ++ .property = "command_serr_enable",\ ++ .value = "off",\ ++ },{\ ++ .driver = "AC97",\ ++ .property = "use_broken_id",\ ++ .value = stringify(1),\ ++ },{\ ++ .driver = "intel-hda",\ ++ .property = "msi",\ ++ .value = "off",\ ++ },{\ ++ .driver = "qemu32-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "486-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "qemu32-" TYPE_X86_CPU,\ ++ .property = "model",\ ++ .value = stringify(3),\ ++ },{\ ++ .driver = "usb-ccid",\ ++ .property = "serial",\ ++ .value = "1",\ ++ },{\ ++ .driver = "virtio-net-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "pentium" "-" TYPE_X86_CPU,\ ++ .property = "apic",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "pentium2" "-" TYPE_X86_CPU,\ ++ .property = "apic",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "pentium3" "-" TYPE_X86_CPU,\ ++ .property = "apic",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Conroe" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Penryn" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Nehalem" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Nehalem-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "pclmulqdq",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "pclmulqdq",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "fxsr",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "fxsr",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "mmx",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "mmx",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "pat",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "pat",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "cmov",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "cmov",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "pge",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "pge",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "apic",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "cx8",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "cx8",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "mce",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "mce",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "pae",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "pae",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "msr",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "msr",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "tsc",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "tsc",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "pse",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "pse",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "de",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "de",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "fpu",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "fpu",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Broadwell" "-" TYPE_X86_CPU,\ ++ .property = "rdtscp",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "rdtscp",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Broadwell" "-" TYPE_X86_CPU,\ ++ .property = "smap",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "smap",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "rdtscp",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G1" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G2" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G4" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G5" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "3dnow",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "3dnowext",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "virtio-net-pci",\ ++ .property = "__com.redhat_rhel6_ctrl_guest_workaround", \ ++ .value = "on",\ ++ }, ++ ++static void pc_compat_rhel660(MachineState *machine) ++{ ++ PCMachineState *pcms = PC_MACHINE(machine); ++ PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms); ++ ++ pc_compat_rhel700(machine); ++ if (!machine->cpu_type) { ++ machine->cpu_type = "cpu64-rhel6"; ++ } ++ ++ x86_cpu_change_kvm_default("kvm-pv-unhalt", NULL); ++ ++ pcmc->gigabyte_align = false; ++ shadow_bios_after_incoming = true; ++ ich9_uhci123_irqpin_override = true; ++} ++ ++static void pc_init_rhel660(MachineState *machine) ++{ ++ pc_compat_rhel660(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel660_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pc_machine_rhel700_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.6.0 PC"; ++ m->rom_file_has_mr = false; ++ m->default_machine_opts = "firmware=bios.bin"; ++ pcmc->has_acpi_build = false; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_6_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel660, "rhel6.6.0", pc_init_rhel660, ++ pc_machine_rhel660_options); ++ ++#define PC_RHEL6_5_COMPAT \ ++ {\ ++ .driver = TYPE_USB_DEVICE,\ ++ .property = "msos-desc",\ ++ .value = "no",\ ++ }, ++ ++static void pc_compat_rhel650(MachineState *machine) ++{ ++ pc_compat_rhel660(machine); ++} ++ ++static void pc_init_rhel650(MachineState *machine) ++{ ++ pc_compat_rhel650(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel650_options(MachineClass *m) ++{ ++ pc_machine_rhel660_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.5.0 PC"; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_5_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel650, "rhel6.5.0", pc_init_rhel650, ++ pc_machine_rhel650_options); ++ ++#define PC_RHEL6_4_COMPAT \ ++ {\ ++ .driver = "virtio-scsi-pci",\ ++ .property = "vectors",\ ++ .value = stringify(2),\ ++ },{\ ++ .driver = "hda-micro",\ ++ .property = "mixer",\ ++ .value = "off",\ ++ },{\ ++ .driver = "hda-duplex",\ ++ .property = "mixer",\ ++ .value = "off",\ ++ },{\ ++ .driver = "hda-output",\ ++ .property = "mixer",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-net-pci",\ ++ .property = "ctrl_mac_addr",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "sep",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "virtio-net-pci",\ ++ .property = "__com.redhat_rhel6_ctrl_guest_workaround", \ ++ .value = "off",\ ++ }, ++ ++static void pc_compat_rhel640(MachineState *machine) ++{ ++ pc_compat_rhel650(machine); ++} ++ ++static void pc_init_rhel640(MachineState *machine) ++{ ++ pc_compat_rhel640(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel640_options(MachineClass *m) ++{ ++ pc_machine_rhel650_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.4.0 PC"; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_4_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel640, "rhel6.4.0", pc_init_rhel640, ++ pc_machine_rhel640_options); ++ ++#define PC_RHEL6_3_COMPAT \ ++ {\ ++ .driver = "Conroe-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(2),\ ++ },{\ ++ .driver = "Penryn-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(2),\ ++ },{\ ++ .driver = "Nehalem-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(2),\ ++ },{\ ++ .driver = "e1000",\ ++ .property = "autonegotiation",\ ++ .value = "off",\ ++ },{\ ++ .driver = "qxl",\ ++ .property = "revision",\ ++ .value = stringify(3),\ ++ },{\ ++ .driver = "qxl-vga",\ ++ .property = "revision",\ ++ .value = stringify(3),\ ++ },{\ ++ .driver = "virtio-scsi-pci",\ ++ .property = "hotplug",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-scsi-pci",\ ++ .property = "param_change",\ ++ .value = "off",\ ++ },{\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "pmu",\ ++ .value = "on",\ ++ },{\ ++ .driver = "usb-hub",\ ++ .property = "serial",\ ++ .value = "314159",\ ++ },{\ ++ .driver = "usb-storage",\ ++ .property = "serial",\ ++ .value = "1",\ ++ },\ ++ {\ ++ .driver = "SandyBridge" "-" TYPE_X86_CPU,\ ++ .property = "tsc-deadline",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL6_3_COMPAT (copied from the entry above) */ \ ++ .driver = "SandyBridge-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "tsc-deadline",\ ++ .value = "off",\ ++ }, ++ ++static void pc_compat_rhel630(MachineState *machine) ++{ ++ pc_compat_rhel640(machine); ++ x86_cpu_change_kvm_default("kvm-pv-eoi",NULL); ++ enable_compat_apic_id_mode(); ++} ++ ++static void pc_init_rhel630(MachineState *machine) ++{ ++ pc_compat_rhel630(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel630_options(MachineClass *m) ++{ ++ pc_machine_rhel640_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.3.0 PC"; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_3_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel630, "rhel6.3.0", pc_init_rhel630, ++ pc_machine_rhel630_options); ++ ++ ++#define PC_RHEL6_2_COMPAT \ ++ {\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "pmu",\ ++ .value = "off",\ ++ }, ++ ++static void pc_compat_rhel620(MachineState *machine) ++{ ++ pc_compat_rhel630(machine); ++} ++ ++static void pc_init_rhel620(MachineState *machine) ++{ ++ pc_compat_rhel620(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel620_options(MachineClass *m) ++{ ++ pc_machine_rhel630_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.2.0 PC"; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_2_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel620, "rhel6.2.0", pc_init_rhel620, ++ pc_machine_rhel620_options); ++ ++/* ++ * NOTE: We don't have the event_idx compat entry for the ++ * virtio-balloon-pci driver because RHEL6 doesn't disable ++ * it either due to a bug (see RHBZ 1029539 fo more info) ++ */ ++#define PC_RHEL6_1_COMPAT \ ++ {\ ++ .driver = "PIIX4_PM",\ ++ .property = "disable_s3",\ ++ .value = "0",\ ++ },{\ ++ .driver = "PIIX4_PM",\ ++ .property = "disable_s4",\ ++ .value = "0",\ ++ },{\ ++ .driver = "qxl",\ ++ .property = "revision",\ ++ .value = stringify(2),\ ++ },{\ ++ .driver = "qxl-vga",\ ++ .property = "revision",\ ++ .value = stringify(2),\ ++ },{\ ++ .driver = "virtio-blk-pci",\ ++ .property = "event_idx",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-serial-pci",\ ++ .property = "event_idx",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-net-pci",\ ++ .property = "event_idx",\ ++ .value = "off",\ ++ },{\ ++ .driver = "usb-kbd",\ ++ .property = "serial",\ ++ .value = "1",\ ++ },{\ ++ .driver = "usb-mouse",\ ++ .property = "serial",\ ++ .value = "1",\ ++ },{\ ++ .driver = "usb-tablet",\ ++ .property = "serial",\ ++ .value = "1",\ ++ }, ++ ++static void pc_compat_rhel610(MachineState *machine) ++{ ++ pc_compat_rhel620(machine); ++} ++ ++static void pc_init_rhel610(MachineState *machine) ++{ ++ pc_compat_rhel610(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel610_options(MachineClass *m) ++{ ++ pc_machine_rhel620_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.1.0 PC"; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_1_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel610, "rhel6.1.0", pc_init_rhel610, ++ pc_machine_rhel610_options); ++ ++#define PC_RHEL6_0_COMPAT \ ++ {\ ++ .driver = "qxl",\ ++ .property = "revision",\ ++ .value = stringify(1),\ ++ },{\ ++ .driver = "qxl-vga",\ ++ .property = "revision",\ ++ .value = stringify(1),\ ++ },{\ ++ .driver = "VGA",\ ++ .property = "rombar",\ ++ .value = stringify(0),\ ++ }, ++ ++static void pc_compat_rhel600(MachineState *machine) ++{ ++ pc_compat_rhel610(machine); ++} ++ ++static void pc_init_rhel600(MachineState *machine) ++{ ++ pc_compat_rhel600(machine); ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE);} ++ ++static void pc_machine_rhel600_options(MachineClass *m) ++{ ++ pc_machine_rhel610_options(m); ++ m->family = "pc_piix_Z"; ++ m->desc = "RHEL 6.0.0 PC"; ++ SET_MACHINE_COMPAT(m, PC_RHEL6_0_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(rhel600, "rhel6.0.0", pc_init_rhel600, ++ pc_machine_rhel600_options); +diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c +index 9ae9163..dbf6bfa 100644 +--- a/hw/i386/pc_q35.c ++++ b/hw/i386/pc_q35.c +@@ -143,8 +143,8 @@ static void pc_q35_init(MachineState *machine) + + if (pcmc->smbios_defaults) { + /* These values are guest ABI, do not change */ +- smbios_set_defaults("QEMU", "Standard PC (Q35 + ICH9, 2009)", +- mc->name, pcmc->smbios_legacy_mode, ++ smbios_set_defaults("Red Hat", "KVM", ++ mc->desc, pcmc->smbios_legacy_mode, + pcmc->smbios_uuid_encoded, + SMBIOS_ENTRY_POINT_21); + } +@@ -292,6 +292,7 @@ static void pc_q35_init(MachineState *machine) + DEFINE_PC_MACHINE(suffix, name, pc_init_##suffix, optionfn) + + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + static void pc_q35_machine_options(MachineClass *m) + { + PCMachineClass *pcmc = PC_MACHINE_CLASS(m); +@@ -404,3 +405,72 @@ static void pc_q35_2_4_machine_options(MachineClass *m) + + DEFINE_Q35_MACHINE(v2_4, "pc-q35-2.4", NULL, + pc_q35_2_4_machine_options); ++#endif /* Disabled for Red Hat Enterprise Linux */ ++ ++/* Red Hat Enterprise Linux machine types */ ++ ++/* Options for the latest rhel7 q35 machine type */ ++static void pc_q35_machine_rhel7_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pcmc->default_nic_model = "e1000e"; ++ m->family = "pc_q35_Z"; ++ m->default_machine_opts = "firmware=bios-256k.bin"; ++ m->default_display = "std"; ++ m->no_floppy = 1; ++ machine_class_allow_dynamic_sysbus_dev(m, TYPE_SYS_BUS_DEVICE); ++ m->alias = "q35"; ++ m->max_cpus = 384; ++ SET_MACHINE_COMPAT(m, PC_RHEL_COMPAT); ++} ++ ++static void pc_q35_init_rhel750(MachineState *machine) ++{ ++ pc_q35_init(machine); ++} ++ ++static void pc_q35_machine_rhel750_options(MachineClass *m) ++{ ++ pc_q35_machine_rhel7_options(m); ++ m->desc = "RHEL-7.5.0 PC (Q35 + ICH9, 2009)"; ++} ++ ++DEFINE_PC_MACHINE(q35_rhel750, "pc-q35-rhel7.5.0", pc_q35_init_rhel750, ++ pc_q35_machine_rhel750_options); ++ ++static void pc_q35_init_rhel740(MachineState *machine) ++{ ++ pc_q35_init(machine); ++} ++ ++static void pc_q35_machine_rhel740_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pc_q35_machine_rhel750_options(m); ++ m->alias = NULL; ++ m->desc = "RHEL-7.4.0 PC (Q35 + ICH9, 2009)"; ++ m->numa_auto_assign_ram = numa_legacy_auto_assign_ram; ++ pcmc->pc_rom_ro = false; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_4_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(q35_rhel740, "pc-q35-rhel7.4.0", pc_q35_init_rhel740, ++ pc_q35_machine_rhel740_options); ++ ++static void pc_q35_init_rhel730(MachineState *machine) ++{ ++ pc_q35_init(machine); ++} ++ ++static void pc_q35_machine_rhel730_options(MachineClass *m) ++{ ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); ++ pc_q35_machine_rhel740_options(m); ++ m->desc = "RHEL-7.3.0 PC (Q35 + ICH9, 2009)"; ++ m->max_cpus = 255; ++ pcmc->linuxboot_dma_enabled = false; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_3_COMPAT); ++} ++ ++DEFINE_PC_MACHINE(q35_rhel730, "pc-q35-rhel7.3.0", pc_q35_init_rhel730, ++ pc_q35_machine_rhel730_options); +diff --git a/hw/i386/pc_sysfw.c b/hw/i386/pc_sysfw.c +index 73ac783..2a6de35 100644 +--- a/hw/i386/pc_sysfw.c ++++ b/hw/i386/pc_sysfw.c +@@ -207,6 +207,13 @@ static void old_pc_system_rom_init(MemoryRegion *rom_memory, bool isapc_ram_fw) + (bios_size % 65536) != 0) { + goto bios_error; + } ++ if (shadow_bios_after_incoming && bios_size != 128 * 1024) { ++ MachineClass *mc; ++ ++ mc = MACHINE_GET_CLASS(current_machine); ++ error_report("machine %s only supports a 128KB BIOS image", mc->name); ++ exit(1); ++ } + bios = g_malloc(sizeof(*bios)); + memory_region_init_ram(bios, NULL, "pc.bios", bios_size, &error_fatal); + if (!isapc_ram_fw) { +@@ -254,6 +261,15 @@ void pc_system_firmware_init(MemoryRegion *rom_memory, bool isapc_ram_fw) + return; + } + ++ if (shadow_bios_after_incoming) { ++ MachineClass *mc; ++ ++ mc = MACHINE_GET_CLASS(current_machine); ++ error_report("flash-based firmware is not supported by machine %s", ++ mc->name); ++ exit(1); ++ } ++ + if (kvm_enabled() && !kvm_readonly_mem_enabled()) { + /* Older KVM cannot execute from device memory. So, flash memory + * cannot be used unless the readonly memory kvm capability is present. */ +diff --git a/hw/i386/shadow-bios.c b/hw/i386/shadow-bios.c +new file mode 100644 +index 0000000..65a4cb8 +--- /dev/null ++++ b/hw/i386/shadow-bios.c +@@ -0,0 +1,64 @@ ++#include "qemu/osdep.h" ++#include "sysemu/sysemu.h" ++#include "target/i386/cpu.h" ++#include "exec/ram_addr.h" ++#include "qemu/cutils.h" ++ ++void shadow_bios(void) ++{ ++ RAMBlock *block, *ram, *oprom, *bios; ++ size_t one_meg, oprom_size, bios_size; ++ uint8_t *cd_seg_host, *ef_seg_host; ++ ++ ram = NULL; ++ oprom = NULL; ++ bios = NULL; ++ rcu_read_lock(); ++ QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { ++ if (strcmp("pc.ram", block->idstr) == 0) { ++ assert(ram == NULL); ++ ram = block; ++ } else if (strcmp("pc.rom", block->idstr) == 0) { ++ assert(oprom == NULL); ++ oprom = block; ++ } else if (strcmp("pc.bios", block->idstr) == 0) { ++ assert(bios == NULL); ++ bios = block; ++ } ++ } ++ assert(ram != NULL); ++ assert(oprom != NULL); ++ assert(bios != NULL); ++ assert(memory_region_is_ram(ram->mr)); ++ assert(memory_region_is_ram(oprom->mr)); ++ assert(memory_region_is_ram(bios->mr)); ++ assert(int128_eq(ram->mr->size, int128_make64(ram->used_length))); ++ assert(int128_eq(oprom->mr->size, int128_make64(oprom->used_length))); ++ assert(int128_eq(bios->mr->size, int128_make64(bios->used_length))); ++ ++ one_meg = 1024 * 1024; ++ oprom_size = 128 * 1024; ++ bios_size = 128 * 1024; ++ assert(ram->used_length >= one_meg); ++ assert(oprom->used_length == oprom_size); ++ assert(bios->used_length == bios_size); ++ ++ ef_seg_host = memory_region_get_ram_ptr(ram->mr) + (one_meg - bios_size); ++ cd_seg_host = ef_seg_host - oprom_size; ++ ++ /* This is a crude hack, but we must distinguish a rhel6.x.0 machtype guest ++ * coming in from a RHEL-6 emulator (where shadowing has had no effect on ++ * "pc.ram") from a similar guest coming in from a RHEL-7 emulator (where ++ * shadowing has worked). In the latter case we must not trample the live ++ * SeaBIOS variables in "pc.ram". ++ */ ++ if (buffer_is_zero(ef_seg_host, bios_size)) { ++ fprintf(stderr, "copying E and F segments from pc.bios to pc.ram\n"); ++ memcpy(ef_seg_host, memory_region_get_ram_ptr(bios->mr), bios_size); ++ } ++ if (buffer_is_zero(cd_seg_host, oprom_size)) { ++ fprintf(stderr, "copying C and D segments from pc.rom to pc.ram\n"); ++ memcpy(cd_seg_host, memory_region_get_ram_ptr(oprom->mr), oprom_size); ++ } ++ rcu_read_unlock(); ++} +diff --git a/hw/net/e1000.c b/hw/net/e1000.c +index 742cd0a..7d568da 100644 +--- a/hw/net/e1000.c ++++ b/hw/net/e1000.c +@@ -1663,6 +1663,16 @@ static void pci_e1000_realize(PCIDevice *pci_dev, Error **errp) + + pci_conf = pci_dev->config; + ++ if (!(d->compat_flags & E1000_FLAG_AUTONEG)) { ++ /* ++ * We have no capabilities, so capability list bit should normally be 0. ++ * Keep it on for compat machine types to avoid breaking migration. ++ * HACK: abuse E1000_FLAG_AUTONEG, which is off exactly for ++ * the machine types that need this. ++ */ ++ pci_set_word(pci_conf + PCI_STATUS, PCI_STATUS_CAP_LIST); ++ } ++ + /* TODO: RST# value should be 0, PCI spec 6.2.4 */ + pci_conf[PCI_CACHE_LINE_SIZE] = 0x10; + +@@ -1763,7 +1773,7 @@ static const TypeInfo e1000_base_info = { + + static const E1000Info e1000_devices[] = { + { +- .name = "e1000", ++ .name = "e1000-82540em", + .device_id = E1000_DEV_ID_82540EM, + .revision = 0x03, + .phy_id2 = E1000_PHY_ID2_8254xx_DEFAULT, +@@ -1784,6 +1794,11 @@ static const E1000Info e1000_devices[] = { + #endif + }; + ++static const TypeInfo e1000_default_info = { ++ .name = "e1000", ++ .parent = "e1000-82540em", ++}; ++ + static void e1000_register_types(void) + { + int i; +@@ -1801,6 +1816,7 @@ static void e1000_register_types(void) + + type_register(&type_info); + } ++ type_register_static(&e1000_default_info); + } + + type_init(e1000_register_types) +diff --git a/hw/net/e1000e.c b/hw/net/e1000e.c +index 16a9417..460123f 100644 +--- a/hw/net/e1000e.c ++++ b/hw/net/e1000e.c +@@ -74,6 +74,11 @@ typedef struct E1000EState { + + E1000ECore core; + ++ /* 7.3 had the intr_state field that was in the original e1000e code ++ * but that was removed prior to 2.7's release ++ */ ++ bool redhat_7_3_intr_state_enable; ++ uint32_t redhat_7_3_intr_state; + } E1000EState; + + #define E1000E_MMIO_IDX 0 +@@ -89,6 +94,10 @@ typedef struct E1000EState { + #define E1000E_MSIX_TABLE (0x0000) + #define E1000E_MSIX_PBA (0x2000) + ++/* Values as in RHEL 7.3 build and original upstream */ ++#define RH_E1000E_USE_MSI BIT(0) ++#define RH_E1000E_USE_MSIX BIT(1) ++ + static uint64_t + e1000e_mmio_read(void *opaque, hwaddr addr, unsigned size) + { +@@ -300,6 +309,8 @@ e1000e_init_msix(E1000EState *s) + } else { + if (!e1000e_use_msix_vectors(s, E1000E_MSIX_VEC_NUM)) { + msix_uninit(d, &s->msix, &s->msix); ++ } else { ++ s->redhat_7_3_intr_state |= RH_E1000E_USE_MSIX; + } + } + } +@@ -471,6 +482,8 @@ static void e1000e_pci_realize(PCIDevice *pci_dev, Error **errp) + ret = msi_init(PCI_DEVICE(s), 0xD0, 1, true, false, NULL); + if (ret) { + trace_e1000e_msi_init_fail(ret); ++ } else { ++ s->redhat_7_3_intr_state |= RH_E1000E_USE_MSI; + } + + if (e1000e_add_pm_capability(pci_dev, e1000e_pmrb_offset, +@@ -594,6 +607,11 @@ static const VMStateDescription e1000e_vmstate_intr_timer = { + VMSTATE_STRUCT_ARRAY(_f, _s, _num, 0, \ + e1000e_vmstate_intr_timer, E1000IntrDelayTimer) + ++static bool rhel_7_3_check(void *opaque, int version_id) ++{ ++ return ((E1000EState *)opaque)->redhat_7_3_intr_state_enable; ++} ++ + static const VMStateDescription e1000e_vmstate = { + .name = "e1000e", + .version_id = 1, +@@ -605,6 +623,7 @@ static const VMStateDescription e1000e_vmstate = { + VMSTATE_MSIX(parent_obj, E1000EState), + + VMSTATE_UINT32(ioaddr, E1000EState), ++ VMSTATE_UINT32_TEST(redhat_7_3_intr_state, E1000EState, rhel_7_3_check), + VMSTATE_UINT32(core.rxbuf_min_shift, E1000EState), + VMSTATE_UINT8(core.rx_desc_len, E1000EState), + VMSTATE_UINT32_ARRAY(core.rxbuf_sizes, E1000EState, +@@ -653,6 +672,8 @@ static PropertyInfo e1000e_prop_disable_vnet, + + static Property e1000e_properties[] = { + DEFINE_NIC_PROPERTIES(E1000EState, conf), ++ DEFINE_PROP_BOOL("__redhat_e1000e_7_3_intr_state", E1000EState, ++ redhat_7_3_intr_state_enable, false), + DEFINE_PROP_SIGNED("disable_vnet_hdr", E1000EState, disable_vnet, false, + e1000e_prop_disable_vnet, bool), + DEFINE_PROP_SIGNED("subsys_ven", E1000EState, subsys_ven, +diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c +index 46daa16..05453e7 100644 +--- a/hw/net/rtl8139.c ++++ b/hw/net/rtl8139.c +@@ -3174,7 +3174,7 @@ static int rtl8139_pre_save(void *opaque) + + static const VMStateDescription vmstate_rtl8139 = { + .name = "rtl8139", +- .version_id = 5, ++ .version_id = 4, + .minimum_version_id = 3, + .post_load = rtl8139_post_load, + .pre_save = rtl8139_pre_save, +@@ -3255,7 +3255,9 @@ static const VMStateDescription vmstate_rtl8139 = { + VMSTATE_UINT32(tally_counters.TxMCol, RTL8139State), + VMSTATE_UINT64(tally_counters.RxOkPhy, RTL8139State), + VMSTATE_UINT64(tally_counters.RxOkBrd, RTL8139State), ++#if 0 /* Disabled for Red Hat Enterprise Linux bz 1420195 */ + VMSTATE_UINT32_V(tally_counters.RxOkMul, RTL8139State, 5), ++#endif + VMSTATE_UINT16(tally_counters.TxAbt, RTL8139State), + VMSTATE_UINT16(tally_counters.TxUndrn, RTL8139State), + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index 6a92b20..c751111 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -3935,6 +3935,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) + smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_BROKEN; + smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_BROKEN; + spapr_caps_add_properties(smc, &error_abort); ++ smc->has_power9_support = true; + } + + static const TypeInfo spapr_machine_info = { +@@ -3985,6 +3986,7 @@ static const TypeInfo spapr_machine_info = { + } \ + type_init(spapr_machine_register_##suffix) + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + /* + * pseries-2.12 + */ +@@ -4136,6 +4138,7 @@ DEFINE_SPAPR_MACHINE(2_8, "2.8", false); + .property = "pre-2.8-migration", \ + .value = "on", \ + }, ++#endif + + static void phb_placement_2_7(sPAPRMachineState *spapr, uint32_t index, + uint64_t *buid, hwaddr *pio, +@@ -4186,6 +4189,7 @@ static void phb_placement_2_7(sPAPRMachineState *spapr, uint32_t index, + */ + } + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ + static void spapr_machine_2_7_instance_options(MachineState *machine) + { + sPAPRMachineState *spapr = SPAPR_MACHINE(machine); +@@ -4345,6 +4349,210 @@ static void spapr_machine_2_1_class_options(MachineClass *mc) + SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_2_1); + } + DEFINE_SPAPR_MACHINE(2_1, "2.1", false); ++#endif ++ ++/* ++ * pseries-rhel7.5.0 ++ */ ++ ++static void spapr_machine_rhel750_instance_options(MachineState *machine) ++{ ++} ++ ++static void spapr_machine_rhel750_class_options(MachineClass *mc) ++{ ++ /* Defaults for the latest behaviour inherited from the base class */ ++} ++ ++DEFINE_SPAPR_MACHINE(rhel750, "rhel7.5.0", true); ++ ++/* ++ * pseries-rhel7.5.0-sxxm ++ * ++ * pseries-rhel7.5.0 with speculative execution exploit mitigations enabled by default ++ */ ++static void spapr_machine_rhel750sxxm_instance_options(MachineState *machine) ++{ ++ spapr_machine_rhel750_instance_options(machine); ++} ++ ++static void spapr_machine_rhel750sxxm_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel750_class_options(mc); ++ smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_FIXED_CCD; ++} ++ ++DEFINE_SPAPR_MACHINE(rhel750sxxm, "rhel7.5.0-sxxm", false); ++ ++/* ++ * pseries-rhel7.4.0 ++ * like SPAPR_COMPAT_2_9 ++ */ ++ ++#define SPAPR_COMPAT_RHEL7_4 \ ++ HW_COMPAT_RHEL7_4 \ ++ { \ ++ .driver = TYPE_POWERPC_CPU, \ ++ .property = "pre-2.10-migration", \ ++ .value = "on", \ ++ }, \ ++ ++static void spapr_machine_rhel740_instance_options(MachineState *machine) ++{ ++ spapr_machine_rhel750_instance_options(machine); ++} ++ ++static void spapr_machine_rhel740_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel750_class_options(mc); ++ SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_RHEL7_4); ++ mc->numa_auto_assign_ram = numa_legacy_auto_assign_ram; ++ smc->has_power9_support = false; ++ smc->pre_2_10_has_unused_icps = true; ++ smc->resize_hpt_default = SPAPR_RESIZE_HPT_DISABLED; ++ smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_ON; ++} ++ ++DEFINE_SPAPR_MACHINE(rhel740, "rhel7.4.0", false); ++ ++/* ++ * pseries-rhel7.4.0-sxxm ++ * ++ * pseries-rhel7.4.0 with speculative execution exploit mitigations enabled by default ++ */ ++static void spapr_machine_rhel740sxxm_instance_options(MachineState *machine) ++{ ++ spapr_machine_rhel740_instance_options(machine); ++} ++ ++static void spapr_machine_rhel740sxxm_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel740_class_options(mc); ++ smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_FIXED_CCD; ++} ++ ++DEFINE_SPAPR_MACHINE(rhel740sxxm, "rhel7.4.0-sxxm", false); ++ ++/* ++ * pseries-rhel7.3.0 ++ * like SPAPR_COMPAT_2_6/_2_7/_2_8 but "ddw" has been backported to RHEL7_3 ++ */ ++#define SPAPR_COMPAT_RHEL7_3 \ ++ HW_COMPAT_RHEL7_3 \ ++ { \ ++ .driver = TYPE_SPAPR_PCI_HOST_BRIDGE, \ ++ .property = "mem_win_size", \ ++ .value = stringify(SPAPR_PCI_2_7_MMIO_WIN_SIZE),\ ++ }, \ ++ { \ ++ .driver = TYPE_SPAPR_PCI_HOST_BRIDGE, \ ++ .property = "mem64_win_size", \ ++ .value = "0", \ ++ }, \ ++ { \ ++ .driver = TYPE_POWERPC_CPU, \ ++ .property = "pre-2.8-migration", \ ++ .value = "on", \ ++ }, \ ++ { \ ++ .driver = TYPE_SPAPR_PCI_HOST_BRIDGE, \ ++ .property = "pre-2.8-migration", \ ++ .value = "on", \ ++ }, \ ++ { \ ++ .driver = TYPE_SPAPR_PCI_HOST_BRIDGE, \ ++ .property = "pcie-extended-configuration-space",\ ++ .value = "off", \ ++ }, ++ ++static void spapr_machine_rhel730_instance_options(MachineState *machine) ++{ ++ sPAPRMachineState *spapr = SPAPR_MACHINE(machine); ++ ++ spapr_machine_rhel740_instance_options(machine); ++ spapr->use_hotplug_event_source = false; ++} ++ ++static void spapr_machine_rhel730_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel740_class_options(mc); ++ mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power7_v2.3"); ++ SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_RHEL7_3); ++ smc->phb_placement = phb_placement_2_7; ++} ++ ++DEFINE_SPAPR_MACHINE(rhel730, "rhel7.3.0", false); ++ ++/* ++ * pseries-rhel7.3.0-sxxm ++ * ++ * pseries-rhel7.3.0 with speculative execution exploit mitigations enabled by default ++ */ ++static void spapr_machine_rhel730sxxm_instance_options(MachineState *machine) ++{ ++ spapr_machine_rhel730_instance_options(machine); ++} ++ ++static void spapr_machine_rhel730sxxm_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel730_class_options(mc); ++ smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_FIXED_CCD; ++} ++ ++DEFINE_SPAPR_MACHINE(rhel730sxxm, "rhel7.3.0-sxxm", false); ++ ++/* ++ * pseries-rhel7.2.0 ++ */ ++/* Should be like SPAPR_COMPAT_2_5 + 2_4 + 2_3, but "dynamic-reconfiguration" ++ * has been backported to RHEL7_2 so we don't need it here. ++ */ ++ ++#define SPAPR_COMPAT_RHEL7_2 \ ++ HW_COMPAT_RHEL7_2 \ ++ { \ ++ .driver = "spapr-vlan", \ ++ .property = "use-rx-buffer-pools", \ ++ .value = "off", \ ++ },{ \ ++ .driver = TYPE_SPAPR_PCI_HOST_BRIDGE,\ ++ .property = "ddw",\ ++ .value = stringify(off),\ ++ }, ++ ++ ++static void spapr_machine_rhel720_instance_options(MachineState *machine) ++{ ++ spapr_machine_rhel730_instance_options(machine); ++} ++ ++static void spapr_machine_rhel720_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel730_class_options(mc); ++ smc->use_ohci_by_default = true; ++ mc->has_hotpluggable_cpus = NULL; ++ SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_RHEL7_2); ++} ++ ++DEFINE_SPAPR_MACHINE(rhel720, "rhel7.2.0", false); + + static void spapr_machine_register_types(void) + { +diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c +index 1eda854..c5f0c72 100644 +--- a/hw/ppc/spapr_cpu_core.c ++++ b/hw/ppc/spapr_cpu_core.c +@@ -21,6 +21,7 @@ + #include "sysemu/numa.h" + #include "sysemu/hw_accel.h" + #include "qemu/error-report.h" ++#include "cpu-models.h" + + static void spapr_cpu_reset(void *opaque) + { +@@ -62,6 +63,7 @@ static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, + Error **errp) + { + CPUPPCState *env = &cpu->env; ++ sPAPRMachineClass *smc = SPAPR_MACHINE_GET_CLASS(spapr); + + /* Set time-base frequency to 512 MHz */ + cpu_ppc_tb_init(env, SPAPR_TIMEBASE_FREQ); +@@ -69,6 +71,17 @@ static void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, + /* Enable PAPR mode in TCG or KVM */ + cpu_ppc_set_papr(cpu, PPC_VIRTUAL_HYPERVISOR(spapr)); + ++ if (!smc->has_power9_support && ++ (((spapr->max_compat_pvr && ++ ppc_compat_cmp(spapr->max_compat_pvr, ++ CPU_POWERPC_LOGICAL_3_00) >= 0)) || ++ (!spapr->max_compat_pvr && ++ ppc_check_compat(cpu, CPU_POWERPC_LOGICAL_3_00, 0, 0)))) { ++ error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND, ++ "POWER9 CPU is not supported by this machine class"); ++ return; ++ } ++ + qemu_register_reset(spapr_cpu_reset, cpu); + spapr_cpu_reset(cpu); + } +diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c +index 435f7c9..3956ac3 100644 +--- a/hw/s390x/s390-virtio-ccw.c ++++ b/hw/s390x/s390-virtio-ccw.c +@@ -647,7 +647,7 @@ bool css_migration_enabled(void) + { \ + MachineClass *mc = MACHINE_CLASS(oc); \ + ccw_machine_##suffix##_class_options(mc); \ +- mc->desc = "VirtIO-ccw based S390 machine v" verstr; \ ++ mc->desc = "VirtIO-ccw based S390 machine " verstr; \ + if (latest) { \ + mc->alias = "s390-ccw-virtio"; \ + mc->is_default = 1; \ +@@ -682,6 +682,8 @@ bool css_migration_enabled(void) + #define CCW_COMPAT_2_10 \ + HW_COMPAT_2_10 + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ ++ + #define CCW_COMPAT_2_9 \ + HW_COMPAT_2_9 \ + {\ +@@ -879,6 +881,19 @@ static void ccw_machine_2_4_class_options(MachineClass *mc) + } + DEFINE_CCW_MACHINE(2_4, "2.4", false); + ++#else ++ ++static void ccw_machine_rhel750_instance_options(MachineState *machine) ++{ ++} ++ ++static void ccw_machine_rhel750_class_options(MachineClass *mc) ++{ ++} ++DEFINE_CCW_MACHINE(rhel750, "rhel7.5.0", true); ++ ++#endif ++ + static void ccw_machine_register_types(void) + { + type_register_static(&ccw_machine_info); +diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c +index 27a07e9..df8f0ae 100644 +--- a/hw/smbios/smbios.c ++++ b/hw/smbios/smbios.c +@@ -778,6 +778,7 @@ void smbios_set_defaults(const char *manufacturer, const char *product, + SMBIOS_SET_DEFAULT(type1.manufacturer, manufacturer); + SMBIOS_SET_DEFAULT(type1.product, product); + SMBIOS_SET_DEFAULT(type1.version, version); ++ SMBIOS_SET_DEFAULT(type1.family, "Red Hat Enterprise Linux"); + SMBIOS_SET_DEFAULT(type2.manufacturer, manufacturer); + SMBIOS_SET_DEFAULT(type2.product, product); + SMBIOS_SET_DEFAULT(type2.version, version); +diff --git a/hw/timer/i8254_common.c b/hw/timer/i8254_common.c +index 6190b6f..ad2ad2d 100644 +--- a/hw/timer/i8254_common.c ++++ b/hw/timer/i8254_common.c +@@ -268,7 +268,7 @@ static const VMStateDescription vmstate_pit_common = { + .pre_save = pit_dispatch_pre_save, + .post_load = pit_dispatch_post_load, + .fields = (VMStateField[]) { +- VMSTATE_UINT32_V(channels[0].irq_disabled, PITCommonState, 3), ++ VMSTATE_UINT32(channels[0].irq_disabled, PITCommonState), /* qemu-kvm's v2 had 'flags' here */ + VMSTATE_STRUCT_ARRAY(channels, PITCommonState, 3, 2, + vmstate_pit_channel, PITChannelState), + VMSTATE_INT64(channels[0].next_transition_time, +diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c +index 6f1f723..68c353f 100644 +--- a/hw/timer/mc146818rtc.c ++++ b/hw/timer/mc146818rtc.c +@@ -34,6 +34,7 @@ + #include "qapi/qapi-commands-misc.h" + #include "qapi/qapi-events-misc.h" + #include "qapi/visitor.h" ++#include "migration/migration.h" + + #ifdef TARGET_I386 + #include "hw/i386/apic.h" +@@ -839,6 +840,11 @@ static int rtc_post_load(void *opaque, int version_id) + static bool rtc_irq_reinject_on_ack_count_needed(void *opaque) + { + RTCState *s = (RTCState *)opaque; ++ ++ if (migrate_pre_2_2) { ++ return false; ++ } ++ + return s->irq_reinject_on_ack_count != 0; + } + +diff --git a/hw/usb/hcd-uhci.c b/hw/usb/hcd-uhci.c +index 836b11f..86d6ab8 100644 +--- a/hw/usb/hcd-uhci.c ++++ b/hw/usb/hcd-uhci.c +@@ -152,6 +152,8 @@ typedef struct UHCI_QH { + uint32_t el_link; + } UHCI_QH; + ++bool ich9_uhci123_irqpin_override; ++ + static void uhci_async_cancel(UHCIAsync *async); + static void uhci_queue_fill(UHCIQueue *q, UHCI_TD *td); + static void uhci_resume(void *opaque); +@@ -1214,12 +1216,23 @@ static void usb_uhci_common_realize(PCIDevice *dev, Error **errp) + UHCIState *s = UHCI(dev); + uint8_t *pci_conf = s->dev.config; + int i; ++ int irq_pin; + + pci_conf[PCI_CLASS_PROG] = 0x00; + /* TODO: reset value should be 0. */ + pci_conf[USB_SBRN] = USB_RELEASE_1; // release number + +- pci_config_set_interrupt_pin(pci_conf, u->info.irq_pin + 1); ++ if (ich9_uhci123_irqpin_override && ++ u->info.vendor_id == PCI_VENDOR_ID_INTEL && ++ (u->info.device_id == PCI_DEVICE_ID_INTEL_82801I_UHCI1 || ++ u->info.device_id == PCI_DEVICE_ID_INTEL_82801I_UHCI2 || ++ u->info.device_id == PCI_DEVICE_ID_INTEL_82801I_UHCI3)) { ++ fprintf(stderr, "RHEL-6 compat: %s: irq_pin = 3\n", u->info.name); ++ irq_pin = 3; ++ } else { ++ irq_pin = u->info.irq_pin; ++ } ++ pci_config_set_interrupt_pin(pci_conf, irq_pin + 1); + + if (s->masterbus) { + USBPort *ports[NB_PORTS]; +diff --git a/hw/usb/hcd-xhci.c b/hw/usb/hcd-xhci.c +index 721beb5..883141f 100644 +--- a/hw/usb/hcd-xhci.c ++++ b/hw/usb/hcd-xhci.c +@@ -3555,9 +3555,27 @@ static const VMStateDescription vmstate_xhci_slot = { + } + }; + ++static int xhci_event_pre_save(void *opaque) ++{ ++ XHCIEvent *s = opaque; ++ ++ s->cve_2014_5263_a = ((uint8_t *)&s->type)[0]; ++ s->cve_2014_5263_b = ((uint8_t *)&s->type)[1]; ++ ++ return 0; ++} ++ ++bool migrate_cve_2014_5263_xhci_fields; ++ ++static bool xhci_event_cve_2014_5263(void *opaque, int version_id) ++{ ++ return migrate_cve_2014_5263_xhci_fields; ++} ++ + static const VMStateDescription vmstate_xhci_event = { + .name = "xhci-event", + .version_id = 1, ++ .pre_save = xhci_event_pre_save, + .fields = (VMStateField[]) { + VMSTATE_UINT32(type, XHCIEvent), + VMSTATE_UINT32(ccode, XHCIEvent), +@@ -3566,6 +3584,8 @@ static const VMStateDescription vmstate_xhci_event = { + VMSTATE_UINT32(flags, XHCIEvent), + VMSTATE_UINT8(slotid, XHCIEvent), + VMSTATE_UINT8(epid, XHCIEvent), ++ VMSTATE_UINT8_TEST(cve_2014_5263_a, XHCIEvent, xhci_event_cve_2014_5263), ++ VMSTATE_UINT8_TEST(cve_2014_5263_b, XHCIEvent, xhci_event_cve_2014_5263), + VMSTATE_END_OF_LIST() + } + }; +diff --git a/hw/usb/hcd-xhci.h b/hw/usb/hcd-xhci.h +index fc36a4c..89d4cf7 100644 +--- a/hw/usb/hcd-xhci.h ++++ b/hw/usb/hcd-xhci.h +@@ -153,6 +153,8 @@ typedef struct XHCIEvent { + uint32_t flags; + uint8_t slotid; + uint8_t epid; ++ uint8_t cve_2014_5263_a; ++ uint8_t cve_2014_5263_b; + } XHCIEvent; + + typedef struct XHCIInterrupter { +diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c +index 006d3d1..4bcb4f4 100644 +--- a/hw/virtio/virtio.c ++++ b/hw/virtio/virtio.c +@@ -24,6 +24,7 @@ + #include "hw/virtio/virtio-access.h" + #include "sysemu/dma.h" + ++#include "standard-headers/linux/virtio_net.h" + /* + * The alignment to use between consumer and producer parts of vring. + * x86 pagesize again. This is the default, used by transports like PCI +@@ -1991,7 +1992,24 @@ const VMStateInfo virtio_vmstate_info = { + static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val) + { + VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev); +- bool bad = (val & ~(vdev->host_features)) != 0; ++ bool bad; ++ uint64_t ctrl_guest_mask = 1ull << VIRTIO_NET_F_CTRL_GUEST_OFFLOADS; ++ ++ if (vdev->rhel6_ctrl_guest_workaround && (val & ctrl_guest_mask) && ++ !(vdev->host_features & ctrl_guest_mask)) { ++ /* ++ * This works around a mistake in the definition of the rhel6.[56].0 ++ * machinetypes, ctrl-guest-offload was not set in qemu-kvm-rhev for ++ * those machine types, but is set on the rhel6 qemu-kvm-rhev build. ++ * If an incoming rhel6 guest uses it then we need to allow it. ++ * Note: There's a small race where a guest read the flag but didn't ++ * declare it's useage yet. ++ */ ++ fprintf(stderr, "RHEL6 ctrl_guest_offload workaround\n"); ++ vdev->host_features |= ctrl_guest_mask; ++ } ++ ++ bad = (val & ~(vdev->host_features)) != 0; + + val &= vdev->host_features; + if (k->set_features) { +@@ -2566,6 +2584,8 @@ static void virtio_device_instance_finalize(Object *obj) + + static Property virtio_properties[] = { + DEFINE_VIRTIO_COMMON_FEATURES(VirtIODevice, host_features), ++ DEFINE_PROP_BOOL("__com.redhat_rhel6_ctrl_guest_workaround", VirtIODevice, ++ rhel6_ctrl_guest_workaround, false), + DEFINE_PROP_END_OF_LIST(), + }; + +diff --git a/include/hw/acpi/ich9.h b/include/hw/acpi/ich9.h +index 59aeb06..7b5cc25 100644 +--- a/include/hw/acpi/ich9.h ++++ b/include/hw/acpi/ich9.h +@@ -61,6 +61,9 @@ typedef struct ICH9LPCPMRegs { + uint8_t smm_enabled; + bool enable_tco; + TCOIORegs tco_regs; ++ ++ /* RH addition, see bz 1489800 */ ++ bool force_rev1_fadt; + } ICH9LPCPMRegs; + + #define ACPI_PM_PROP_TCO_ENABLED "enable_tco" +diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h +index ba0c1a4..2e8e78a 100644 +--- a/include/hw/arm/virt.h ++++ b/include/hw/arm/virt.h +@@ -109,6 +109,7 @@ typedef struct { + int psci_conduit; + } VirtMachineState; + ++#if 0 /* disabled for Red Hat Enterprise Linux */ + #define TYPE_VIRT_MACHINE MACHINE_TYPE_NAME("virt") + #define VIRT_MACHINE(obj) \ + OBJECT_CHECK(VirtMachineState, (obj), TYPE_VIRT_MACHINE) +@@ -117,6 +118,27 @@ typedef struct { + #define VIRT_MACHINE_CLASS(klass) \ + OBJECT_CLASS_CHECK(VirtMachineClass, klass, TYPE_VIRT_MACHINE) + ++#else ++#define TYPE_RHEL_MACHINE MACHINE_TYPE_NAME("virt-rhel") ++#define VIRT_MACHINE(obj) \ ++ OBJECT_CHECK(VirtMachineState, (obj), TYPE_RHEL_MACHINE) ++#define VIRT_MACHINE_GET_CLASS(obj) \ ++ OBJECT_GET_CLASS(VirtMachineClass, obj, TYPE_RHEL_MACHINE) ++#define VIRT_MACHINE_CLASS(klass) \ ++ OBJECT_CLASS_CHECK(VirtMachineClass, klass, TYPE_RHEL_MACHINE) ++#endif ++ ++/* This macro is for changes to properties that are RHEL specific, ++ * different to the current upstream and to be applied to the latest ++ * machine type. ++ */ ++#define ARM_RHEL_COMPAT \ ++ {\ ++ .driver = "virtio-net-pci",\ ++ .property = "romfile",\ ++ .value = "",\ ++ }, ++ + void virt_acpi_setup(VirtMachineState *vms); + + #endif /* QEMU_ARM_VIRT_H */ +diff --git a/include/hw/compat.h b/include/hw/compat.h +index 13242b8..de251fd 100644 +--- a/include/hw/compat.h ++++ b/include/hw/compat.h +@@ -255,4 +255,195 @@ + .value = "on",\ + }, + ++/* Mostly like HW_COMPAT_2_1 but: ++ * we don't need virtio-scsi-pci since 7.0 already had that on ++ * ++ * RH: Note, qemu-extended-regs should have been enabled in the 7.1 ++ * machine type, but was accidentally turned off in 7.2 onwards. ++ * ++ */ ++#define HW_COMPAT_RHEL7_1 \ ++ { /* COMPAT_RHEL7.1 */ \ ++ .driver = "intel-hda-generic",\ ++ .property = "old_msi_addr",\ ++ .value = "on",\ ++ },{\ ++ .driver = "VGA",\ ++ .property = "qemu-extended-regs",\ ++ .value = "off",\ ++ },{\ ++ .driver = "secondary-vga",\ ++ .property = "qemu-extended-regs",\ ++ .value = "off",\ ++ },{\ ++ .driver = "usb-mouse",\ ++ .property = "usb_version",\ ++ .value = stringify(1),\ ++ },{\ ++ .driver = "usb-kbd",\ ++ .property = "usb_version",\ ++ .value = stringify(1),\ ++ },{\ ++ .driver = "virtio-pci",\ ++ .property = "virtio-pci-bus-master-bug-migration",\ ++ .value = "on",\ ++ },{\ ++ .driver = "virtio-blk-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-balloon-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-serial-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-9p-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-rng-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_1 - introduced with 2.10.0 */ \ ++ .driver = "migration",\ ++ .property = "send-configuration",\ ++ .value = "off",\ ++ }, ++ ++/* Mostly like HW_COMPAT_2_4 + 2_3 but: ++ * we don't need "any_layout" as it has been backported to 7.2 ++ */ ++ ++#define HW_COMPAT_RHEL7_2 \ ++ {\ ++ .driver = "virtio-blk-device",\ ++ .property = "scsi",\ ++ .value = "true",\ ++ },{\ ++ .driver = "e1000-82540em",\ ++ .property = "extra_mac_registers",\ ++ .value = "off",\ ++ },{\ ++ .driver = "virtio-pci",\ ++ .property = "x-disable-pcie",\ ++ .value = "on",\ ++ },{\ ++ .driver = "virtio-pci",\ ++ .property = "migrate-extra",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = "fw_cfg_mem",\ ++ .property = "dma_enabled",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = "fw_cfg_io",\ ++ .property = "dma_enabled",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = "isa-fdc",\ ++ .property = "fallback",\ ++ .value = "144",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = "virtio-pci",\ ++ .property = "disable-modern",\ ++ .value = "on",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = "virtio-pci",\ ++ .property = "disable-legacy",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = TYPE_PCI_DEVICE,\ ++ .property = "x-pcie-lnksta-dllla",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_2 */ \ ++ .driver = "virtio-pci",\ ++ .property = "page-per-vq",\ ++ .value = "on",\ ++ },{ /* HW_COMPAT_RHEL7_2 - introduced with 2.10.0 */ \ ++ .driver = "migration",\ ++ .property = "send-section-footer",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_2 - introduced with 2.10.0 */ \ ++ .driver = "migration",\ ++ .property = "store-global-state",\ ++ .value = "off",\ ++ }, ++ ++/* Mostly like HW_COMPAT_2_6 + HW_COMPAT_2_7 + HW_COMPAT_2_8 except ++ * disable-modern, disable-legacy, page-per-vq have already been ++ * backported to RHEL7.3 ++ */ ++#define HW_COMPAT_RHEL7_3 \ ++ { /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-mmio",\ ++ .property = "format_transport_address",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-serial-device",\ ++ .property = "emergency-write",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "ioapic",\ ++ .property = "version",\ ++ .value = "0x11",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "intel-iommu",\ ++ .property = "x-buggy-eim",\ ++ .value = "true",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-pci",\ ++ .property = "x-ignore-backend-features",\ ++ .value = "on",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "fw_cfg_mem",\ ++ .property = "x-file-slots",\ ++ .value = stringify(0x10),\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "fw_cfg_io",\ ++ .property = "x-file-slots",\ ++ .value = stringify(0x10),\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "pflash_cfi01",\ ++ .property = "old-multiple-chip-handling",\ ++ .value = "on",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = TYPE_PCI_DEVICE,\ ++ .property = "x-pcie-extcap-init",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-pci",\ ++ .property = "x-pcie-deverr-init",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-pci",\ ++ .property = "x-pcie-lnkctl-init",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-pci",\ ++ .property = "x-pcie-pm-init",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "virtio-net-device",\ ++ .property = "x-mtu-bypass-backend",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_3 */ \ ++ .driver = "e1000e",\ ++ .property = "__redhat_e1000e_7_3_intr_state",\ ++ .value = "on",\ ++ }, ++ ++/* Mostly like HW_COMPAT_2_9 except ++ * x-mtu-bypass-backend, x-migrate-msix has already been ++ * backported to RHEL7.4. shpc was already on in 7.4. ++ */ ++#define HW_COMPAT_RHEL7_4 \ ++ { /* HW_COMPAT_RHEL7_4 */ \ ++ .driver = "intel-iommu",\ ++ .property = "pt",\ ++ .value = "off",\ ++ }, ++ + #endif /* HW_COMPAT_H */ +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index ffee841..faddeba 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -142,6 +142,9 @@ struct PCMachineClass { + + /* use DMA capable linuxboot option rom */ + bool linuxboot_dma_enabled; ++ ++ /* RH only, see bz 1489800 */ ++ bool pc_rom_ro; + }; + + #define TYPE_PC_MACHINE "generic-pc-machine" +@@ -947,4 +950,555 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); + type_init(pc_machine_init_##suffix) + + extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); ++ ++/* See include/hw/compat.h for shared compatibility lists */ ++ ++/* This macro is for changes to properties that are RHEL specific, ++ * different to the current upstream and to be applied to the latest ++ * machine type. ++ */ ++#define PC_RHEL_COMPAT \ ++ { /* PC_RHEL_COMPAT */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "host-phys-bits",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL_COMPAT bz 1508330 */ \ ++ .driver = "vfio-pci",\ ++ .property = "x-no-geforce-quirks",\ ++ .value = "on",\ ++ }, ++ ++#define PC_RHEL7_4_COMPAT \ ++ HW_COMPAT_RHEL7_4 \ ++ { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_9 */ \ ++ .driver = "mch",\ ++ .property = "extended-tseg-mbytes",\ ++ .value = stringify(0),\ ++ },\ ++ { /* PC_RHEL7_4_COMPAT bz 1489800 */ \ ++ .driver = "ICH9-LPC",\ ++ .property = "__com.redhat_force-rev1-fadt",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_10 */ \ ++ .driver = "i440FX-pcihost",\ ++ .property = "x-pci-hole64-fix",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_10 */ \ ++ .driver = "q35-pcihost",\ ++ .property = "x-pci-hole64-fix",\ ++ .value = "off",\ ++ }, ++ ++ ++#define PC_RHEL7_3_COMPAT \ ++ HW_COMPAT_RHEL7_3 \ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_8 */ \ ++ .driver = "kvmclock",\ ++ .property = "x-mach-use-reliable-get-clock",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_7 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "l3-cache",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_7 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "full-cpuid-auto-level",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_7 */ \ ++ .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ ++ .property = "family",\ ++ .value = "15",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_7 */ \ ++ .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ ++ .property = "model",\ ++ .value = "6",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_7 */ \ ++ .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ ++ .property = "stepping",\ ++ .value = "1",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_7 */ \ ++ .driver = "isa-pcspk",\ ++ .property = "migrate",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_6 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "cpuid-0xb",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_8 */ \ ++ .driver = "ICH9-LPC",\ ++ .property = "x-smi-broadcast",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_8 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "vmware-cpuid-freq",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_8 */ \ ++ .driver = "Haswell-" TYPE_X86_CPU,\ ++ .property = "stepping",\ ++ .value = "1",\ ++ },\ ++ { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_3 added in 2.9 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "kvm-no-smi-migration",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_10 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "x-hv-max-vps",\ ++ .value = "0x40",\ ++ }, ++ ++#define PC_RHEL7_2_COMPAT \ ++ HW_COMPAT_RHEL7_2 \ ++ {\ ++ .driver = "phenom" "-" TYPE_X86_CPU,\ ++ .property = "rdtscp",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "qemu64" "-" TYPE_X86_CPU,\ ++ .property = "sse4a",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "qemu64" "-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "Haswell-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT (copied from the entry above) */ \ ++ .driver = "Haswell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "Haswell-noTSX-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT (copied from the entry above) */ \ ++ .driver = "Haswell-noTSX-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "Broadwell-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "Broadwell-noTSX-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-noTSX-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "abm",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "host" "-" TYPE_X86_CPU,\ ++ .property = "host-cache-info",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "check",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "qemu32" "-" TYPE_X86_CPU,\ ++ .property = "popcnt",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "arat",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "usb-redir",\ ++ .property = "streams",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "fill-mtrr-mask",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_2_COMPAT */ \ ++ .driver = "apic-common",\ ++ .property = "legacy-instance-id",\ ++ .value = "on",\ ++ }, ++ ++ ++ ++#define PC_RHEL7_1_COMPAT \ ++ HW_COMPAT_RHEL7_1 \ ++ {\ ++ .driver = "kvm64" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "kvm32" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Conroe" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Penryn" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Nehalem" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Nehalem-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "SandyBridge" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "SandyBridge-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Haswell" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Haswell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Broadwell" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G1" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G2" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G4" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Opteron_G5" "-" TYPE_X86_CPU,\ ++ .property = "vme",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Haswell" "-" TYPE_X86_CPU,\ ++ .property = "f16c",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Haswell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "f16c",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Haswell" "-" TYPE_X86_CPU,\ ++ .property = "rdrand",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Haswell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "rdrand",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Broadwell" "-" TYPE_X86_CPU,\ ++ .property = "f16c",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "f16c",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "Broadwell" "-" TYPE_X86_CPU,\ ++ .property = "rdrand",\ ++ .value = "off",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT (copied from the entry above) */ \ ++ .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "rdrand",\ ++ .value = "off",\ ++ },\ ++ {\ ++ .driver = "coreduo" "-" TYPE_X86_CPU,\ ++ .property = "vmx",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "core2duo" "-" TYPE_X86_CPU,\ ++ .property = "vmx",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "qemu64" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(4),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "kvm64" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(5),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "pentium3" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(2),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "n270" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(5),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Conroe" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(4),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Penryn" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(4),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Nehalem" "-" TYPE_X86_CPU,\ ++ .property = "min-level",\ ++ .value = stringify(4),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "n270" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Penryn" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Conroe" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Nehalem" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "SandyBridge" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "IvyBridge" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Haswell" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Haswell-noTSX" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Broadwell" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_1_COMPAT */ \ ++ .driver = "Broadwell-noTSX" "-" TYPE_X86_CPU,\ ++ .property = "min-xlevel",\ ++ .value = stringify(0x8000000a),\ ++ }, ++ ++/* ++ * The PC_RHEL_*_COMPAT serve the same purpose for RHEL-7 machine ++ * types as the PC_COMPAT_* do for upstream types. ++ * PC_RHEL_7_*_COMPAT apply both to i440fx and q35 types. ++ * PC_RHEL6_*_COMPAT apply to i440fx types only, and therefore live ++ * in pc_piix.c. ++ */ ++ ++/* ++ * RHEL-7 is based on QEMU 1.5.3, so this needs the PC_COMPAT_* ++ * between our base and 1.5, less stuff backported to RHEL-7.0 ++ * (usb-device.msos-desc), less stuff for devices we changed ++ * (qemu64-x86_64-cpu) or don't support (hpet, pci-serial-2x, ++ * pci-serial-4x) in 7.0. ++ */ ++#define PC_RHEL7_0_COMPAT \ ++ {\ ++ .driver = "virtio-scsi-pci",\ ++ .property = "any_layout",\ ++ .value = "off",\ ++ },{\ ++ .driver = "PIIX4_PM",\ ++ .property = "memory-hotplug-support",\ ++ .value = "off",\ ++ },{\ ++ .driver = "apic",\ ++ .property = "version",\ ++ .value = stringify(0x11),\ ++ },{\ ++ .driver = "nec-usb-xhci",\ ++ .property = "superspeed-ports-first",\ ++ .value = "off",\ ++ },{\ ++ .driver = "nec-usb-xhci",\ ++ .property = "force-pcie-endcap",\ ++ .value = "on",\ ++ },{\ ++ .driver = "pci-serial",\ ++ .property = "prog_if",\ ++ .value = stringify(0),\ ++ },{\ ++ .driver = "virtio-net-pci",\ ++ .property = "guest_announce",\ ++ .value = "off",\ ++ },{\ ++ .driver = "ICH9-LPC",\ ++ .property = "memory-hotplug-support",\ ++ .value = "off",\ ++ },{\ ++ .driver = "xio3130-downstream",\ ++ .property = COMPAT_PROP_PCP,\ ++ .value = "off",\ ++ },{\ ++ .driver = "ioh3420",\ ++ .property = COMPAT_PROP_PCP,\ ++ .value = "off",\ ++ },{\ ++ .driver = "PIIX4_PM",\ ++ .property = "acpi-pci-hotplug-with-bridge-support",\ ++ .value = "off",\ ++ },{\ ++ .driver = "e1000",\ ++ .property = "mitigation",\ ++ .value = "off",\ ++ },{ \ ++ .driver = "virtio-net-pci", \ ++ .property = "ctrl_guest_offloads", \ ++ .value = "off", \ ++ },\ ++ {\ ++ .driver = "Conroe" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Penryn" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Nehalem" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_0_COMPAT (copied from the entry above) */ \ ++ .driver = "Nehalem-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Westmere" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ { /* PC_RHEL7_0_COMPAT (copied from the entry above) */ \ ++ .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G1" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G2" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G4" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ },\ ++ {\ ++ .driver = "Opteron_G5" "-" TYPE_X86_CPU,\ ++ .property = "x2apic",\ ++ .value = "on",\ ++ }, + #endif +diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h +index d60b7c6..5118af6 100644 +--- a/include/hw/ppc/spapr.h ++++ b/include/hw/ppc/spapr.h +@@ -98,6 +98,7 @@ struct sPAPRMachineClass { + bool dr_lmb_enabled; /* enable dynamic-reconfig/hotplug of LMBs */ + bool use_ohci_by_default; /* use USB-OHCI instead of XHCI */ + bool pre_2_10_has_unused_icps; ++ bool has_power9_support; + void (*phb_placement)(sPAPRMachineState *spapr, uint32_t index, + uint64_t *buid, hwaddr *pio, + hwaddr *mmio32, hwaddr *mmio64, +diff --git a/include/hw/usb.h b/include/hw/usb.h +index a5080ad..5b3fb1f 100644 +--- a/include/hw/usb.h ++++ b/include/hw/usb.h +@@ -606,4 +606,11 @@ int usb_get_quirks(uint16_t vendor_id, uint16_t product_id, + uint8_t interface_class, uint8_t interface_subclass, + uint8_t interface_protocol); + ++ ++/* hcd-uhci.c -- RHEL-6 machine type compatibility */ ++extern bool ich9_uhci123_irqpin_override; ++ ++/* hcd-xhci.c -- rhel7.0.0 machine type compatibility */ ++extern bool migrate_cve_2014_5263_xhci_fields; ++ + #endif +diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h +index 098bdaa..41e13d2 100644 +--- a/include/hw/virtio/virtio.h ++++ b/include/hw/virtio/virtio.h +@@ -95,6 +95,7 @@ struct VirtIODevice + uint8_t device_endian; + bool use_guest_notifier_mask; + AddressSpace *dma_as; ++ bool rhel6_ctrl_guest_workaround; + QLIST_HEAD(, VirtQueue) *vector_queues; + }; + +diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h +index 2b42151..5832c38 100644 +--- a/include/sysemu/sysemu.h ++++ b/include/sysemu/sysemu.h +@@ -94,6 +94,8 @@ void qemu_add_machine_init_done_notifier(Notifier *notify); + void qemu_remove_machine_init_done_notifier(Notifier *notify); + + void qemu_announce_self(void); ++extern bool shadow_bios_after_incoming; ++void shadow_bios(void); + + extern int autostart; + +diff --git a/migration/migration.c b/migration/migration.c +index 52a5092..ceb1697 100644 +--- a/migration/migration.c ++++ b/migration/migration.c +@@ -99,6 +99,8 @@ enum mig_rp_message_type { + MIG_RP_MSG_MAX + }; + ++bool migrate_pre_2_2; ++ + /* When we add fault tolerance, we could have several + migrations at once. For now we don't need to add + dynamic creation of migration */ +diff --git a/migration/migration.h b/migration/migration.h +index 8d2f320..06833d7 100644 +--- a/migration/migration.h ++++ b/migration/migration.h +@@ -241,5 +241,10 @@ int migrate_send_rp_req_pages(MigrationIncomingState *mis, const char* rbname, + + void dirty_bitmap_mig_before_vm_start(void); + void init_dirty_bitmap_incoming_migration(void); ++/* ++ * Disables a load of subsections that were added in 2.2/rh7.2 for backwards ++ * migration compatibility. ++ */ ++extern bool migrate_pre_2_2; + + #endif +diff --git a/migration/savevm.c b/migration/savevm.c +index e2be02a..56c9feb 100644 +--- a/migration/savevm.c ++++ b/migration/savevm.c +@@ -45,6 +45,7 @@ + #include "qapi/qapi-commands-misc.h" + #include "qapi/qmp/qerror.h" + #include "qemu/error-report.h" ++#include "qemu/rcu_queue.h" + #include "sysemu/cpus.h" + #include "exec/memory.h" + #include "exec/target_page.h" +@@ -83,6 +84,7 @@ enum qemu_vm_cmd { + MIG_CMD_PACKAGED, /* Send a wrapped stream within this stream */ + MIG_CMD_MAX + }; ++bool shadow_bios_after_incoming; + + #define MAX_VM_CMD_PACKAGED_SIZE UINT32_MAX + static struct mig_cmd_args { +@@ -2204,6 +2206,13 @@ int qemu_loadvm_state(QEMUFile *f) + } + + qemu_loadvm_state_cleanup(); ++ /* Supplement SeaBIOS's shadowing now, because it was useless when the ++ * incoming VM started on the RHEL-6 emulator. ++ */ ++ if (shadow_bios_after_incoming) { ++ shadow_bios(); ++ } ++ + cpu_synchronize_all_post_init(); + + return ret; +diff --git a/numa.c b/numa.c +index 1116c90..daf10d8 100644 +--- a/numa.c ++++ b/numa.c +@@ -493,6 +493,19 @@ void memory_region_allocate_system_memory(MemoryRegion *mr, Object *owner, + return; + } + ++ /* The shadow_bios_after_incoming hack at savevm.c:shadow_bios() is not ++ * able to handle the multiple memory blocks added when using NUMA ++ * memdevs. We can disallow -numa memdev= when using rhel6.* machine-types ++ * because RHEL-6 didn't support the NUMA memdev option. ++ */ ++ if (shadow_bios_after_incoming) { ++ MachineClass *mc; ++ mc = MACHINE_GET_CLASS(current_machine); ++ error_report("-numa memdev is not supported by machine %s", ++ mc->name); ++ exit(1); ++ } ++ + memory_region_init(mr, owner, name, ram_size); + for (i = 0; i < nb_numa_nodes; i++) { + uint64_t size = numa_info[i].node_mem; +diff --git a/qdev-monitor.c b/qdev-monitor.c +index 61e0300..f439b83 100644 +--- a/qdev-monitor.c ++++ b/qdev-monitor.c +@@ -47,7 +47,6 @@ typedef struct QDevAlias + + /* Please keep this table sorted by typename. */ + static const QDevAlias qdev_alias_table[] = { +- { "e1000", "e1000-82540em" }, + { "ich9-ahci", "ahci" }, + { "lsi53c895a", "lsi" }, + { "virtio-9p-ccw", "virtio-9p", QEMU_ARCH_S390X }, +diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-checker.py +index bcef7ee..ffb13d1 100755 +--- a/scripts/vmstate-static-checker.py ++++ b/scripts/vmstate-static-checker.py +@@ -104,7 +104,6 @@ def get_changed_sec_name(sec): + # Section names can change -- see commit 292b1634 for an example. + changes = { + "ICH9 LPC": "ICH9-LPC", +- "e1000-82540em": "e1000", + } + + for item in changes: +diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs +index dfdfca7..8f111c5 100644 +--- a/stubs/Makefile.objs ++++ b/stubs/Makefile.objs +@@ -44,3 +44,4 @@ stub-obj-y += xen-hvm.o + stub-obj-y += pci-host-piix.o + stub-obj-y += ram-block.o + stub-obj-y += ide-isa.o ++stub-obj-y += shadow-bios.o +diff --git a/stubs/shadow-bios.c b/stubs/shadow-bios.c +new file mode 100644 +index 0000000..c77cd7a +--- /dev/null ++++ b/stubs/shadow-bios.c +@@ -0,0 +1,7 @@ ++#include "qemu/osdep.h" ++#include "sysemu/sysemu.h" ++ ++void shadow_bios(void) ++{ ++ abort(); ++} +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index f483a71..a9db495 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -819,11 +819,17 @@ struct X86CPUDefinition { + + static X86CPUDefinition builtin_x86_defs[] = { + { ++ /* qemu64 is the default CPU model for all *-rhel7.* machine-types. ++ * The default on RHEL-6 was cpu64-rhel6. ++ * libvirt assumes that qemu64 is the default for _all_ machine-types, ++ * so we should try to keep qemu64 and cpu64-rhel6 as similar as ++ * possible. ++ */ + .name = "qemu64", + .level = 0xd, + .vendor = CPUID_VENDOR_AMD, + .family = 6, +- .model = 6, ++ .model = 13, + .stepping = 3, + .features[FEAT_1_EDX] = CPUID_SSE2 | CPUID_SSE | CPUID_FXSR | + CPUID_MMX | CPUID_CLFLUSH | CPUID_PSE36 | CPUID_PAT | CPUID_CMOV | +@@ -2097,6 +2103,7 @@ static PropValue kvm_default_props[] = { + { "acpi", "off" }, + { "monitor", "off" }, + { "svm", "off" }, ++ { "kvm-pv-unhalt", "on" }, + { NULL, NULL }, + }; + +diff --git a/target/i386/machine.c b/target/i386/machine.c +index bd2d82e..c9a3b5c 100644 +--- a/target/i386/machine.c ++++ b/target/i386/machine.c +@@ -893,6 +893,26 @@ static const VMStateDescription vmstate_msr_intel_pt = { + } + }; + ++static bool vmstate_xsave_needed(void *opaque) ++{ ++ /* The xsave state is already on the main "cpu" section */ ++ return false; ++} ++ ++static const VMStateDescription vmstate_xsave ={ ++ .name = "cpu/xsave", ++ .version_id = 1, ++ .minimum_version_id = 1, ++ .minimum_version_id_old = 1, ++ .needed = vmstate_xsave_needed, ++ .fields = (VMStateField []) { ++ VMSTATE_UINT64_V(env.xcr0, X86CPU, 1), ++ VMSTATE_UINT64_V(env.xstate_bv, X86CPU, 1), ++ VMSTATE_YMMH_REGS_VARS(env.xmm_regs, X86CPU, CPU_NB_REGS, 1), ++ VMSTATE_END_OF_LIST() ++ } ++}; ++ + VMStateDescription vmstate_x86_cpu = { + .name = "cpu", + .version_id = 12, +@@ -1015,6 +1035,7 @@ VMStateDescription vmstate_x86_cpu = { + &vmstate_spec_ctrl, + &vmstate_mcg_ext_ctl, + &vmstate_msr_intel_pt, ++ &vmstate_xsave, + NULL + } + }; +diff --git a/target/ppc/compat.c b/target/ppc/compat.c +index 807c906..33658fb 100644 +--- a/target/ppc/compat.c ++++ b/target/ppc/compat.c +@@ -105,6 +105,17 @@ static const CompatInfo *compat_by_pvr(uint32_t pvr) + return NULL; + } + ++long ppc_compat_cmp(uint32_t pvr1, uint32_t pvr2) ++{ ++ const CompatInfo *compat1 = compat_by_pvr(pvr1); ++ const CompatInfo *compat2 = compat_by_pvr(pvr2); ++ ++ g_assert(compat1); ++ g_assert(compat2); ++ ++ return compat1 - compat2; ++} ++ + bool ppc_check_compat(PowerPCCPU *cpu, uint32_t compat_pvr, + uint32_t min_compat_pvr, uint32_t max_compat_pvr) + { +diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h +index c621a6b..1932c2e 100644 +--- a/target/ppc/cpu.h ++++ b/target/ppc/cpu.h +@@ -1391,6 +1391,7 @@ static inline int cpu_mmu_index (CPUPPCState *env, bool ifetch) + + /* Compatibility modes */ + #if defined(TARGET_PPC64) ++long ppc_compat_cmp(uint32_t pvr1, uint32_t pvr2); + bool ppc_check_compat(PowerPCCPU *cpu, uint32_t compat_pvr, + uint32_t min_compat_pvr, uint32_t max_compat_pvr); + void ppc_set_compat(PowerPCCPU *cpu, uint32_t compat_pvr, Error **errp); +diff --git a/tests/Makefile.include b/tests/Makefile.include +index 3fd4706..0464e52 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -331,14 +331,14 @@ check-qtest-ppc-y += tests/boot-order-test$(EXESUF) + #check-qtest-ppc-y += tests/prom-env-test$(EXESUF) + check-qtest-ppc-y += tests/drive_del-test$(EXESUF) + check-qtest-ppc-y += tests/boot-serial-test$(EXESUF) +-check-qtest-ppc-y += tests/m48t59-test$(EXESUF) +-gcov-files-ppc-y += hw/timer/m48t59.c ++#check-qtest-ppc-y += tests/m48t59-test$(EXESUF) ++#gcov-files-ppc-y += hw/timer/m48t59.c + + check-qtest-ppc64-y = $(check-qtest-ppc-y) + gcov-files-ppc64-y = $(subst ppc-softmmu/,ppc64-softmmu/,$(gcov-files-ppc-y)) + check-qtest-ppc64-y += tests/spapr-phb-test$(EXESUF) + gcov-files-ppc64-y += ppc64-softmmu/hw/ppc/spapr_pci.c +-check-qtest-ppc64-y += tests/pnv-xscom-test$(EXESUF) ++#check-qtest-ppc64-y += tests/pnv-xscom-test$(EXESUF) + check-qtest-ppc64-y += tests/migration-test$(EXESUF) + check-qtest-ppc64-y += tests/rtas-test$(EXESUF) + check-qtest-ppc64-$(CONFIG_SLIRP) += tests/pxe-test$(EXESUF) +@@ -754,7 +754,7 @@ libqos-virtio-obj-y = $(libqos-spapr-obj-y) $(libqos-pc-obj-y) tests/libqos/virt + tests/qmp-test$(EXESUF): tests/qmp-test.o + tests/device-introspect-test$(EXESUF): tests/device-introspect-test.o + tests/rtc-test$(EXESUF): tests/rtc-test.o +-tests/m48t59-test$(EXESUF): tests/m48t59-test.o ++#tests/m48t59-test$(EXESUF): tests/m48t59-test.o + tests/endianness-test$(EXESUF): tests/endianness-test.o + tests/spapr-phb-test$(EXESUF): tests/spapr-phb-test.o $(libqos-obj-y) + #tests/prom-env-test$(EXESUF): tests/prom-env-test.o $(libqos-obj-y) +diff --git a/tests/cpu-plug-test.c b/tests/cpu-plug-test.c +index 8b5ab1f..112869b 100644 +--- a/tests/cpu-plug-test.c ++++ b/tests/cpu-plug-test.c +@@ -192,7 +192,8 @@ static void add_pseries_test_case(const char *mname) + PlugTestData *data; + + if (!g_str_has_prefix(mname, "pseries-") || +- (g_str_has_prefix(mname, "pseries-2.") && atoi(&mname[10]) < 7)) { ++ (g_str_has_prefix(mname, "pseries-2.") && atoi(&mname[10]) < 7) || ++ strcmp(mname,"pseries-rhel7.2.0") == 0) { + return; + } + data = g_new(PlugTestData, 1); +-- +1.8.3.1 + diff --git a/SOURCES/0004-block-vxhs-modularize-VXHS-via-g_module.patch b/SOURCES/0004-block-vxhs-modularize-VXHS-via-g_module.patch new file mode 100644 index 0000000..51cf039 --- /dev/null +++ b/SOURCES/0004-block-vxhs-modularize-VXHS-via-g_module.patch @@ -0,0 +1,515 @@ +From e58ce3910208fead9e24eb08e19a11bb2eba2f1e Mon Sep 17 00:00:00 2001 +From: "Danilo C. L. de Paula" +Date: Wed, 7 Mar 2018 13:05:43 -0300 +Subject: block/vxhs: modularize VXHS via g_module + +RH-Author: Jeffrey Cody +Message-id: <8a91a423440b7a5a14e868279c772e99b865bfc6.1494281291.git.jcody@redhat.com> +Patchwork-id: 75046 +O-Subject: [RHEV-7.4 qemu-kvm-rhev 4/4] block/vxhs: modularize VXHS via g_module Bugzilla: 1265869 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Laurent Vivier +RH-Acked-by: Miroslav Rezanina + +This converts the VXHS driver to be a runtime loaded module, with no +external build dependencies on libvxhs / libqnio. This (libvxhs) is a +3rd party library, written by Veritas, to interface with the VXHS +server. + +Red Hat is not going to distribute libvxhs, nor will Red Hat use libvxhs +in the build systems. So that creates two criteria for the +modularization, for business reasons: + +1. No runtime dependencies on libvxhs (aside from opening the library on +vxhs open) + +2. No build dependencies on libvxhs packages. + +There is support in QEMU for modular block drivers, however there are +two issues with using the built-in support: + + A. It is all-or-none; if modules are enabled all protocols are built + as modules. This wouldn't be that bad, as it would of course + enable more granular dependencies for qemu rpm packages. But... + + B. It is not designed with criteria #2 in mind; it reduces runtime + dependencies, not build dependencies. The protocol libraries + that are still built linked against external libraries and using + external headers. + +This patch uses g_module to load the libvxhs library, and incorporates +the libvxhs.h header in the build tree. If block driver protocols are +also built as modules, libvxhs will still work and be built as a module +too, except the shared library will still not have any dependency on +libvxhs. + +There are a few changes in this patch from upstream (aside from the +module loading aspects): + +1. It enables VXHS support to be built as a protocl module if +--enable-modules is used during configure. + +2. If the init call to iio_init() fails, populate errp with a +meaningful error message. + +3. Since we are loading the library dynamically, make check the min and +max supported versions in the libvxhs library on load. + +Patches for items #1 and #2 have been posted upstream. + +It is expected that the libvxhs library is located at the following +pathname: /usr/lib64/qemu/libvxhs.so.1 + +VXHS support is only built for x86_64 in RHEV. + +Signed-off-by: Jeff Cody +Signed-off-by: Miroslav Rezanina + +block/vxhs: improve error message for missing / bad vxhs module + +RH-Author: Jeffrey Cody +Message-id: <59af10d83125fff42beacd30dbca83d50409bbed.1513031708.git.jcody@redhat.com> +Patchwork-id: 78305 +O-Subject: [RHV7.5 qemu-kvm-rhev PATCH 1/1] block/vxhs: improve error message for missing / bad vxhs module +Bugzilla: 1505654 +RH-Acked-by: Markus Armbruster +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Laszlo Ersek + +[Downstream only, as the module load of libvxhs is downstream only] + +In the case of missing libvxhs libraries, the original error message, +while technically accurate, may lead a user to think there is a QEMU bug +if trying to using the VXHS protocol. Update the message so that it is +clear that the likely issue is that the Veritas QEMU libvxhs RPM is not +installed (or not installed correctly, if there are permission or file +corruption issues, etc.). + +An example error message before this change: + +> qemu-img info vxhs://localhost/test +qemu-img: Could not open 'vxhs://localhost/test': \ + error loading libvxhs: /usr/lib64/qemu/libvxhs.so.1: \ + cannot open shared object file: No such file or directory + +An example error message after this change: + +> qemu-img info vxhs://localhost/test +qemu-img: Could not open 'vxhs://localhost/test': \ + The VXHS library from Veritas might not be installed correctly \ + (/usr/lib64/qemu/libvxhs.so.1: \ + cannot open shared object file: No such file or directory) + +Signed-off-by: Jeff Cody +Signed-off-by: Miroslav Rezanina +--- + block/vxhs.c | 123 ++++++++++++++++++++++++++++++++---- + configure | 33 +--------- + include/block/vxhs_shim.h | 143 ++++++++++++++++++++++++++++++++++++++++++ + redhat/build_configure.sh | 9 ++- + redhat/qemu-kvm.spec.template | 7 +++ + 5 files changed, 273 insertions(+), 42 deletions(-) + create mode 100644 include/block/vxhs_shim.h + +diff --git a/block/vxhs.c b/block/vxhs.c +index 75cc6c8..68edb51 100644 +--- a/block/vxhs.c ++++ b/block/vxhs.c +@@ -9,7 +9,8 @@ + */ + + #include "qemu/osdep.h" +-#include ++#include "block/vxhs_shim.h" ++#include + #include + #include "block/block_int.h" + #include "qapi/qmp/qerror.h" +@@ -58,6 +59,97 @@ typedef struct BDRVVXHSState { + char *tlscredsid; /* tlscredsid */ + } BDRVVXHSState; + ++#define LIBVXHS_FULL_PATHNAME "/usr/lib64/qemu/libvxhs.so.1" ++static bool libvxhs_loaded; ++static GModule *libvxhs_handle; ++ ++static LibVXHSFuncs libvxhs; ++ ++typedef struct LibVXHSSymbols { ++ const char *name; ++ gpointer *addr; ++} LibVXHSSymbols; ++ ++static LibVXHSSymbols libvxhs_symbols[] = { ++ {"iio_init", (gpointer *) &libvxhs.iio_init}, ++ {"iio_fini", (gpointer *) &libvxhs.iio_fini}, ++ {"iio_min_version", (gpointer *) &libvxhs.iio_min_version}, ++ {"iio_max_version", (gpointer *) &libvxhs.iio_max_version}, ++ {"iio_open", (gpointer *) &libvxhs.iio_open}, ++ {"iio_close", (gpointer *) &libvxhs.iio_close}, ++ {"iio_writev", (gpointer *) &libvxhs.iio_writev}, ++ {"iio_readv", (gpointer *) &libvxhs.iio_readv}, ++ {"iio_ioctl", (gpointer *) &libvxhs.iio_ioctl}, ++ {NULL} ++}; ++ ++static void bdrv_vxhs_set_funcs(GModule *handle, Error **errp) ++{ ++ int i = 0; ++ while (libvxhs_symbols[i].name) { ++ const char *name = libvxhs_symbols[i].name; ++ if (!g_module_symbol(handle, name, libvxhs_symbols[i].addr)) { ++ error_setg(errp, "%s could not be loaded from libvxhs: %s", ++ name, g_module_error()); ++ return; ++ } ++ ++i; ++ } ++} ++ ++static void bdrv_vxhs_load_libs(Error **errp) ++{ ++ Error *local_err = NULL; ++ int32_t ver; ++ ++ if (libvxhs_loaded) { ++ return; ++ } ++ ++ if (!g_module_supported()) { ++ error_setg(errp, "modules are not supported on this platform: %s", ++ g_module_error()); ++ return; ++ } ++ ++ libvxhs_handle = g_module_open(LIBVXHS_FULL_PATHNAME, ++ G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL); ++ if (!libvxhs_handle) { ++ error_setg(errp, "The VXHS library from Veritas might not be installed " ++ "correctly (%s)", g_module_error()); ++ return; ++ } ++ ++ g_module_make_resident(libvxhs_handle); ++ ++ bdrv_vxhs_set_funcs(libvxhs_handle, &local_err); ++ if (local_err) { ++ error_propagate(errp, local_err); ++ return; ++ } ++ ++ /* Now check to see if the libvxhs we are using here is supported ++ * by the loaded version */ ++ ++ ver = (*libvxhs.iio_min_version)(); ++ if (ver > QNIO_VERSION) { ++ error_setg(errp, "Trying to use libvxhs version %"PRId32" API, but " ++ "only %"PRId32" or newer is supported by %s", ++ QNIO_VERSION, ver, LIBVXHS_FULL_PATHNAME); ++ return; ++ } ++ ++ ver = (*libvxhs.iio_max_version)(); ++ if (ver < QNIO_VERSION) { ++ error_setg(errp, "Trying to use libvxhs version %"PRId32" API, but " ++ "only %"PRId32" or earlier is supported by %s", ++ QNIO_VERSION, ver, LIBVXHS_FULL_PATHNAME); ++ return; ++ } ++ ++ libvxhs_loaded = true; ++} ++ + static void vxhs_complete_aio_bh(void *opaque) + { + VXHSAIOCB *acb = opaque; +@@ -219,7 +311,7 @@ static void vxhs_parse_filename(const char *filename, QDict *options, + static int vxhs_init_and_ref(void) + { + if (vxhs_ref++ == 0) { +- if (iio_init(QNIO_VERSION, vxhs_iio_callback)) { ++ if ((*libvxhs.iio_init)(QNIO_VERSION, vxhs_iio_callback)) { + return -ENODEV; + } + } +@@ -229,7 +321,7 @@ static int vxhs_init_and_ref(void) + static void vxhs_unref(void) + { + if (--vxhs_ref == 0) { +- iio_fini(); ++ (*libvxhs.iio_fini)(); + } + } + +@@ -299,8 +391,17 @@ static int vxhs_open(BlockDriverState *bs, QDict *options, + char *client_key = NULL; + char *client_cert = NULL; + ++ bdrv_vxhs_load_libs(&local_err); ++ if (local_err) { ++ error_propagate(errp, local_err); ++ /* on error, cannot cleanup because the iio_fini() function ++ * is not loaded */ ++ return -EINVAL; ++ } ++ + ret = vxhs_init_and_ref(); + if (ret < 0) { ++ error_setg(&local_err, "libvxhs iio_init() failed"); + ret = -EINVAL; + goto out; + } +@@ -385,8 +486,8 @@ static int vxhs_open(BlockDriverState *bs, QDict *options, + /* + * Open qnio channel to storage agent if not opened before + */ +- dev_handlep = iio_open(of_vsa_addr, s->vdisk_guid, 0, +- cacert, client_key, client_cert); ++ dev_handlep = (*libvxhs.iio_open)(of_vsa_addr, s->vdisk_guid, 0, ++ cacert, client_key, client_cert); + if (dev_handlep == NULL) { + trace_vxhs_open_iio_open(of_vsa_addr); + ret = -ENODEV; +@@ -450,12 +551,12 @@ static BlockAIOCB *vxhs_aio_rw(BlockDriverState *bs, int64_t sector_num, + + switch (iodir) { + case VDISK_AIO_WRITE: +- ret = iio_writev(dev_handle, acb, qiov->iov, qiov->niov, +- offset, (uint64_t)size, iio_flags); ++ ret = (*libvxhs.iio_writev)(dev_handle, acb, qiov->iov, qiov->niov, ++ offset, (uint64_t)size, iio_flags); + break; + case VDISK_AIO_READ: +- ret = iio_readv(dev_handle, acb, qiov->iov, qiov->niov, +- offset, (uint64_t)size, iio_flags); ++ ret = (*libvxhs.iio_readv)(dev_handle, acb, qiov->iov, qiov->niov, ++ offset, (uint64_t)size, iio_flags); + break; + default: + trace_vxhs_aio_rw_invalid(iodir); +@@ -505,7 +606,7 @@ static void vxhs_close(BlockDriverState *bs) + * Close vDisk device + */ + if (s->vdisk_hostinfo.dev_handle) { +- iio_close(s->vdisk_hostinfo.dev_handle); ++ (*libvxhs.iio_close)(s->vdisk_hostinfo.dev_handle); + s->vdisk_hostinfo.dev_handle = NULL; + } + +@@ -527,7 +628,7 @@ static int64_t vxhs_get_vdisk_stat(BDRVVXHSState *s) + int ret = 0; + void *dev_handle = s->vdisk_hostinfo.dev_handle; + +- ret = iio_ioctl(dev_handle, IOR_VDISK_STAT, &vdisk_size, 0); ++ ret = (*libvxhs.iio_ioctl)(dev_handle, IOR_VDISK_STAT, &vdisk_size, 0); + if (ret < 0) { + trace_vxhs_get_vdisk_stat_err(s->vdisk_guid, ret, errno); + return -EIO; +diff --git a/configure b/configure +index 0a19b03..7358269 100755 +--- a/configure ++++ b/configure +@@ -3369,7 +3369,7 @@ else + glib_req_ver=2.22 + fi + glib_modules=gthread-2.0 +-if test "$modules" = yes; then ++if test "$modules" = yes -o "$vxhs" = yes; then + glib_modules="$glib_modules gmodule-export-2.0" + fi + +@@ -5314,33 +5314,6 @@ if compile_prog "" "" ; then + fi + + ########################################## +-# Veritas HyperScale block driver VxHS +-# Check if libvxhs is installed +- +-if test "$vxhs" != "no" ; then +- cat > $TMPC < +-#include +- +-void *vxhs_callback; +- +-int main(void) { +- iio_init(QNIO_VERSION, vxhs_callback); +- return 0; +-} +-EOF +- vxhs_libs="-lvxhs -lssl" +- if compile_prog "" "$vxhs_libs" ; then +- vxhs=yes +- else +- if test "$vxhs" = "yes" ; then +- feature_not_found "vxhs block device" "Install libvxhs See github" +- fi +- vxhs=no +- fi +-fi +- +-########################################## + # check for _Static_assert() + + have_static_assert=no +@@ -6614,8 +6587,8 @@ if test "$pthread_setname_np" = "yes" ; then + fi + + if test "$vxhs" = "yes" ; then +- echo "CONFIG_VXHS=y" >> $config_host_mak +- echo "VXHS_LIBS=$vxhs_libs" >> $config_host_mak ++ echo "CONFIG_VXHS=m" >> $config_host_mak ++ echo "VXHS_LIBS= -lssl" >> $config_host_mak + fi + + if test "$tcg_interpreter" = "yes"; then +diff --git a/include/block/vxhs_shim.h b/include/block/vxhs_shim.h +new file mode 100644 +index 0000000..42519ae +--- /dev/null ++++ b/include/block/vxhs_shim.h +@@ -0,0 +1,143 @@ ++/* ++ * Network IO library for VxHS QEMU block driver (Veritas Technologies) ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2. See ++ * the COPYING file in the top-level directory. ++ * ++ * Contributions after 2014-08-15 are licensed under the terms of the ++ * GNU GPL, version 2 or (at your option) any later version. ++ */ ++ ++#ifndef QNIO_API_H ++#define QNIO_API_H ++ ++#include ++ ++/* ++ * Bump up the version everytime this file is modified ++ */ ++#define QNIO_VERSION 34 ++ ++/* ++ * These are the opcodes referenced by callback routine. ++ */ ++#define IRP_READ_REQUEST 0x1FFF ++#define IRP_WRITE_REQUEST 0x2FFF ++#define IRP_VDISK_CHECK_IO_FAILOVER_READY 2020 ++ ++/* ++ * opcodes for iio_ioctl. ++ */ ++#define IOR_VDISK_STAT 1005 ++ ++/* ++ * Error values for iio_cb_t callback function. ++ */ ++#define QNIOERROR_HUP 901 /* Retriable error */ ++#define QNIOERROR_NOCONN 902 /* Non-retriable error */ ++ ++ ++/* Operation Flags */ ++#define IIO_FLAG_ASYNC 0x0001 /* Do an async send */ ++ ++/* ++ * INPUT: ++ * ctx - opaque context ++ * opcode - Operation ++ * error - 0 for sucess, non-zero for failure. ++ * RETURNS: ++ * void ++ * DESCRIPTION: ++ * This callback is called, after Async request completes. ++ * ++ * CONTEXT: ++ * The callback should be wait-free. ++ */ ++typedef void (*iio_cb_t) (void *ctx, uint32_t opcode, uint32_t error); ++ ++typedef struct LibVXHSFuncs { ++/* ++ * RETURNS: ++ * 0 for sucess, non-zero for failure. ++ * DESCRIPTION: ++ * Intilize the library state. This should be called at the ++ * begining before issuing any library call. ++ */ ++ int (*iio_init)(int32_t version, iio_cb_t cb); ++/* ++ * RETURNS: ++ * void ++ * DESCRIPTION: ++ * Relinquish library resources. This should be called on the ++ * close of last open device. ++ */ ++ void (*iio_fini)(void); ++/* ++ * DESCRIPTION: ++ * Returns minimum QNIO API version supported by library. ++ */ ++ int32_t (*iio_min_version)(void); ++/* ++ * DESCRIPTION: ++ * Returns maximum QNIO API version supported by library. ++ */ ++ int32_t (*iio_max_version)(void); ++/* ++ * INPUT: ++ * uri - const string of the format of://:port ++ * devid - Device ID. ++ * flags - currently unused, this must be set to 0 ++ * cacert - CA certificates file in PEM format ++ * client_key - Client private key file in PEM format ++ * client_cert - Client certificate file in PEM format ++ * RETURNS: ++ * opeque device handle on success, NULL on failure. ++ * DESCRIPTION: ++ * This call returns device handle on success. Returns NULL on ++ * failure with errno set ++ * errno can be one of: ++ * ENODEV - remote device not found ++ * EBADF - Unable to open communication channel. ++ * EBUSY - The call cannot be completed right now ++ */ ++ void *(*iio_open)(const char *uri, const char *devid, uint32_t flags, ++ const char *cacert, const char *client_key, ++ const char *client_cert); ++/* ++ * Close the device. ++ * For every matching iio_open() there should be a matching iio_close() ++ * The last close free all data structures associated with the device. ++ */ ++ int32_t (*iio_close)(void *dev_handle); ++/* ++ * INPUT: ++ * dev_handle - device descriptor on which read/write needs to be performed ++ * ctx - an opaque context that is not interpreted This is set for ++ * async calls only. It can be NULL. ++ * iov - an array of iovecs (This is a scatter gather operation) ++ * iovcnt - the number of iovecs ++ * offset - an offset to perform the write ++ * size - I/O size ++ * flags - can be one of ++ * IIO_FLAG_ASYNC - indicating this is a aio call. ++ * RETURNS: ++ * -1 on error, sets errno ++ * EBADF - the remote fd is bad ++ * EBUSY - The call cannot be completed right now ++ * EPIPE - the channel got disconnected, call back would be called in ++ * addition to this. ++ */ ++ int32_t (*iio_writev)(void *dev_handle, void *ctx, struct iovec *iov, ++ int iovcnt, uint64_t offset, uint64_t size, ++ uint32_t flags); ++ ++ int32_t (*iio_readv)(void *dev_handle, void *ctx, struct iovec *iov, ++ int iovcnt, uint64_t offset, uint64_t size, ++ uint32_t flags); ++ ++ int32_t (*iio_ioctl)(void *dev_handle, uint32_t opcode, void *opaque, ++ uint32_t flags); ++ ++} LibVXHSFuncs; ++ ++#endif diff --git a/SOURCES/0005-Use-kvm-by-default.patch b/SOURCES/0005-Use-kvm-by-default.patch new file mode 100644 index 0000000..51111c3 --- /dev/null +++ b/SOURCES/0005-Use-kvm-by-default.patch @@ -0,0 +1,41 @@ +From a737b4e82a67c87c6c34bbe5826dc9ed5c6318da Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 18 Dec 2014 06:27:49 +0100 +Subject: Use kvm by default + +Bugzilla: 906185 + +RHEL uses kvm accelerator by default, if available. + +Signed-off-by: Miroslav Rezanina + +Rebase notes (2.10.0) +- variable rename (upstream) + +Rebase notes (2.2.0): +- Move code from vl.c to accel.c + +(cherry picked from commit abcd662eb8e516ebe4a6b401e83a62f749491a15) +(cherry picked from commit eca6d5766d956c37e3f7f28d70903d357308c846) +--- + accel/accel.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/accel/accel.c b/accel/accel.c +index 93e2434..5f3d73f 100644 +--- a/accel/accel.c ++++ b/accel/accel.c +@@ -79,8 +79,8 @@ void configure_accelerator(MachineState *ms) + + accel = qemu_opt_get(qemu_get_machine_opts(), "accel"); + if (accel == NULL) { +- /* Use the default "accelerator", tcg */ +- accel = "tcg"; ++ /* RHEL uses kvm as the default accelerator, fallback to tcg */ ++ accel = "kvm:tcg"; + } + + p = accel; +-- +1.8.3.1 + diff --git a/SOURCES/0006-vfio-cap-number-of-devices-that-can-be-assigned.patch b/SOURCES/0006-vfio-cap-number-of-devices-that-can-be-assigned.patch new file mode 100644 index 0000000..8f00d28 --- /dev/null +++ b/SOURCES/0006-vfio-cap-number-of-devices-that-can-be-assigned.patch @@ -0,0 +1,74 @@ +From caa475eb19e0e235920f15828c6879c4b64499a2 Mon Sep 17 00:00:00 2001 +From: Bandan Das +Date: Tue, 3 Dec 2013 20:05:13 +0100 +Subject: vfio: cap number of devices that can be assigned + +RH-Author: Bandan Das +Message-id: <1386101113-31560-3-git-send-email-bsd@redhat.com> +Patchwork-id: 55984 +O-Subject: [PATCH RHEL7 qemu-kvm v2 2/2] vfio: cap number of devices that can be assigned +Bugzilla: 678368 +RH-Acked-by: Alex Williamson +RH-Acked-by: Marcelo Tosatti +RH-Acked-by: Michael S. Tsirkin + +Go through all groups to get count of total number of devices +active to enforce limit + +Reasoning from Alex for the limit(32) - Assuming 3 slots per +device, with 125 slots (number of memory slots for RHEL 7), +we can support almost 40 devices and still have few slots left +for other uses. Stepping down a bit, the number 32 arbitrarily +matches the number of slots on a PCI bus and is also a nice power +of two. + +Signed-off-by: Bandan Das + +Rebase notes (2.8.0): +- removed return value for vfio_realize (commit 1a22aca) + +Merged patches (2.9.0): +- 17eb774 vfio: Use error_setg when reporting max assigned device overshoot + +(cherry picked from commit 9fa3c9fc6dfcde76d80db1aa601b2d577f72ceec) +(cherry picked from commit 3cb35556dc7d994f203d732fe952f95fcdb03c0a) +--- + hw/vfio/pci.c | 15 ++++++++++++++- + 1 file changed, 14 insertions(+), 1 deletion(-) + +diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c +index b9bc6cd..34b9d19 100644 +--- a/hw/vfio/pci.c ++++ b/hw/vfio/pci.c +@@ -35,6 +35,7 @@ + #include "qapi/error.h" + + #define MSIX_CAP_LENGTH 12 ++#define MAX_DEV_ASSIGN_CMDLINE 32 + + static void vfio_disable_interrupts(VFIOPCIDevice *vdev); + static void vfio_mmap_set_enabled(VFIOPCIDevice *vdev, bool enabled); +@@ -2807,7 +2808,19 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) + ssize_t len; + struct stat st; + int groupid; +- int i, ret; ++ int ret, i = 0; ++ ++ QLIST_FOREACH(group, &vfio_group_list, next) { ++ QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { ++ i++; ++ } ++ } ++ ++ if (i >= MAX_DEV_ASSIGN_CMDLINE) { ++ error_setg(errp, "Maximum supported vfio devices (%d) " ++ "already attached", MAX_DEV_ASSIGN_CMDLINE); ++ return; ++ } + + if (!vdev->vbasedev.sysfsdev) { + if (!(~vdev->host.domain || ~vdev->host.bus || +-- +1.8.3.1 + diff --git a/SOURCES/0007-Add-support-statement-to-help-output.patch b/SOURCES/0007-Add-support-statement-to-help-output.patch new file mode 100644 index 0000000..ca68bfb --- /dev/null +++ b/SOURCES/0007-Add-support-statement-to-help-output.patch @@ -0,0 +1,57 @@ +From 9fad36498006352be79a39ca3428079b6b7ddcc9 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Wed, 4 Dec 2013 18:53:17 +0100 +Subject: Add support statement to -help output + +RH-Author: Eduardo Habkost +Message-id: <1386183197-27761-1-git-send-email-ehabkost@redhat.com> +Patchwork-id: 55994 +O-Subject: [qemu-kvm RHEL7 PATCH] Add support statement to -help output +Bugzilla: 972773 +RH-Acked-by: Miroslav Rezanina +RH-Acked-by: knoel@redhat.com +RH-Acked-by: Paolo Bonzini + +Add support statement to -help output, reporting direct qemu-kvm usage +as unsupported by Red Hat, and advising users to use libvirt instead. + +Signed-off-by: Eduardo Habkost +(cherry picked from commit 2a07700936e39856cc9f149c6a6517f0715536a6) +(cherry picked from commit 5dd2f4706e2fef945771949e59a8fcc1b5452de9) +--- + vl.c | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/vl.c b/vl.c +index 03950fc..8c89bee 100644 +--- a/vl.c ++++ b/vl.c +@@ -1953,9 +1953,17 @@ static void version(void) + QEMU_COPYRIGHT "\n"); + } + ++static void print_rh_warning(void) ++{ ++ printf("\nWARNING: Direct use of qemu-kvm from the command line is not supported by Red Hat.\n" ++ "WARNING: Use libvirt as the stable management interface.\n" ++ "WARNING: Some command line options listed here may not be available in future releases.\n\n"); ++} ++ + static void help(int exitcode) + { + version(); ++ print_rh_warning(); + printf("usage: %s [options] [disk_image]\n\n" + "'disk_image' is a raw hard disk image for IDE hard disk 0\n\n", + error_get_progname()); +@@ -1972,6 +1980,7 @@ static void help(int exitcode) + "\n" + QEMU_HELP_BOTTOM "\n"); + ++ print_rh_warning(); + exit(exitcode); + } + +-- +1.8.3.1 + diff --git a/SOURCES/0008-globally-limit-the-maximum-number-of-CPUs.patch b/SOURCES/0008-globally-limit-the-maximum-number-of-CPUs.patch new file mode 100644 index 0000000..7d3bc2f --- /dev/null +++ b/SOURCES/0008-globally-limit-the-maximum-number-of-CPUs.patch @@ -0,0 +1,151 @@ +From 9324eac6e41aa7667042e117dc3581859cecbf5f Mon Sep 17 00:00:00 2001 +From: Andrew Jones +Date: Tue, 21 Jan 2014 10:46:52 +0100 +Subject: globally limit the maximum number of CPUs + +We now globally limit the number of VCPUs. +Especially, there is no way one can specify more than +max_cpus VCPUs for a VM. + +This allows us the restore the ppc max_cpus limitation to the upstream +default and minimize the ppc hack in kvm-all.c. + +Signed-off-by: David Hildenbrand +Signed-off-by: Miroslav Rezanina +Signed-off-by: Danilo Cesar Lemes de Paula + +Rebase notes (2.11.0): +- Removed CONFIG_RHV reference +- Update commit log + +Merged patches (2.11.0): +- 92fef14623 redhat: remove manual max_cpus limitations for ppc +- bb722e9eff redhat: globally limit the maximum number of CPUs +- fdeef3c1c7 RHEL: Set vcpus hard limit to 240 for Power +- 0584216921 Match POWER max cpus to x86 + +Signed-off-by: Andrew Jones +(cherry picked from commit a4ceb63bdc5cbac19f5f633ec761b9de0dedb55e) +(cherry picked from commit a1f26d85171b4d554225150053700e93ba6eba10) + +redhat: globally limit the maximum number of CPUs + +RH-Author: David Hildenbrand +Message-id: <20180109103253.24517-2-david@redhat.com> +Patchwork-id: 78531 +O-Subject: [RHEL-7.5 qemu-kvm-ma PATCH v2 1/2] redhat: globally limit the maximum number of CPUs +Bugzilla: 1527449 +RH-Acked-by: David Gibson +RH-Acked-by: Thomas Huth +RH-Acked-by: Cornelia Huck + +Upstream-status: n/a + +For RHEL, we support 240, for RHV up to 384 VCPUs. Let's limit this +globally instead of fixing up all machines. This way, we can easily +change (increase) the product specific levels later. + +Signed-off-by: David Hildenbrand +Signed-off-by: Miroslav Rezanina + +redhat: remove manual max_cpus limitations for ppc + +RH-Author: David Hildenbrand +Message-id: <20180109103253.24517-3-david@redhat.com> +Patchwork-id: 78532 +O-Subject: [RHEL-7.5 qemu-kvm-ma PATCH v2 2/2] redhat: remove manual max_cpus limitations for ppc +Bugzilla: 1527449 +RH-Acked-by: David Gibson +RH-Acked-by: Thomas Huth +RH-Acked-by: Cornelia Huck + +Upstream-status: n/a + +RH-Author: Andrew Jones +Message-id: <1390301212-15344-1-git-send-email-drjones@redhat.com> +Patchwork-id: 56862 +O-Subject: [RHEL7.0 qemu-kvm PATCH v6] use recommended max vcpu count +Bugzilla: 998708 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Marcelo Tosatti + +The recommended vcpu max limit (KVM_CAP_NR_VCPUS) should be used instead +of the actual max vcpu limit (KVM_CAP_MAX_VCPUS) to give an error. + +This commit matches the limit to current KVM_CAP_NR_VCPUS value. +--- + accel/kvm/kvm-all.c | 12 ++++++++++++ + vl.c | 19 +++++++++++++++++++ + 2 files changed, 31 insertions(+) + +diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c +index ffee68e..3f1c06e 100644 +--- a/accel/kvm/kvm-all.c ++++ b/accel/kvm/kvm-all.c +@@ -1587,6 +1587,18 @@ static int kvm_init(MachineState *ms) + soft_vcpus_limit = kvm_recommended_vcpus(s); + hard_vcpus_limit = kvm_max_vcpus(s); + ++#ifdef HOST_PPC64 ++ /* ++ * On POWER, the kernel advertises a soft limit based on the ++ * number of CPU threads on the host. We want to allow exceeding ++ * this for testing purposes, so we don't want to set hard limit ++ * to soft limit as on x86. ++ */ ++#else ++ /* RHEL doesn't support nr_vcpus > soft_vcpus_limit */ ++ hard_vcpus_limit = soft_vcpus_limit; ++#endif ++ + while (nc->name) { + if (nc->num > soft_vcpus_limit) { + warn_report("Number of %s cpus requested (%d) exceeds " +diff --git a/vl.c b/vl.c +index 8c89bee..ce7d04d 100644 +--- a/vl.c ++++ b/vl.c +@@ -135,6 +135,8 @@ int main(int argc, char **argv) + #define MAX_VIRTIO_CONSOLES 1 + #define MAX_SCLP_CONSOLES 1 + ++#define RHEL_MAX_CPUS 384 ++ + static const char *data_dir[16]; + static int data_dir_idx; + const char *bios_name = NULL; +@@ -1520,6 +1522,20 @@ MachineClass *find_default_machine(void) + return mc; + } + ++/* Maximum number of CPUs limited for Red Hat Enterprise Linux */ ++static void limit_max_cpus_in_machines(void) ++{ ++ GSList *el, *machines = object_class_get_list(TYPE_MACHINE, false); ++ ++ for (el = machines; el; el = el->next) { ++ MachineClass *mc = el->data; ++ ++ if (mc->max_cpus > RHEL_MAX_CPUS) { ++ mc->max_cpus = RHEL_MAX_CPUS; ++ } ++ } ++} ++ + MachineInfoList *qmp_query_machines(Error **errp) + { + GSList *el, *machines = object_class_get_list(TYPE_MACHINE, false); +@@ -4082,6 +4098,9 @@ int main(int argc, char **argv, char **envp) + + replay_configure(icount_opts); + ++ /* Maximum number of CPUs limited for Red Hat Enterprise Linux */ ++ limit_max_cpus_in_machines(); ++ + machine_class = select_machine(); + + set_memory_options(&ram_slots, &maxram_size, machine_class); +-- +1.8.3.1 + diff --git a/SOURCES/0009-Add-support-for-simpletrace.patch b/SOURCES/0009-Add-support-for-simpletrace.patch new file mode 100644 index 0000000..c2796b3 --- /dev/null +++ b/SOURCES/0009-Add-support-for-simpletrace.patch @@ -0,0 +1,118 @@ +From e10de328869f0b7b990b74863111c172fb45d7a4 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 8 Oct 2015 09:50:17 +0200 +Subject: Add support for simpletrace + +As simpletrace is upstream, we just need to properly handle it during rpmbuild. + +Signed-off-by: Miroslav Rezanina + +Rebase notes (2.9.0): +- Added group argument for tracetool.py (upstream) + +Rebase notes (2.8.0): +- Changed tracetool.py parameters + +Merged patches (2.3.0): +- db959d6 redhat/qemu-kvm.spec.template: Install qemu-kvm-simpletrace.stp +- 5292fc3 trace: add SystemTap init scripts for simpletrace bridge +- eda9e5e simpletrace: install simpletrace.py +- 85c4c8f trace: add systemtap-initscript README file to RPM + +(cherry picked from commit bfc1d7f3628f2ffbabbae71d57a506cea6663ddf) +--- + .gitignore | 2 ++ + Makefile | 4 +++ + README.systemtap | 43 +++++++++++++++++++++++++++++++++ + redhat/qemu-kvm.spec.template | 27 +++++++++++++++++++-- + scripts/systemtap/conf.d/qemu_kvm.conf | 4 +++ + scripts/systemtap/script.d/qemu_kvm.stp | 1 + + 6 files changed, 79 insertions(+), 2 deletions(-) + create mode 100644 README.systemtap + create mode 100644 scripts/systemtap/conf.d/qemu_kvm.conf + create mode 100644 scripts/systemtap/script.d/qemu_kvm.stp + +diff --git a/Makefile b/Makefile +index 89ba4c5..d0a848e 100644 +--- a/Makefile ++++ b/Makefile +@@ -864,6 +864,10 @@ endif + $(INSTALL_DATA) $(SRC_PATH)/pc-bios/keymaps/$$x "$(DESTDIR)$(qemu_datadir)/keymaps"; \ + done + $(INSTALL_DATA) $(BUILD_DIR)/trace-events-all "$(DESTDIR)$(qemu_datadir)/trace-events-all" ++ $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)/systemtap/script.d" ++ $(INSTALL_DATA) $(SRC_PATH)/scripts/systemtap/script.d/qemu_kvm.stp "$(DESTDIR)$(qemu_datadir)/systemtap/script.d/" ++ $(INSTALL_DIR) "$(DESTDIR)$(qemu_datadir)/systemtap/conf.d" ++ $(INSTALL_DATA) $(SRC_PATH)/scripts/systemtap/conf.d/qemu_kvm.conf "$(DESTDIR)$(qemu_datadir)/systemtap/conf.d/" + for d in $(TARGET_DIRS); do \ + $(MAKE) $(SUBDIR_MAKEFLAGS) TARGET_DIR=$$d/ -C $$d $@ || exit 1 ; \ + done +diff --git a/README.systemtap b/README.systemtap +new file mode 100644 +index 0000000..ad913fc +--- /dev/null ++++ b/README.systemtap +@@ -0,0 +1,43 @@ ++QEMU tracing using systemtap-initscript ++--------------------------------------- ++ ++You can capture QEMU trace data all the time using systemtap-initscript. This ++uses SystemTap's flight recorder mode to trace all running guests to a ++fixed-size buffer on the host. Old trace entries are overwritten by new ++entries when the buffer size wraps. ++ ++1. Install the systemtap-initscript package: ++ # yum install systemtap-initscript ++ ++2. Install the systemtap scripts and the conf file: ++ # cp /usr/share/qemu-kvm/systemtap/script.d/qemu_kvm.stp /etc/systemtap/script.d/ ++ # cp /usr/share/qemu-kvm/systemtap/conf.d/qemu_kvm.conf /etc/systemtap/conf.d/ ++ ++The set of trace events to enable is given in qemu_kvm.stp. This SystemTap ++script can be customized to add or remove trace events provided in ++/usr/share/systemtap/tapset/qemu-kvm-simpletrace.stp. ++ ++SystemTap customizations can be made to qemu_kvm.conf to control the flight ++recorder buffer size and whether to store traces in memory only or disk too. ++See stap(1) for option documentation. ++ ++3. Start the systemtap service. ++　# service systemtap start qemu_kvm ++ ++4. Make the service start at boot time. ++　# chkconfig systemtap on ++ ++5. Confirm that the service works. ++ # service systemtap status qemu_kvm ++ qemu_kvm is running... ++ ++When you want to inspect the trace buffer, perform the following steps: ++ ++1. Dump the trace buffer. ++ # staprun -A qemu_kvm >/tmp/trace.log ++ ++2. Start the systemtap service because the preceding step stops the service. ++ # service systemtap start qemu_kvm ++ ++3. Translate the trace record to readable format. ++ # /usr/share/qemu-kvm/simpletrace.py --no-header /usr/share/qemu-kvm/trace-events /tmp/trace.log +diff --git a/scripts/systemtap/conf.d/qemu_kvm.conf b/scripts/systemtap/conf.d/qemu_kvm.conf +new file mode 100644 +index 0000000..372d816 +--- /dev/null ++++ b/scripts/systemtap/conf.d/qemu_kvm.conf +@@ -0,0 +1,4 @@ ++# Force load uprobes (see BZ#1118352) ++stap -e 'probe process("/usr/libexec/qemu-kvm").function("main") { printf("") }' -c true ++ ++qemu_kvm_OPT="-s4" # per-CPU buffer size, in megabytes +diff --git a/scripts/systemtap/script.d/qemu_kvm.stp b/scripts/systemtap/script.d/qemu_kvm.stp +new file mode 100644 +index 0000000..c04abf9 +--- /dev/null ++++ b/scripts/systemtap/script.d/qemu_kvm.stp +@@ -0,0 +1 @@ ++probe qemu.kvm.simpletrace.handle_qmp_command,qemu.kvm.simpletrace.monitor_protocol_*,qemu.kvm.simpletrace.migrate_set_state {} +-- +1.8.3.1 + diff --git a/SOURCES/0010-Use-qemu-kvm-in-documentation-instead-of-qemu-system.patch b/SOURCES/0010-Use-qemu-kvm-in-documentation-instead-of-qemu-system.patch new file mode 100644 index 0000000..79a2bfe --- /dev/null +++ b/SOURCES/0010-Use-qemu-kvm-in-documentation-instead-of-qemu-system.patch @@ -0,0 +1,943 @@ +From 9d7996484c665193e02927bb76ba93c84efb273f Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Fri, 14 Nov 2014 08:51:50 +0100 +Subject: Use qemu-kvm in documentation instead of qemu-system- + +Patchwork-id: 62380 +O-Subject: [RHEV-7.1 qemu-kvm-rhev PATCHv4] Use qemu-kvm in documentation instead of qemu-system-i386 +Bugzilla: 1140620 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Markus Armbruster +RH-Acked-by: Stefan Hajnoczi + +From: Miroslav Rezanina + +We change the name and location of qemu-kvm binaries. Update documentation +to reflect this change. Only architectures available in RHEL are updated. + +Signed-off-by: Miroslav Rezanina + +Rebase Notes (2.12.0): +- Additional fixes included + +Rebase Notes (2.11.0): +- Fixing docs/qemu-block-drivers.texi + +Rebase Notes (2.10.0): +- Changed patch name and updated commit message. + +Rebase Notes (2.9.0): +- fixed chunks missed on 2.8 rebase + +(cherry picked from commit 1c2dac56d5e710faebe25b7aa9cac594ec0f9d4b) +(cherry picked from commit dfa2037d390047a7d7c7b13f779443bfc6c3709d) + +Conflicts: + qemu-options.hx + +(cherry picked from commit c7985367ba8258c99526549ab94ef066ae52da14) + +Conflicts: + qemu-options.hx + +(cherry picked from commit e7dc2155506c1ead844f1faef85e5f71bc2adf9e) +--- + docs/can.txt | 10 +-- + docs/pr-manager.rst | 4 +- + docs/qemu-block-drivers.texi | 70 ++++++++++----------- + docs/replay.txt | 4 +- + docs/specs/tpm.txt | 4 +- + qemu-doc.texi | 40 ++++++------ + qemu-options.hx | 144 ++++++++++++++++++++++--------------------- + 7 files changed, 140 insertions(+), 136 deletions(-) + +diff --git a/docs/can.txt b/docs/can.txt +index a357105..0c0fc11 100644 +--- a/docs/can.txt ++++ b/docs/can.txt +@@ -50,9 +50,9 @@ CAN boards can be selected + The ''kvaser_pci'' board/device model is compatible with and has been tested with + ''kvaser_pci'' driver included in mainline Linux kernel. + The tested setup was Linux 4.9 kernel on the host and guest side. +-Example for qemu-system-x86_64: ++Example for qemu-kvm (intel architecture): + +- qemu-system-x86_64 -enable-kvm -kernel /boot/vmlinuz-4.9.0-4-amd64 \ ++ qemu-kvm -enable-kvm -kernel /boot/vmlinuz-4.9.0-4-amd64 \ + -initrd ramdisk.cpio \ + -virtfs local,path=shareddir,security_model=none,mount_tag=shareddir \ + -object can-bus,id=canbus0 \ +@@ -60,9 +60,9 @@ Example for qemu-system-x86_64: + -device kvaser_pci,canbus=canbus0 \ + -nographic -append "console=ttyS0" + +-Example for qemu-system-arm: ++Example for qemu-kvm (arm architecture): + +- qemu-system-arm -cpu arm1176 -m 256 -M versatilepb \ ++ qemu-kvm -cpu arm1176 -m 256 -M versatilepb \ + -kernel kernel-qemu-arm1176-versatilepb \ + -hda rpi-wheezy-overlay \ + -append "console=ttyAMA0 root=/dev/sda2 ro init=/sbin/init-overlay" \ +@@ -104,4 +104,4 @@ Links to other resources + Slides + http://rtime.felk.cvut.cz/publications/public/rtlws2015-qemu-can-slides.pdf + (5) Linux SocketCAN utilities +- https://github.com/linux-can/can-utils/ +\ No newline at end of file ++ https://github.com/linux-can/can-utils/ +diff --git a/docs/pr-manager.rst b/docs/pr-manager.rst +index 9b1de19..45cb8be 100644 +--- a/docs/pr-manager.rst ++++ b/docs/pr-manager.rst +@@ -36,7 +36,7 @@ accepts the path to the helper program's Unix socket. For example, + the following command line defines a ``pr-manager-helper`` object and + attaches it to a SCSI passthrough device:: + +- $ qemu-system-x86_64 ++ $ qemu-kvm + -device virtio-scsi \ + -object pr-manager-helper,id=helper0,path=/var/run/qemu-pr-helper.sock + -drive if=none,id=hd,driver=raw,file.filename=/dev/sdb,file.pr-manager=helper0 +@@ -44,7 +44,7 @@ attaches it to a SCSI passthrough device:: + + Alternatively, using ``-blockdev``:: + +- $ qemu-system-x86_64 ++ $ qemu-kvm + -device virtio-scsi \ + -object pr-manager-helper,id=helper0,path=/var/run/qemu-pr-helper.sock + -blockdev node-name=hd,driver=raw,file.driver=host_device,file.filename=/dev/sdb,file.pr-manager=helper0 +diff --git a/docs/qemu-block-drivers.texi b/docs/qemu-block-drivers.texi +index f179369..e0d752a 100644 +--- a/docs/qemu-block-drivers.texi ++++ b/docs/qemu-block-drivers.texi +@@ -405,7 +405,7 @@ QEMU can automatically create a virtual FAT disk image from a + directory tree. In order to use it, just type: + + @example +-qemu-system-i386 linux.img -hdb fat:/my_directory ++qemu-kvm linux.img -hdb fat:/my_directory + @end example + + Then you access access to all the files in the @file{/my_directory} +@@ -415,14 +415,14 @@ them via SAMBA or NFS. The default access is @emph{read-only}. + Floppies can be emulated with the @code{:floppy:} option: + + @example +-qemu-system-i386 linux.img -fda fat:floppy:/my_directory ++qemu-kvm linux.img -fda fat:floppy:/my_directory + @end example + + A read/write support is available for testing (beta stage) with the + @code{:rw:} option: + + @example +-qemu-system-i386 linux.img -fda fat:floppy:rw:/my_directory ++qemu-kvm linux.img -fda fat:floppy:rw:/my_directory + @end example + + What you should @emph{never} do: +@@ -440,14 +440,14 @@ QEMU can access directly to block device exported using the Network Block Device + protocol. + + @example +-qemu-system-i386 linux.img -hdb nbd://my_nbd_server.mydomain.org:1024/ ++qemu-kvm linux.img -hdb nbd://my_nbd_server.mydomain.org:1024/ + @end example + + If the NBD server is located on the same host, you can use an unix socket instead + of an inet socket: + + @example +-qemu-system-i386 linux.img -hdb nbd+unix://?socket=/tmp/my_socket ++qemu-kvm linux.img -hdb nbd+unix://?socket=/tmp/my_socket + @end example + + In this case, the block device must be exported using qemu-nbd: +@@ -464,23 +464,23 @@ qemu-nbd --socket=/tmp/my_socket --share=2 my_disk.qcow2 + @noindent + and then you can use it with two guests: + @example +-qemu-system-i386 linux1.img -hdb nbd+unix://?socket=/tmp/my_socket +-qemu-system-i386 linux2.img -hdb nbd+unix://?socket=/tmp/my_socket ++qemu-kvm linux1.img -hdb nbd+unix://?socket=/tmp/my_socket ++qemu-kvm linux2.img -hdb nbd+unix://?socket=/tmp/my_socket + @end example + + If the nbd-server uses named exports (supported since NBD 2.9.18, or with QEMU's + own embedded NBD server), you must specify an export name in the URI: + @example +-qemu-system-i386 -cdrom nbd://localhost/debian-500-ppc-netinst +-qemu-system-i386 -cdrom nbd://localhost/openSUSE-11.1-ppc-netinst ++qemu-kvm -cdrom nbd://localhost/debian-500-ppc-netinst ++qemu-kvm -cdrom nbd://localhost/openSUSE-11.1-ppc-netinst + @end example + + The URI syntax for NBD is supported since QEMU 1.3. An alternative syntax is + also available. Here are some example of the older syntax: + @example +-qemu-system-i386 linux.img -hdb nbd:my_nbd_server.mydomain.org:1024 +-qemu-system-i386 linux2.img -hdb nbd:unix:/tmp/my_socket +-qemu-system-i386 -cdrom nbd:localhost:10809:exportname=debian-500-ppc-netinst ++qemu-kvm linux.img -hdb nbd:my_nbd_server.mydomain.org:1024 ++qemu-kvm linux2.img -hdb nbd:unix:/tmp/my_socket ++qemu-kvm -cdrom nbd:localhost:10809:exportname=debian-500-ppc-netinst + @end example + + @node disk_images_sheepdog +@@ -505,7 +505,7 @@ qemu-img convert @var{filename} sheepdog:///@var{image} + + You can boot from the Sheepdog disk image with the command: + @example +-qemu-system-i386 sheepdog:///@var{image} ++qemu-kvm sheepdog:///@var{image} + @end example + + You can also create a snapshot of the Sheepdog image like qcow2. +@@ -517,7 +517,7 @@ where @var{tag} is a tag name of the newly created snapshot. + To boot from the Sheepdog snapshot, specify the tag name of the + snapshot. + @example +-qemu-system-i386 sheepdog:///@var{image}#@var{tag} ++qemu-kvm sheepdog:///@var{image}#@var{tag} + @end example + + You can create a cloned image from the existing snapshot. +@@ -530,14 +530,14 @@ is its tag name. + You can use an unix socket instead of an inet socket: + + @example +-qemu-system-i386 sheepdog+unix:///@var{image}?socket=@var{path} ++qemu-kvm sheepdog+unix:///@var{image}?socket=@var{path} + @end example + + If the Sheepdog daemon doesn't run on the local host, you need to + specify one of the Sheepdog servers to connect to. + @example + qemu-img create sheepdog://@var{hostname}:@var{port}/@var{image} @var{size} +-qemu-system-i386 sheepdog://@var{hostname}:@var{port}/@var{image} ++qemu-kvm sheepdog://@var{hostname}:@var{port}/@var{image} + @end example + + @node disk_images_iscsi +@@ -627,7 +627,7 @@ cat >iscsi.conf < /sys/bus/pci/devices/0000:06:0d.0/driver/unbind + # echo 1102 0002 > /sys/bus/pci/drivers/vfio-pci/new_id + +-# qemu-system-x86_64 -drive file=nvme://@var{host}:@var{bus}:@var{slot}.@var{func}/@var{namespace} ++# qemu-kvm -drive file=nvme://@var{host}:@var{bus}:@var{slot}.@var{func}/@var{namespace} + @end example + + Alternative syntax using properties: + + @example +-qemu-system-x86_64 -drive file.driver=nvme,file.device=@var{host}:@var{bus}:@var{slot}.@var{func},file.namespace=@var{namespace} ++qemu-kvm -drive file.driver=nvme,file.device=@var{host}:@var{bus}:@var{slot}.@var{func},file.namespace=@var{namespace} + @end example + + @var{host}:@var{bus}:@var{slot}.@var{func} is the NVMe controller's PCI device +diff --git a/docs/replay.txt b/docs/replay.txt +index 2e21e9c..f1923e8 100644 +--- a/docs/replay.txt ++++ b/docs/replay.txt +@@ -25,7 +25,7 @@ Deterministic replay has the following features: + + Usage of the record/replay: + * First, record the execution with the following command line: +- qemu-system-i386 \ ++ qemu-kvm \ + -icount shift=7,rr=record,rrfile=replay.bin \ + -drive file=disk.qcow2,if=none,id=img-direct \ + -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay \ +@@ -33,7 +33,7 @@ Usage of the record/replay: + -netdev user,id=net1 -device rtl8139,netdev=net1 \ + -object filter-replay,id=replay,netdev=net1 + * After recording, you can replay it by using another command line: +- qemu-system-i386 \ ++ qemu-kvm \ + -icount shift=7,rr=replay,rrfile=replay.bin \ + -drive file=disk.qcow2,if=none,id=img-direct \ + -drive driver=blkreplay,if=none,image=img-direct,id=img-blkreplay \ +diff --git a/docs/specs/tpm.txt b/docs/specs/tpm.txt +index d1d7157..897c300 100644 +--- a/docs/specs/tpm.txt ++++ b/docs/specs/tpm.txt +@@ -98,7 +98,7 @@ QEMU files related to the TPM passthrough device: + Command line to start QEMU with the TPM passthrough device using the host's + hardware TPM /dev/tpm0: + +-qemu-system-x86_64 -display sdl -enable-kvm \ ++qemu-kvm -display vnc -enable-kvm \ + -m 1024 -boot d -bios bios-256k.bin -boot menu=on \ + -tpmdev passthrough,id=tpm0,path=/dev/tpm0 \ + -device tpm-tis,tpmdev=tpm0 test.img +@@ -164,7 +164,7 @@ swtpm socket --tpmstate dir=/tmp/mytpm1 \ + Command line to start QEMU with the TPM emulator device communicating with + the swtpm: + +-qemu-system-x86_64 -display sdl -enable-kvm \ ++qemu-kvm -display sdl -enable-kvm \ + -m 1024 -boot d -bios bios-256k.bin -boot menu=on \ + -chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \ + -tpmdev emulator,id=tpm0,chardev=chrtpm \ +diff --git a/qemu-doc.texi b/qemu-doc.texi +index 5813d27..de5097a 100644 +--- a/qemu-doc.texi ++++ b/qemu-doc.texi +@@ -203,12 +203,12 @@ Note that, by default, GUS shares IRQ(7) with parallel ports and so + QEMU must be told to not have parallel ports to have working GUS. + + @example +-qemu-system-i386 dos.img -soundhw gus -parallel none ++qemu-kvm dos.img -soundhw gus -parallel none + @end example + + Alternatively: + @example +-qemu-system-i386 dos.img -device gus,irq=5 ++qemu-kvm dos.img -device gus,irq=5 + @end example + + Or some other unclaimed IRQ. +@@ -224,7 +224,7 @@ CS4231A is the chip used in Windows Sound System and GUSMAX products + Download and uncompress the linux image (@file{linux.img}) and type: + + @example +-qemu-system-i386 linux.img ++qemu-kvm linux.img + @end example + + Linux should boot and give you a prompt. +@@ -234,7 +234,7 @@ Linux should boot and give you a prompt. + + @example + @c man begin SYNOPSIS +-@command{qemu-system-i386} [@var{options}] [@var{disk_image}] ++@command{qemu-kvm} [@var{options}] [@var{disk_image}] + @c man end + @end example + +@@ -813,7 +813,7 @@ On Linux hosts, a shared memory device is available. The basic syntax + is: + + @example +-qemu-system-x86_64 -device ivshmem-plain,memdev=@var{hostmem} ++qemu-kvm -device ivshmem-plain,memdev=@var{hostmem} + @end example + + where @var{hostmem} names a host memory backend. For a POSIX shared +@@ -834,7 +834,7 @@ memory server is: + ivshmem-server -p @var{pidfile} -S @var{path} -m @var{shm-name} -l @var{shm-size} -n @var{vectors} + + # Then start your qemu instances with matching arguments +-qemu-system-x86_64 -device ivshmem-doorbell,vectors=@var{vectors},chardev=@var{id} ++qemu-kvm -device ivshmem-doorbell,vectors=@var{vectors},chardev=@var{id} + -chardev socket,path=@var{path},id=@var{id} + @end example + +@@ -859,7 +859,7 @@ Instead of specifying the using POSIX shm, you may specify + a memory backend that has hugepage support: + + @example +-qemu-system-x86_64 -object memory-backend-file,size=1G,mem-path=/dev/hugepages/my-shmem-file,share,id=mb1 ++qemu-kvm -object memory-backend-file,size=1G,mem-path=/dev/hugepages/my-shmem-file,share,id=mb1 + -device ivshmem-plain,memdev=mb1 + @end example + +@@ -875,7 +875,7 @@ kernel testing. + + The syntax is: + @example +-qemu-system-i386 -kernel arch/i386/boot/bzImage -hda root-2.4.20.img -append "root=/dev/hda" ++qemu-kvm -kernel arch/i386/boot/bzImage -hda root-2.4.20.img -append "root=/dev/hda" + @end example + + Use @option{-kernel} to provide the Linux kernel image and +@@ -890,7 +890,7 @@ If you do not need graphical output, you can disable it and redirect + the virtual serial port and the QEMU monitor to the console with the + @option{-nographic} option. The typical command line is: + @example +-qemu-system-i386 -kernel arch/i386/boot/bzImage -hda root-2.4.20.img \ ++qemu-kvm -kernel arch/i386/boot/bzImage -hda root-2.4.20.img \ + -append "root=/dev/hda console=ttyS0" -nographic + @end example + +@@ -956,7 +956,7 @@ Network adapter that supports CDC ethernet and RNDIS protocols. @var{id} + specifies a netdev defined with @code{-netdev @dots{},id=@var{id}}. + For instance, user-mode networking can be used with + @example +-qemu-system-i386 [...] -netdev user,id=net0 -device usb-net,netdev=net0 ++qemu-kvm [...] -netdev user,id=net0 -device usb-net,netdev=net0 + @end example + @item usb-ccid + Smartcard reader device +@@ -975,7 +975,7 @@ no type is given, the HCI logic corresponds to @code{-bt hci,vlan=0}. + This USB device implements the USB Transport Layer of HCI. Example + usage: + @example +-@command{qemu-system-i386} [...@var{OPTIONS}...] @option{-usbdevice} bt:hci,vlan=3 @option{-bt} device:keyboard,vlan=3 ++@command{qemu-kvm} [...@var{OPTIONS}...] @option{-usbdevice} bt:hci,vlan=3 @option{-bt} device:keyboard,vlan=3 + @end example + @end table + +@@ -1052,7 +1052,7 @@ For this setup it is recommended to restrict it to listen on a UNIX domain + socket only. For example + + @example +-qemu-system-i386 [...OPTIONS...] -vnc unix:/home/joebloggs/.qemu-myvm-vnc ++qemu-kvm [...OPTIONS...] -vnc unix:/home/joebloggs/.qemu-myvm-vnc + @end example + + This ensures that only users on local box with read/write access to that +@@ -1075,7 +1075,7 @@ is running the password is set with the monitor. Until the monitor is used to + set the password all clients will be rejected. + + @example +-qemu-system-i386 [...OPTIONS...] -vnc :1,password -monitor stdio ++qemu-kvm [...OPTIONS...] -vnc :1,password -monitor stdio + (qemu) change vnc password + Password: ******** + (qemu) +@@ -1092,7 +1092,7 @@ support provides a secure session, but no authentication. This allows any + client to connect, and provides an encrypted session. + + @example +-qemu-system-i386 [...OPTIONS...] -vnc :1,tls,x509=/etc/pki/qemu -monitor stdio ++qemu-kvm [...OPTIONS...] -vnc :1,tls,x509=/etc/pki/qemu -monitor stdio + @end example + + In the above example @code{/etc/pki/qemu} should contain at least three files, +@@ -1110,7 +1110,7 @@ then validate against the CA certificate. This is a good choice if deploying + in an environment with a private internal certificate authority. + + @example +-qemu-system-i386 [...OPTIONS...] -vnc :1,tls,x509verify=/etc/pki/qemu -monitor stdio ++qemu-kvm [...OPTIONS...] -vnc :1,tls,x509verify=/etc/pki/qemu -monitor stdio + @end example + + +@@ -1121,7 +1121,7 @@ Finally, the previous method can be combined with VNC password authentication + to provide two layers of authentication for clients. + + @example +-qemu-system-i386 [...OPTIONS...] -vnc :1,password,tls,x509verify=/etc/pki/qemu -monitor stdio ++qemu-kvm [...OPTIONS...] -vnc :1,password,tls,x509verify=/etc/pki/qemu -monitor stdio + (qemu) change vnc password + Password: ******** + (qemu) +@@ -1144,7 +1144,7 @@ used for authentication, but assuming use of one supporting SSF, + then QEMU can be launched with: + + @example +-qemu-system-i386 [...OPTIONS...] -vnc :1,sasl -monitor stdio ++qemu-kvm [...OPTIONS...] -vnc :1,sasl -monitor stdio + @end example + + @node vnc_sec_certificate_sasl +@@ -1158,7 +1158,7 @@ credentials. This can be enabled, by combining the 'sasl' option + with the aforementioned TLS + x509 options: + + @example +-qemu-system-i386 [...OPTIONS...] -vnc :1,tls,x509,sasl -monitor stdio ++qemu-kvm [...OPTIONS...] -vnc :1,tls,x509,sasl -monitor stdio + @end example + + @node vnc_setup_sasl +@@ -1514,7 +1514,7 @@ QEMU has a primitive support to work with gdb, so that you can do + In order to use gdb, launch QEMU with the '-s' option. It will wait for a + gdb connection: + @example +-qemu-system-i386 -s -kernel arch/i386/boot/bzImage -hda root-2.4.20.img \ ++qemu-kvm -s -kernel arch/i386/boot/bzImage -hda root-2.4.20.img \ + -append "root=/dev/hda" + Connected to host network interface: tun0 + Waiting gdb connection on port 1234 +@@ -1760,7 +1760,7 @@ Set the initial VGA graphic mode. The default is 800x600x32. + Set OpenBIOS variables in NVRAM, for example: + + @example +-qemu-system-ppc -prom-env 'auto-boot?=false' \ ++qemu-kvm -prom-env 'auto-boot?=false' \ + -prom-env 'boot-device=hd:2,\yaboot' \ + -prom-env 'boot-args=conf=hd:2,\yaboot.conf' + @end example +diff --git a/qemu-options.hx b/qemu-options.hx +index 2042dba..43f10b1 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -263,7 +263,7 @@ This option defines a free-form string that can be used to describe @var{fd}. + + You can open an image using pre-opened file descriptors from an fd set: + @example +-qemu-system-i386 ++qemu-kvm + -add-fd fd=3,set=2,opaque="rdwr:/path/to/file" + -add-fd fd=4,set=2,opaque="rdonly:/path/to/file" + -drive file=/dev/fdset/2,index=0,media=disk +@@ -292,7 +292,7 @@ STEXI + Set default value of @var{driver}'s property @var{prop} to @var{value}, e.g.: + + @example +-qemu-system-i386 -global ide-hd.physical_block_size=4096 disk-image.img ++qemu-kvm -global ide-hd.physical_block_size=4096 disk-image.img + @end example + + In particular, you can use this to set driver properties for devices which are +@@ -346,11 +346,11 @@ bootindex options. The default is non-strict boot. + + @example + # try to boot from network first, then from hard disk +-qemu-system-i386 -boot order=nc ++qemu-kvm -boot order=nc + # boot from CD-ROM first, switch back to default order after reboot +-qemu-system-i386 -boot once=d ++qemu-kvm -boot once=d + # boot with a splash picture for 5 seconds. +-qemu-system-i386 -boot menu=on,splash=/root/boot.bmp,splash-time=5000 ++qemu-kvm -boot menu=on,splash=/root/boot.bmp,splash-time=5000 + @end example + + Note: The legacy format '-boot @var{drives}' is still supported but its +@@ -379,7 +379,7 @@ For example, the following command-line sets the guest startup RAM size to + memory the guest can reach to 4GB: + + @example +-qemu-system-x86_64 -m 1G,slots=3,maxmem=4G ++qemu-kvm -m 1G,slots=3,maxmem=4G + @end example + + If @var{slots} and @var{maxmem} are not specified, memory hotplug won't +@@ -448,12 +448,12 @@ Enable audio and selected sound hardware. Use 'help' to print all + available sound hardware. + + @example +-qemu-system-i386 -soundhw sb16,adlib disk.img +-qemu-system-i386 -soundhw es1370 disk.img +-qemu-system-i386 -soundhw ac97 disk.img +-qemu-system-i386 -soundhw hda disk.img +-qemu-system-i386 -soundhw all disk.img +-qemu-system-i386 -soundhw help ++qemu-kvm -soundhw sb16,adlib disk.img ++qemu-kvm -soundhw es1370 disk.img ++qemu-kvm -soundhw ac97 disk.img ++qemu-kvm -soundhw hda disk.img ++qemu-kvm -soundhw all disk.img ++qemu-kvm -soundhw help + @end example + + Note that Linux's i810_audio OSS kernel (for AC97) module might +@@ -946,21 +946,21 @@ is off. + + Instead of @option{-cdrom} you can use: + @example +-qemu-system-i386 -drive file=file,index=2,media=cdrom ++qemu-kvm -drive file=file,index=2,media=cdrom + @end example + + Instead of @option{-hda}, @option{-hdb}, @option{-hdc}, @option{-hdd}, you can + use: + @example +-qemu-system-i386 -drive file=file,index=0,media=disk +-qemu-system-i386 -drive file=file,index=1,media=disk +-qemu-system-i386 -drive file=file,index=2,media=disk +-qemu-system-i386 -drive file=file,index=3,media=disk ++qemu-kvm -drive file=file,index=0,media=disk ++qemu-kvm -drive file=file,index=1,media=disk ++qemu-kvm -drive file=file,index=2,media=disk ++qemu-kvm -drive file=file,index=3,media=disk + @end example + + You can open an image using pre-opened file descriptors from an fd set: + @example +-qemu-system-i386 ++qemu-kvm + -add-fd fd=3,set=2,opaque="rdwr:/path/to/file" + -add-fd fd=4,set=2,opaque="rdonly:/path/to/file" + -drive file=/dev/fdset/2,index=0,media=disk +@@ -968,28 +968,28 @@ qemu-system-i386 + + You can connect a CDROM to the slave of ide0: + @example +-qemu-system-i386 -drive file=file,if=ide,index=1,media=cdrom ++qemu-kvm -drive file=file,if=ide,index=1,media=cdrom + @end example + + If you don't specify the "file=" argument, you define an empty drive: + @example +-qemu-system-i386 -drive if=ide,index=1,media=cdrom ++qemu-kvm -drive if=ide,index=1,media=cdrom + @end example + + Instead of @option{-fda}, @option{-fdb}, you can use: + @example +-qemu-system-i386 -drive file=file,index=0,if=floppy +-qemu-system-i386 -drive file=file,index=1,if=floppy ++qemu-kvm -drive file=file,index=0,if=floppy ++qemu-kvm -drive file=file,index=1,if=floppy + @end example + + By default, @var{interface} is "ide" and @var{index} is automatically + incremented: + @example +-qemu-system-i386 -drive file=a -drive file=b" ++qemu-kvm -drive file=a -drive file=b" + @end example + is interpreted like: + @example +-qemu-system-i386 -hda a -hdb b ++qemu-kvm -hda a -hdb b + @end example + ETEXI + +@@ -2056,8 +2056,8 @@ The following two example do exactly the same, to show how @option{-nic} can + be used to shorten the command line length (note that the e1000 is the default + on i386, so the @option{model=e1000} parameter could even be omitted here, too): + @example +-qemu-system-i386 -netdev user,id=n1,ipv6=off -device e1000,netdev=n1,mac=52:54:98:76:54:32 +-qemu-system-i386 -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32 ++qemu-kvm -netdev user,id=n1,ipv6=off -device e1000,netdev=n1,mac=52:54:98:76:54:32 ++qemu-kvm -nic user,ipv6=off,model=e1000,mac=52:54:98:76:54:32 + @end example + + @item -nic none +@@ -2128,7 +2128,7 @@ can not be resolved. + + Example: + @example +-qemu-system-i386 -nic user,dnssearch=mgmt.example.org,dnssearch=example.org ++qemu-kvm -nic user,dnssearch=mgmt.example.org,dnssearch=example.org + @end example + + @item tftp=@var{dir} +@@ -2144,7 +2144,7 @@ a guest from a local directory. + + Example (using pxelinux): + @example +-qemu-system-i386 -hda linux.img -boot n -device e1000,netdev=n1 \ ++qemu-kvm -hda linux.img -boot n -device e1000,netdev=n1 \ + -netdev user,id=n1,tftp=/path/to/tftp/files,bootfile=/pxelinux.0 + @end example + +@@ -2178,7 +2178,7 @@ screen 0, use the following: + + @example + # on the host +-qemu-system-i386 -nic user,hostfwd=tcp:127.0.0.1:6001-:6000 ++qemu-kvm -nic user,hostfwd=tcp:127.0.0.1:6001-:6000 + # this host xterm should open in the guest X11 server + xterm -display :1 + @end example +@@ -2188,7 +2188,7 @@ the guest, use the following: + + @example + # on the host +-qemu-system-i386 -nic user,hostfwd=tcp::5555-:23 ++qemu-kvm -nic user,hostfwd=tcp::5555-:23 + telnet localhost 5555 + @end example + +@@ -2207,7 +2207,7 @@ lifetime, like in the following example: + @example + # open 10.10.1.1:4321 on bootup, connect 10.0.2.100:1234 to it whenever + # the guest accesses it +-qemu-system-i386 -nic user,guestfwd=tcp:10.0.2.100:1234-tcp:10.10.1.1:4321 ++qemu-kvm -nic user,guestfwd=tcp:10.0.2.100:1234-tcp:10.10.1.1:4321 + @end example + + Or you can execute a command on every TCP connection established by the guest, +@@ -2216,7 +2216,7 @@ so that QEMU behaves similar to an inetd process for that virtual server: + @example + # call "netcat 10.10.1.1 4321" on every TCP connection to 10.0.2.100:1234 + # and connect the TCP stream to its stdin/stdout +-qemu-system-i386 -nic 'user,id=n1,guestfwd=tcp:10.0.2.100:1234-cmd:netcat 10.10.1.1 4321' ++qemu-kvm -nic 'user,id=n1,guestfwd=tcp:10.0.2.100:1234-cmd:netcat 10.10.1.1 4321' + @end example + + @end table +@@ -2248,21 +2248,22 @@ Examples: + + @example + #launch a QEMU instance with the default network script +-qemu-system-i386 linux.img -nic tap ++qemu-kvm linux.img -nic tap + @end example + + @example + #launch a QEMU instance with two NICs, each one connected + #to a TAP device +-qemu-system-i386 linux.img \ ++qemu-kvm linux.img \ + -netdev tap,id=nd0,ifname=tap0 -device e1000,netdev=nd0 \ + -netdev tap,id=nd1,ifname=tap1 -device rtl8139,netdev=nd1 ++ -net nic,vlan=1 -net tap,vlan=1,ifname=tap1 + @end example + + @example + #launch a QEMU instance with the default network helper to + #connect a TAP device to bridge br0 +-qemu-system-i386 linux.img -device virtio-net-pci,netdev=n1 \ ++qemu-kvm linux.img -device virtio-net-pci,netdev=n1 \ + -netdev tap,id=n1,"helper=/path/to/qemu-bridge-helper" + @end example + +@@ -2279,13 +2280,13 @@ Examples: + @example + #launch a QEMU instance with the default network helper to + #connect a TAP device to bridge br0 +-qemu-system-i386 linux.img -netdev bridge,id=n1 -device virtio-net,netdev=n1 ++qemu-kvm linux.img -netdev bridge,id=n1 -device virtio-net,netdev=n1 + @end example + + @example + #launch a QEMU instance with the default network helper to + #connect a TAP device to bridge qemubr0 +-qemu-system-i386 linux.img -netdev bridge,br=qemubr0,id=n1 -device virtio-net,netdev=n1 ++qemu-kvm linux.img -netdev bridge,br=qemubr0,id=n1 -device virtio-net,netdev=n1 + @end example + + @item -netdev socket,id=@var{id}[,fd=@var{h}][,listen=[@var{host}]:@var{port}][,connect=@var{host}:@var{port}] +@@ -2300,13 +2301,13 @@ specifies an already opened TCP socket. + Example: + @example + # launch a first QEMU instance +-qemu-system-i386 linux.img \ +- -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ +- -netdev socket,id=n1,listen=:1234 ++qemu-kvm linux.img \ ++ -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ ++ -netdev socket,id=n1,listen=:1234 + # connect the network of this instance to the network of the first instance +-qemu-system-i386 linux.img \ +- -device e1000,netdev=n2,mac=52:54:00:12:34:57 \ +- -netdev socket,id=n2,connect=127.0.0.1:1234 ++qemu-kvm linux.img \ ++ -device e1000,netdev=n2,mac=52:54:00:12:34:57 \ ++ -netdev socket,id=n2,connect=127.0.0.1:1234 + @end example + + @item -netdev socket,id=@var{id}[,fd=@var{h}][,mcast=@var{maddr}:@var{port}[,localaddr=@var{addr}]] +@@ -2329,23 +2330,23 @@ Use @option{fd=h} to specify an already opened UDP multicast socket. + Example: + @example + # launch one QEMU instance +-qemu-system-i386 linux.img \ +- -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ +- -netdev socket,id=n1,mcast=230.0.0.1:1234 ++qemu-kvm linux.img \ ++ -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ ++ -netdev socket,id=n1,mcast=230.0.0.1:1234 + # launch another QEMU instance on same "bus" +-qemu-system-i386 linux.img \ +- -device e1000,netdev=n2,mac=52:54:00:12:34:57 \ +- -netdev socket,id=n2,mcast=230.0.0.1:1234 ++qemu-kvm linux.img \ ++ -device e1000,netdev=n2,mac=52:54:00:12:34:57 \ ++ -netdev socket,id=n2,mcast=230.0.0.1:1234 + # launch yet another QEMU instance on same "bus" +-qemu-system-i386 linux.img \ +- -device e1000,netdev=n3,macaddr=52:54:00:12:34:58 \ +- -netdev socket,id=n3,mcast=230.0.0.1:1234 ++qemu-kvm linux.img \ ++ -device e1000,netdev=n3,macaddr=52:54:00:12:34:58 \ ++ -netdev socket,id=n3,mcast=230.0.0.1:1234 + @end example + + Example (User Mode Linux compat.): + @example + # launch QEMU instance (note mcast address selected is UML's default) +-qemu-system-i386 linux.img \ ++qemu-kvm linux.img \ + -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ + -netdev socket,id=n1,mcast=239.192.168.1:1102 + # launch UML +@@ -2354,9 +2355,12 @@ qemu-system-i386 linux.img \ + + Example (send packets from host's 1.2.3.4): + @example +-qemu-system-i386 linux.img \ +- -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ +- -netdev socket,id=n1,mcast=239.192.168.1:1102,localaddr=1.2.3.4 ++qemu-kvm linux.img \ ++ -device e1000,netdev=n1,mac=52:54:00:12:34:56 \ ++ -netdev socket,id=n1,mcast=239.192.168.1:1102,localaddr=1.2.3.4 ++qemu-kvm linux.img \ ++ -net nic,macaddr=52:54:00:12:34:56 \ ++ -net socket,mcast=239.192.168.1:1102,localaddr=1.2.3.4 + @end example + + @item -netdev l2tpv3,id=@var{id},src=@var{srcaddr},dst=@var{dstaddr}[,srcport=@var{srcport}][,dstport=@var{dstport}],txsession=@var{txsession}[,rxsession=@var{rxsession}][,ipv6][,udp][,cookie64][,counter][,pincounter][,txcookie=@var{txcookie}][,rxcookie=@var{rxcookie}][,offset=@var{offset}] +@@ -2414,7 +2418,7 @@ brctl addif br-lan vmtunnel0 + # on 4.3.2.1 + # launch QEMU instance - if your network has reorder or is very lossy add ,pincounter + +-qemu-system-i386 linux.img -device e1000,netdev=n1 \ ++qemu-kvm linux.img -device e1000,netdev=n1 \ + -netdev l2tpv3,id=n1,src=4.2.3.1,dst=1.2.3.4,udp,srcport=16384,dstport=16384,rxsession=0xffffffff,txsession=0xffffffff,counter + + @end example +@@ -2431,7 +2435,7 @@ Example: + # launch vde switch + vde_switch -F -sock /tmp/myswitch + # launch QEMU instance +-qemu-system-i386 linux.img -nic vde,sock=/tmp/myswitch ++qemu-kvm linux.img -nic vde,sock=/tmp/myswitch + @end example + + @item -netdev vhost-user,chardev=@var{id}[,vhostforce=on|off][,queues=n] +@@ -2445,11 +2449,11 @@ be created for multiqueue vhost-user. + + Example: + @example +-qemu -m 512 -object memory-backend-file,id=mem,size=512M,mem-path=/hugetlbfs,share=on \ +- -numa node,memdev=mem \ +- -chardev socket,id=chr0,path=/path/to/socket \ +- -netdev type=vhost-user,id=net0,chardev=chr0 \ +- -device virtio-net-pci,netdev=net0 ++qemu-kvm -m 512 -object memory-backend-file,id=mem,size=512M,mem-path=/hugetlbfs,share=on \ ++ -numa node,memdev=mem \ ++ -chardev socket,id=chr0,path=/path/to/socket \ ++ -netdev type=vhost-user,id=net0,chardev=chr0 \ ++ -device virtio-net-pci,netdev=net0 + @end example + + @item -netdev hubport,id=@var{id},hubid=@var{hubid}[,netdev=@var{nd}] +@@ -2879,7 +2883,7 @@ and communicate. Requires the Linux @code{vhci} driver installed. Can + be used as following: + + @example +-qemu-system-i386 [...OPTIONS...] -bt hci,vlan=5 -bt vhci,vlan=5 ++qemu-kvm [...OPTIONS...] -bt hci,vlan=5 -bt vhci,vlan=5 + @end example + + @item -bt device:@var{dev}[,vlan=@var{n}] +@@ -3310,14 +3314,14 @@ ETEXI + + DEF("realtime", HAS_ARG, QEMU_OPTION_realtime, + "-realtime [mlock=on|off]\n" +- " run qemu with realtime features\n" ++ " run qemu-kvm with realtime features\n" + " mlock=on|off controls mlock support (default: on)\n", + QEMU_ARCH_ALL) + STEXI + @item -realtime mlock=on|off + @findex -realtime +-Run qemu with realtime features. +-mlocking qemu and guest memory can be enabled via @option{mlock=on} ++Run qemu-kvm with realtime features. ++mlocking qemu-kvm and guest memory can be enabled via @option{mlock=on} + (enabled by default). + ETEXI + +@@ -3331,7 +3335,7 @@ connections will likely be TCP-based, but also UDP, pseudo TTY, or even + stdio are reasonable use case. The latter is allowing to start QEMU from + within gdb and establish the connection via a pipe: + @example +-(gdb) target remote | exec qemu-system-i386 -gdb stdio ... ++(gdb) target remote | exec qemu-kvm -gdb stdio ... + @end example + ETEXI + +@@ -4251,7 +4255,7 @@ which specify the queue number of cryptodev backend, the default of + + @example + +- # qemu-system-x86_64 \ ++ # qemu-kvm \ + [...] \ + -object cryptodev-backend-builtin,id=cryptodev0 \ + -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0 \ +@@ -4271,7 +4275,7 @@ of cryptodev backend for multiqueue vhost-user, the default of @var{queues} is 1 + + @example + +- # qemu-system-x86_64 \ ++ # qemu-kvm \ + [...] \ + -chardev socket,id=chardev0,path=/path/to/socket \ + -object cryptodev-vhost-user,id=cryptodev0,chardev=chardev0 \ +-- +1.8.3.1 + diff --git a/SOURCES/0011-usb-xhci-Fix-PCI-capability-order.patch b/SOURCES/0011-usb-xhci-Fix-PCI-capability-order.patch new file mode 100644 index 0000000..89d1127 --- /dev/null +++ b/SOURCES/0011-usb-xhci-Fix-PCI-capability-order.patch @@ -0,0 +1,95 @@ +From b4b549d6ab0d43ca16d492aa1b6ac75a0f880942 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Fri, 5 May 2017 19:06:14 +0200 +Subject: usb-xhci: Fix PCI capability order + +RH-Author: Dr. David Alan Gilbert +Message-id: <20170505190614.15987-2-dgilbert@redhat.com> +Patchwork-id: 75038 +O-Subject: [RHEL-7.4 qemu-kvm-rhev PATCH 1/1] usb-xhci: Fix PCI capability order +Bugzilla: 1447874 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Gerd Hoffmann +RH-Acked-by: Juan Quintela + +From: "Dr. David Alan Gilbert" + +Upstream commit 1108b2f8a9 in 2.7.0 changed the order +of the PCI capability chain in the XHCI pci device in the case +where the device has the PCIe endpoint capability (i.e. only +older machine types, pc-i440fx-2.0 upstream, pc-i440fx-rhel7.0.0 +apparently for us). + +Changing the order breaks migration compatibility; fixing this +upstream would mean breaking the same case going from 2.7.0->current +that currently works 2.7.0->2.9.0 - so upstream it's a choice +of two breakages. + +Since we never released 2.7.0/2.8.0 we can fix this downstream. + +This reverts the order so that we create the capabilities in the +order: + PCIe + MSI + MSI-X + +The symptom is: +qemu-kvm: get_pci_config_device: Bad config data: i=0x71 read: a0 device: 0 cmask: ff wmask: 0 w1cmask:0 +qemu-kvm: Failed to load PCIDevice:config +qemu-kvm: Failed to load xhci:parent_obj +qemu-kvm: error while loading state for instance 0x0 of device '0000:00:0d.0/xhci' +qemu-kvm: load of migration failed: Invalid argument + +Signed-off-by: Dr. David Alan Gilbert +Signed-off-by: Miroslav Rezanina + +-- +Rebase notes (2.9.0): +- Change in assert condition (upstream) + +(cherry picked from commit aad727a5ecde1ad4935eb8427604d4df5a1f1f35) +(cherry picked from commit 2dd7402227e77d748a7375233ac9e7feab244bda) + +Conflicts: + hw/usb/hcd-xhci.c + +(cherry picked from commit a42f86dc906cc7d2c16d02bf125ed76847b469cb) +(cherry picked from commit 992ab2e4f6e15d3e51bc716763aa8d6f45c6d29d) +--- + hw/usb/hcd-xhci.c | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) + +diff --git a/hw/usb/hcd-xhci.c b/hw/usb/hcd-xhci.c +index 883141f..181e803 100644 +--- a/hw/usb/hcd-xhci.c ++++ b/hw/usb/hcd-xhci.c +@@ -3368,6 +3368,12 @@ static void usb_xhci_realize(struct PCIDevice *dev, Error **errp) + xhci->max_pstreams_mask = 0; + } + ++ if (pci_bus_is_express(pci_get_bus(dev)) || ++ xhci_get_flag(xhci, XHCI_FLAG_FORCE_PCIE_ENDCAP)) { ++ ret = pcie_endpoint_cap_init(dev, 0xa0); ++ assert(ret > 0); ++ } ++ + if (xhci->msi != ON_OFF_AUTO_OFF) { + ret = msi_init(dev, 0x70, xhci->numintrs, true, false, &err); + /* Any error other than -ENOTSUP(board's MSI support is broken) +@@ -3416,12 +3422,6 @@ static void usb_xhci_realize(struct PCIDevice *dev, Error **errp) + PCI_BASE_ADDRESS_SPACE_MEMORY|PCI_BASE_ADDRESS_MEM_TYPE_64, + &xhci->mem); + +- if (pci_bus_is_express(pci_get_bus(dev)) || +- xhci_get_flag(xhci, XHCI_FLAG_FORCE_PCIE_ENDCAP)) { +- ret = pcie_endpoint_cap_init(dev, 0xa0); +- assert(ret > 0); +- } +- + if (xhci->msix != ON_OFF_AUTO_OFF) { + /* TODO check for errors, and should fail when msix=on */ + msix_init(dev, xhci->numintrs, +-- +1.8.3.1 + diff --git a/SOURCES/0012-virtio-scsi-Reject-scsi-cd-if-data-plane-enabled-RHE.patch b/SOURCES/0012-virtio-scsi-Reject-scsi-cd-if-data-plane-enabled-RHE.patch new file mode 100644 index 0000000..0168853 --- /dev/null +++ b/SOURCES/0012-virtio-scsi-Reject-scsi-cd-if-data-plane-enabled-RHE.patch @@ -0,0 +1,69 @@ +From 4ae8dc3b7a9a1c24380d68d8babd20f66dc0e368 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Wed, 14 Jun 2017 15:37:01 +0200 +Subject: virtio-scsi: Reject scsi-cd if data plane enabled [RHEL only] + +RH-Author: Fam Zheng +Message-id: <20170614153701.14757-1-famz@redhat.com> +Patchwork-id: 75613 +O-Subject: [RHV-7.4 qemu-kvm-rhev PATCH v3] virtio-scsi: Reject scsi-cd if data plane enabled [RHEL only] +Bugzilla: 1378816 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz + +We need a fix for RHEL 7.4 and 7.3.z, but unfortunately upstream isn't +ready. If it were, the changes will be too invasive. To have an idea: + +https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg05400.html + +is an incomplete attempt to fix part of the issue, and the remaining +work unfortunately involve even more complex changes. + +As a band-aid, this partially reverts the effect of ef8875b +(virtio-scsi: Remove op blocker for dataplane, since v2.7). We cannot +simply revert that commit as a whole because we already shipped it in +qemu-kvm-rhev 7.3, since when, block jobs has been possible. We should +only block what has been broken. Also, faithfully reverting the above +commit means adding back the removed op blocker, but that is not enough, +because it still crashes when inserting media into an initially empty +scsi-cd. + +All in all, scsi-cd on virtio-scsi-dataplane has basically been unusable +unless the scsi-cd never enters an empty state, so, disable it +altogether. Otherwise it would be much more difficult to avoid +crashing. + +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +(cherry picked from commit b0caf00bbc35c7d89e02999bdce86e1f867728e8) +(cherry picked from commit c9c4f117d8b507c2f86035c282d537c0a327364f) +(cherry picked from commit 5d586bb2543337f0ff172c6ce942dba3acbcedff) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/virtio-scsi.c | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c +index 3aa9971..9f754c4 100644 +--- a/hw/scsi/virtio-scsi.c ++++ b/hw/scsi/virtio-scsi.c +@@ -790,6 +790,15 @@ static void virtio_scsi_hotplug(HotplugHandler *hotplug_dev, DeviceState *dev, + VirtIOSCSI *s = VIRTIO_SCSI(vdev); + SCSIDevice *sd = SCSI_DEVICE(dev); + ++ /* XXX: Remove this check once block backend is capable of handling ++ * AioContext change upon eject/insert. ++ * s->ctx is NULL if ioeventfd is off, s->ctx is qemu_get_aio_context() if ++ * data plane is not used, both cases are safe for scsi-cd. */ ++ if (s->ctx && s->ctx != qemu_get_aio_context() && ++ object_dynamic_cast(OBJECT(dev), "scsi-cd")) { ++ error_setg(errp, "scsi-cd is not supported by data plane"); ++ return; ++ } + if (s->ctx && !s->dataplane_fenced) { + if (blk_op_is_blocked(sd->conf.blk, BLOCK_OP_TYPE_DATAPLANE, errp)) { + return; +-- +1.8.3.1 + diff --git a/SOURCES/0013-AArch64-Enable-CONFIG_FW_CFG_DMA-for-aarch64.patch b/SOURCES/0013-AArch64-Enable-CONFIG_FW_CFG_DMA-for-aarch64.patch new file mode 100644 index 0000000..b381b44 --- /dev/null +++ b/SOURCES/0013-AArch64-Enable-CONFIG_FW_CFG_DMA-for-aarch64.patch @@ -0,0 +1,36 @@ +From bfee0603a426dd57f60e70d05a86f5e5786bb4b0 Mon Sep 17 00:00:00 2001 +From: Wei Huang +Date: Thu, 5 Apr 2018 10:01:03 -0500 +Subject: AArch64: Enable CONFIG_FW_CFG_DMA for aarch64 + +Upstream: Downstream only +RH-Author: Wei Huang +Message-id: <20180405150103.21732-1-wei@redhat.com> +Patchwork-id: 79487 +O-Subject: [RHEL-8 qemu-kvm-rhev PATCH 1/1] AArch64: Enable CONFIG_FW_CFG_DMA for aarch64 +Bugzilla: 1564172 +RH-Acked-by: Andrew Jones +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Thomas Huth + +This patch enables the vmcoreinfo device for aarch64. This device is +required for the crash utility to support qemu guest dump when KASLR is +enabled. + +Signed-off-by: Wei Huang +--- + default-configs/aarch64-softmmu.mak | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/default-configs/aarch64-softmmu.mak b/default-configs/aarch64-softmmu.mak +index 001eb8e..860140e 100644 +--- a/default-configs/aarch64-softmmu.mak ++++ b/default-configs/aarch64-softmmu.mak +@@ -27,3 +27,4 @@ CONFIG_IOH3420=y + CONFIG_USB_XHCI=y + CONFIG_USB=y + CONFIG_I2C=y ++CONFIG_FW_CFG_DMA=y +-- +1.8.3.1 + diff --git a/SOURCES/0016-pc-bios-s390-ccw-struct-tpi_info-must-be-declared-as.patch b/SOURCES/0016-pc-bios-s390-ccw-struct-tpi_info-must-be-declared-as.patch new file mode 100644 index 0000000..0d123ab --- /dev/null +++ b/SOURCES/0016-pc-bios-s390-ccw-struct-tpi_info-must-be-declared-as.patch @@ -0,0 +1,41 @@ +From 3e38e82fc6601763cb597d8849a61a871ab06b72 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Tue, 8 May 2018 12:01:10 +0200 +Subject: pc-bios/s390-ccw: struct tpi_info must be declared as aligned(4) + +Upstream-status: n/a yet (likely later, but downstream fix is required now) + +I've run into a compilation error today with the current version of GCC: + +In file included from s390-ccw.h:49, + from main.c:12: +cio.h:128:1: error: alignment 1 of 'struct tpi_info' is less than 4 [-Werror=packed-not-aligned] + } __attribute__ ((packed)); + ^ +cc1: all warnings being treated as errors + +Since the struct tpi_info contains an element ("struct subchannel_id schid") +which is marked as aligned(4), we've got to mark the struct tpi_info as +aligned(4), too. + +Signed-off-by: Thomas Huth +--- + pc-bios/s390-ccw/cio.h | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/pc-bios/s390-ccw/cio.h b/pc-bios/s390-ccw/cio.h +index 55eaeee..1a0795f 100644 +--- a/pc-bios/s390-ccw/cio.h ++++ b/pc-bios/s390-ccw/cio.h +@@ -125,7 +125,7 @@ struct tpi_info { + __u32 reserved3 : 12; + __u32 int_type : 3; + __u32 reserved4 : 12; +-} __attribute__ ((packed)); ++} __attribute__ ((packed, aligned(4))); + + /* channel command word (type 1) */ + struct ccw1 { +-- +1.8.3.1 + diff --git a/SOURCES/0020-pc-pc-rhel75.5.0-compat-code.patch b/SOURCES/0020-pc-pc-rhel75.5.0-compat-code.patch new file mode 100644 index 0000000..dace222 --- /dev/null +++ b/SOURCES/0020-pc-pc-rhel75.5.0-compat-code.patch @@ -0,0 +1,126 @@ +From 411b30bec63d20ebcbc90d933a3ff73851d60f5a Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Sun, 22 Apr 2018 02:44:30 +0100 +Subject: pc: pc-*-rhel75.5.0 compat code + +RH-Author: Eduardo Habkost +Message-id: <20180422024430.10218-1-ehabkost@redhat.com> +Patchwork-id: 79845 +O-Subject: [RHEL-8.0 qemu-kvm PATCH] pc: pc-*-rhel75.5.0 compat code +Bugzilla: 1569675 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Danilo de Paula +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Peter Xu + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1569675 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15862256 + +Based on the pc-*-2.11 and pc-*-2.10 compat code from upstream. + +Not tested yet, but still better than having it completely broken +on RHEL-8.0 Alpha. I plan to test it next week. + +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/pc_piix.c | 2 ++ + hw/i386/pc_q35.c | 4 ++++ + include/hw/compat.h | 28 ++++++++++++++++++++++++++++ + include/hw/i386/pc.h | 13 +++++++++++++ + 4 files changed, 47 insertions(+) + +diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c +index cc72512..e5add39 100644 +--- a/hw/i386/pc_piix.c ++++ b/hw/i386/pc_piix.c +@@ -1166,6 +1166,8 @@ static void pc_machine_rhel750_options(MachineClass *m) + { + pc_machine_rhel7_options(m); + m->desc = "RHEL 7.5.0 PC (i440FX + PIIX, 1996)"; ++ m->auto_enable_numa_with_memhp = false; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_5_COMPAT); + } + + DEFINE_PC_MACHINE(rhel750, "pc-i440fx-rhel7.5.0", pc_init_rhel750, +diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c +index dbf6bfa..ffc461d 100644 +--- a/hw/i386/pc_q35.c ++++ b/hw/i386/pc_q35.c +@@ -431,8 +431,12 @@ static void pc_q35_init_rhel750(MachineState *machine) + + static void pc_q35_machine_rhel750_options(MachineClass *m) + { ++ PCMachineClass *pcmc = PC_MACHINE_CLASS(m); + pc_q35_machine_rhel7_options(m); + m->desc = "RHEL-7.5.0 PC (Q35 + ICH9, 2009)"; ++ m->auto_enable_numa_with_memhp = false; ++ pcmc->default_nic_model = "e1000"; ++ SET_MACHINE_COMPAT(m, PC_RHEL7_5_COMPAT); + } + + DEFINE_PC_MACHINE(q35_rhel750, "pc-q35-rhel7.5.0", pc_q35_init_rhel750, +diff --git a/include/hw/compat.h b/include/hw/compat.h +index de251fd..f7b39c5 100644 +--- a/include/hw/compat.h ++++ b/include/hw/compat.h +@@ -446,4 +446,32 @@ + .value = "off",\ + }, + ++/* The same as HW_COMPAT_2_11 + HW_COMPAT_2_10 */ ++#define HW_COMPAT_RHEL7_5 \ ++ { /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_11 */ \ ++ .driver = "hpet",\ ++ .property = "hpet-offset-saved",\ ++ .value = "false",\ ++ },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_11 */ \ ++ .driver = "virtio-blk-pci",\ ++ .property = "vectors",\ ++ .value = "2",\ ++ },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_11 */ \ ++ .driver = "vhost-user-blk-pci",\ ++ .property = "vectors",\ ++ .value = "2",\ ++ },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_11 */ \ ++ .driver = "e1000",\ ++ .property = "migrate_tso_props",\ ++ .value = "off",\ ++ },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_10 */ \ ++ .driver = "virtio-mouse-device",\ ++ .property = "wheel-axis",\ ++ .value = "false",\ ++ },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_10 */ \ ++ .driver = "virtio-tablet-device",\ ++ .property = "wheel-axis",\ ++ .value = "false",\ ++ }, ++ + #endif /* HW_COMPAT_H */ +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index faddeba..e94424f 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -969,6 +969,19 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .value = "on",\ + }, + ++/* Similar to PC_COMPAT_2_11 + PC_COMPAT_2_10, but: ++ * - x-hv-max-vps was backported to 7.5 ++ * - x-pci-hole64-fix was backported to 7.5 ++ */ ++#define PC_RHEL7_5_COMPAT \ ++ HW_COMPAT_RHEL7_5 \ ++ { /* PC_RHEL7_5_COMPAT from PC_COMPAT_2_11 */ \ ++ .driver = "Skylake-Server" "-" TYPE_X86_CPU,\ ++ .property = "clflushopt",\ ++ .value = "off",\ ++ }, ++ ++ + #define PC_RHEL7_4_COMPAT \ + HW_COMPAT_RHEL7_4 \ + { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_9 */ \ +-- +1.8.3.1 + diff --git a/SOURCES/0022-tcg-workaround-branch-instruction-overflow-in-tcg_ou.patch b/SOURCES/0022-tcg-workaround-branch-instruction-overflow-in-tcg_ou.patch new file mode 100644 index 0000000..1703ef1 --- /dev/null +++ b/SOURCES/0022-tcg-workaround-branch-instruction-overflow-in-tcg_ou.patch @@ -0,0 +1,106 @@ +From 3319e2fd5b151695f30f8574bbd9250f86a96e16 Mon Sep 17 00:00:00 2001 +From: Laurent Vivier +Date: Thu, 3 May 2018 14:59:08 +0100 +Subject: tcg: workaround branch instruction overflow in tcg_out_qemu_ld/st + +RH-Author: Laurent Vivier +Message-id: <20180503145908.8110-1-lvivier@redhat.com> +Patchwork-id: 80019 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH] tcg: workaround branch instruction overflow in tcg_out_qemu_ld/st +Bugzilla: 1571145 +RH-Acked-by: Thomas Huth +RH-Acked-by: Serhii Popovych +RH-Acked-by: David Gibson + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1571145 +BRANCH:rhel8/master-2.12.0 +UPSTREAM: https://github.com/qemu/qemu/commit/6001f7729e12dd1d810291e4cbf83cee8e07441d +BREW: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15973114 + +ppc64 uses a BC instruction to call the tcg_out_qemu_ld/st +slow path. BC instruction uses a relative address encoded +on 14 bits. + +The slow path functions are added at the end of the generated +instructions buffer, in the reverse order of the callers. +So more we have slow path functions more the distance between +the caller (BC) and the function increases. + +This patch changes the behavior to generate the functions in +the same order of the callers. + +Cc: qemu-stable@nongnu.org +Fixes: 15fa08f845 ("tcg: Dynamically allocate TCGOps") +Signed-off-by: Laurent Vivier +Message-Id: <20180429235840.16659-1-lvivier@redhat.com> +Signed-off-by: Richard Henderson +(cherry picked from commit 6001f7729e12dd1d810291e4cbf83cee8e07441d) +Signed-off-by: Laurent Vivier +Signed-off-by: Danilo C. L. de Paula +--- + tcg/tcg-ldst.inc.c | 8 ++++---- + tcg/tcg.c | 2 +- + tcg/tcg.h | 2 +- + 3 files changed, 6 insertions(+), 6 deletions(-) + +diff --git a/tcg/tcg-ldst.inc.c b/tcg/tcg-ldst.inc.c +index 0e14cf4..47f41b9 100644 +--- a/tcg/tcg-ldst.inc.c ++++ b/tcg/tcg-ldst.inc.c +@@ -30,7 +30,7 @@ typedef struct TCGLabelQemuLdst { + TCGReg datahi_reg; /* reg index for high word to be loaded or stored */ + tcg_insn_unit *raddr; /* gen code addr of the next IR of qemu_ld/st IR */ + tcg_insn_unit *label_ptr[2]; /* label pointers to be updated */ +- struct TCGLabelQemuLdst *next; ++ QSIMPLEQ_ENTRY(TCGLabelQemuLdst) next; + } TCGLabelQemuLdst; + + +@@ -46,7 +46,7 @@ static bool tcg_out_ldst_finalize(TCGContext *s) + TCGLabelQemuLdst *lb; + + /* qemu_ld/st slow paths */ +- for (lb = s->ldst_labels; lb != NULL; lb = lb->next) { ++ QSIMPLEQ_FOREACH(lb, &s->ldst_labels, next) { + if (lb->is_ld) { + tcg_out_qemu_ld_slow_path(s, lb); + } else { +@@ -72,7 +72,7 @@ static inline TCGLabelQemuLdst *new_ldst_label(TCGContext *s) + { + TCGLabelQemuLdst *l = tcg_malloc(sizeof(*l)); + +- l->next = s->ldst_labels; +- s->ldst_labels = l; ++ QSIMPLEQ_INSERT_TAIL(&s->ldst_labels, l, next); ++ + return l; + } +diff --git a/tcg/tcg.c b/tcg/tcg.c +index bb24526..b84850b 100644 +--- a/tcg/tcg.c ++++ b/tcg/tcg.c +@@ -3324,7 +3324,7 @@ int tcg_gen_code(TCGContext *s, TranslationBlock *tb) + s->code_ptr = tb->tc.ptr; + + #ifdef TCG_TARGET_NEED_LDST_LABELS +- s->ldst_labels = NULL; ++ QSIMPLEQ_INIT(&s->ldst_labels); + #endif + #ifdef TCG_TARGET_NEED_POOL_LABELS + s->pool_labels = NULL; +diff --git a/tcg/tcg.h b/tcg/tcg.h +index 30896ca..a3076c5 100644 +--- a/tcg/tcg.h ++++ b/tcg/tcg.h +@@ -699,7 +699,7 @@ struct TCGContext { + + /* These structures are private to tcg-target.inc.c. */ + #ifdef TCG_TARGET_NEED_LDST_LABELS +- struct TCGLabelQemuLdst *ldst_labels; ++ QSIMPLEQ_HEAD(ldst_labels, TCGLabelQemuLdst) ldst_labels; + #endif + #ifdef TCG_TARGET_NEED_POOL_LABELS + struct TCGLabelPoolData *pool_labels; +-- +1.8.3.1 + diff --git a/SOURCES/0025-s390-ccw-force-diag-308-subcode-to-unsigned-long.patch b/SOURCES/0025-s390-ccw-force-diag-308-subcode-to-unsigned-long.patch new file mode 100644 index 0000000..a00def1 --- /dev/null +++ b/SOURCES/0025-s390-ccw-force-diag-308-subcode-to-unsigned-long.patch @@ -0,0 +1,41 @@ +From 2b09944ad35c48e37801d5abe9069283f8835fb2 Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Tue, 8 May 2018 09:01:11 +0000 +Subject: s390-ccw: force diag 308 subcode to unsigned long + +We currently pass an integer as the subcode parameter. However, +the upper bits of the register containing the subcode need to +be 0, which is not guaranteed unless we explicitly specify the +subcode to be an unsigned long value. + +Fixes: d046c51dad3 ("pc-bios/s390-ccw: Get device address via diag 308/6") +Cc: qemu-stable@nongnu.org +Signed-off-by: Cornelia Huck +Acked-by: Christian Borntraeger +Tested-by: Thomas Huth +Signed-off-by: Thomas Huth +(cherry picked from commit 63d8b5ace31c1e1f3996fe4cd551d6d377594d5a) +--- + pc-bios/s390-ccw/iplb.h | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/pc-bios/s390-ccw/iplb.h b/pc-bios/s390-ccw/iplb.h +index 5357a36..ded20c8 100644 +--- a/pc-bios/s390-ccw/iplb.h ++++ b/pc-bios/s390-ccw/iplb.h +@@ -101,10 +101,11 @@ static inline bool manage_iplb(IplParameterBlock *iplb, bool store) + { + register unsigned long addr asm("0") = (unsigned long) iplb; + register unsigned long rc asm("1") = 0; ++ unsigned long subcode = store ? 6 : 5; + + asm volatile ("diag %0,%2,0x308\n" + : "+d" (addr), "+d" (rc) +- : "d" (store ? 6 : 5) ++ : "d" (subcode) + : "memory", "cc"); + return rc == 0x01; + } +-- +1.8.3.1 + diff --git a/SOURCES/0026-pc-bios-s390-ccw-size_t-should-be-unsigned.patch b/SOURCES/0026-pc-bios-s390-ccw-size_t-should-be-unsigned.patch new file mode 100644 index 0000000..43789e2 --- /dev/null +++ b/SOURCES/0026-pc-bios-s390-ccw-size_t-should-be-unsigned.patch @@ -0,0 +1,50 @@ +From 0384fba1d0550f0bb2a6cfeb24b13d8c8186524d Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Tue, 8 May 2018 09:01:12 +0000 +Subject: pc-bios/s390-ccw: size_t should be unsigned + +"size_t" should be an unsigned type according to the C standard. +Thus we should also use this convention in the s390-ccw firmware to avoid +confusion. I checked the sources, and apart from one spot in libc.c, the +code should all be fine with this change. + +Buglink: https://bugs.launchpad.net/qemu/+bug/1753437 +Reviewed-by: Christian Borntraeger +Reviewed-by: Halil Pasic +Reviewed-by: Collin Walling +Signed-off-by: Thomas Huth +(cherry picked from commit e4f869621203955761cf274c87d5595e9facd319) +--- + pc-bios/s390-ccw/libc.c | 2 +- + pc-bios/s390-ccw/libc.h | 2 +- + 2 files changed, 2 insertions(+), 2 deletions(-) + +diff --git a/pc-bios/s390-ccw/libc.c b/pc-bios/s390-ccw/libc.c +index 38ea77d..a786566 100644 +--- a/pc-bios/s390-ccw/libc.c ++++ b/pc-bios/s390-ccw/libc.c +@@ -63,7 +63,7 @@ uint64_t atoui(const char *str) + */ + char *uitoa(uint64_t num, char *str, size_t len) + { +- size_t num_idx = 1; /* account for NUL */ ++ long num_idx = 1; /* account for NUL */ + uint64_t tmp = num; + + IPL_assert(str != NULL, "uitoa: no space allocated to store string"); +diff --git a/pc-bios/s390-ccw/libc.h b/pc-bios/s390-ccw/libc.h +index 63ece70..818517f 100644 +--- a/pc-bios/s390-ccw/libc.h ++++ b/pc-bios/s390-ccw/libc.h +@@ -12,7 +12,7 @@ + #ifndef S390_CCW_LIBC_H + #define S390_CCW_LIBC_H + +-typedef long size_t; ++typedef unsigned long size_t; + typedef int bool; + typedef unsigned char uint8_t; + typedef unsigned short uint16_t; +-- +1.8.3.1 + diff --git a/SOURCES/0027-pc-bios-s390-ccw-rename-MAX_TABLE_ENTRIES-to-MAX_BOO.patch b/SOURCES/0027-pc-bios-s390-ccw-rename-MAX_TABLE_ENTRIES-to-MAX_BOO.patch new file mode 100644 index 0000000..3fcda3d --- /dev/null +++ b/SOURCES/0027-pc-bios-s390-ccw-rename-MAX_TABLE_ENTRIES-to-MAX_BOO.patch @@ -0,0 +1,82 @@ +From f21a07e653b25bf6d99c28f709a88ed1dfe6170f Mon Sep 17 00:00:00 2001 +From: Collin Walling +Date: Tue, 8 May 2018 09:01:13 +0000 +Subject: pc-bios/s390-ccw: rename MAX_TABLE_ENTRIES to MAX_BOOT_ENTRIES + +The MAX_TABLE_ENTRIES constant has a name that is too generic. As we +want to declare a limit for boot menu entries, let's rename it to a more +fitting MAX_BOOT_ENTRIES and set its value to 31 (30 boot entries and +1 default entry). Also we move it from bootmap.h to s390-ccw.h to make +it available for menu.c in a later patch. + +Signed-off-by: Collin Walling +Reviewed-by: Thomas Huth +Reviewed-by: Janosch Frank +Signed-off-by: Thomas Huth +(cherry picked from commit 6df2a829dfacfbf10a78199ad4b023a7ea65d9cd) +--- + pc-bios/s390-ccw/bootmap.c | 6 +++--- + pc-bios/s390-ccw/bootmap.h | 2 -- + pc-bios/s390-ccw/s390-ccw.h | 2 ++ + 3 files changed, 5 insertions(+), 5 deletions(-) + +diff --git a/pc-bios/s390-ccw/bootmap.c b/pc-bios/s390-ccw/bootmap.c +index 9287b7a..b767fa2 100644 +--- a/pc-bios/s390-ccw/bootmap.c ++++ b/pc-bios/s390-ccw/bootmap.c +@@ -297,7 +297,7 @@ static void run_eckd_boot_script(block_number_t bmt_block_nr, + } + + debug_print_int("loadparm", loadparm); +- IPL_assert(loadparm <= MAX_TABLE_ENTRIES, "loadparm value greater than" ++ IPL_assert(loadparm < MAX_BOOT_ENTRIES, "loadparm value greater than" + " maximum number of boot entries allowed"); + + memset(sec, FREE_SPACE_FILLER, sizeof(sec)); +@@ -585,7 +585,7 @@ static void ipl_scsi(void) + read_block(mbr->pt.blockno, sec, "Error reading Program Table"); + IPL_assert(magic_match(sec, ZIPL_MAGIC), "No zIPL magic in PT"); + +- while (program_table_entries <= MAX_TABLE_ENTRIES) { ++ while (program_table_entries < MAX_BOOT_ENTRIES) { + if (!prog_table->entry[program_table_entries].scsi.blockno) { + break; + } +@@ -600,7 +600,7 @@ static void ipl_scsi(void) + } + + debug_print_int("loadparm", loadparm); +- IPL_assert(loadparm <= MAX_TABLE_ENTRIES, "loadparm value greater than" ++ IPL_assert(loadparm < MAX_BOOT_ENTRIES, "loadparm value greater than" + " maximum number of boot entries allowed"); + + zipl_run(&prog_table->entry[loadparm].scsi); /* no return */ +diff --git a/pc-bios/s390-ccw/bootmap.h b/pc-bios/s390-ccw/bootmap.h +index 07eb600..732c111 100644 +--- a/pc-bios/s390-ccw/bootmap.h ++++ b/pc-bios/s390-ccw/bootmap.h +@@ -57,8 +57,6 @@ typedef union BootMapPointer { + ExtEckdBlockPtr xeckd; + } __attribute__ ((packed)) BootMapPointer; + +-#define MAX_TABLE_ENTRIES 30 +- + /* aka Program Table */ + typedef struct BootMapTable { + uint8_t magic[4]; +diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h +index fd18da2..2c9e601 100644 +--- a/pc-bios/s390-ccw/s390-ccw.h ++++ b/pc-bios/s390-ccw/s390-ccw.h +@@ -94,6 +94,8 @@ bool menu_is_enabled_zipl(void); + int menu_get_enum_boot_index(int entries); + bool menu_is_enabled_enum(void); + ++#define MAX_BOOT_ENTRIES 31 ++ + static inline void fill_hex(char *out, unsigned char val) + { + const char hex[] = "0123456789abcdef"; +-- +1.8.3.1 + diff --git a/SOURCES/0028-pc-bios-s390-ccw-fix-loadparm-initialization-and-int.patch b/SOURCES/0028-pc-bios-s390-ccw-fix-loadparm-initialization-and-int.patch new file mode 100644 index 0000000..b096bab --- /dev/null +++ b/SOURCES/0028-pc-bios-s390-ccw-fix-loadparm-initialization-and-int.patch @@ -0,0 +1,94 @@ +From 3301328699d574c8d6617eb4105cd9d4794f722c Mon Sep 17 00:00:00 2001 +From: Collin Walling +Date: Tue, 8 May 2018 09:01:14 +0000 +Subject: pc-bios/s390-ccw: fix loadparm initialization and int conversion + +Rename the loadparm char array in main.c to loadparm_str and +increased the size by one byte to account for a null termination +when converting the loadparm string to an int via atoui. We +also allow the boot menu to be enabled when loadparm is set to +an empty string or a series of spaces. + +Signed-off-by: Collin Walling +Reported-by: Vasily Gorbik +Reviewed-by: Thomas Huth +Reviewed-by: Janosch Frank +Signed-off-by: Thomas Huth +(cherry picked from commit 074afe60d4c8167dcfaee7aca1065c6360449eaa) +--- + hw/s390x/ipl.c | 4 ++++ + pc-bios/s390-ccw/main.c | 14 +++++++------- + 2 files changed, 11 insertions(+), 7 deletions(-) + +diff --git a/hw/s390x/ipl.c b/hw/s390x/ipl.c +index fb554ab..150f6c0 100644 +--- a/hw/s390x/ipl.c ++++ b/hw/s390x/ipl.c +@@ -373,6 +373,10 @@ int s390_ipl_set_loadparm(uint8_t *loadparm) + loadparm[i] = ascii2ebcdic[(uint8_t) lp[i]]; + } + ++ if (i < 8) { ++ memset(loadparm + i, 0x40, 8 - i); /* fill with EBCDIC spaces */ ++ } ++ + g_free(lp); + return 0; + } +diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c +index 9d9f8cf..26f9adf 100644 +--- a/pc-bios/s390-ccw/main.c ++++ b/pc-bios/s390-ccw/main.c +@@ -15,11 +15,11 @@ + char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE))); + static SubChannelId blk_schid = { .one = 1 }; + IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE))); +-static char loadparm[8] = { 0, 0, 0, 0, 0, 0, 0, 0 }; ++static char loadparm_str[9] = { 0, 0, 0, 0, 0, 0, 0, 0, 0 }; + QemuIplParameters qipl; + + #define LOADPARM_PROMPT "PROMPT " +-#define LOADPARM_EMPTY "........" ++#define LOADPARM_EMPTY " " + #define BOOT_MENU_FLAG_MASK (QIPL_FLAG_BM_OPTS_CMD | QIPL_FLAG_BM_OPTS_ZIPL) + + /* +@@ -45,7 +45,7 @@ void panic(const char *string) + + unsigned int get_loadparm_index(void) + { +- return atoui(loadparm); ++ return atoui(loadparm_str); + } + + static bool find_dev(Schib *schib, int dev_no) +@@ -80,13 +80,13 @@ static bool find_dev(Schib *schib, int dev_no) + + static void menu_setup(void) + { +- if (memcmp(loadparm, LOADPARM_PROMPT, 8) == 0) { ++ if (memcmp(loadparm_str, LOADPARM_PROMPT, 8) == 0) { + menu_set_parms(QIPL_FLAG_BM_OPTS_CMD, 0); + return; + } + + /* If loadparm was set to any other value, then do not enable menu */ +- if (memcmp(loadparm, LOADPARM_EMPTY, 8) != 0) { ++ if (memcmp(loadparm_str, LOADPARM_EMPTY, 8) != 0) { + return; + } + +@@ -116,8 +116,8 @@ static void virtio_setup(void) + */ + enable_mss_facility(); + +- sclp_get_loadparm_ascii(loadparm); +- memcpy(ldp + 10, loadparm, 8); ++ sclp_get_loadparm_ascii(loadparm_str); ++ memcpy(ldp + 10, loadparm_str, 8); + sclp_print(ldp); + + memcpy(&qipl, early_qipl, sizeof(QemuIplParameters)); +-- +1.8.3.1 + diff --git a/SOURCES/0029-pc-bios-s390-ccw-fix-non-sequential-boot-entries-eck.patch b/SOURCES/0029-pc-bios-s390-ccw-fix-non-sequential-boot-entries-eck.patch new file mode 100644 index 0000000..832a0cf --- /dev/null +++ b/SOURCES/0029-pc-bios-s390-ccw-fix-non-sequential-boot-entries-eck.patch @@ -0,0 +1,105 @@ +From c0577fcb360841afe54f76deee37ea4a52761bf3 Mon Sep 17 00:00:00 2001 +From: Collin Walling +Date: Tue, 8 May 2018 09:01:15 +0000 +Subject: pc-bios/s390-ccw: fix non-sequential boot entries (eckd) + +zIPL boot menu entries can be non-sequential. Let's account +for this issue for the s390 zIPL boot menu. Since this boot +menu is actually an imitation and is not completely capable +of everything the real zIPL menu can do, let's also print a +different banner to the user. + +Signed-off-by: Collin Walling +Reported-by: Vasily Gorbik +Reviewed-by: Thomas Huth +Reviewed-by: Janosch Frank +Signed-off-by: Thomas Huth +(cherry picked from commit 7385e947fc65a44dd05abb86c874beb915c1989c) +--- + pc-bios/s390-ccw/menu.c | 29 ++++++++++++++++++++--------- + 1 file changed, 20 insertions(+), 9 deletions(-) + +diff --git a/pc-bios/s390-ccw/menu.c b/pc-bios/s390-ccw/menu.c +index 96eec81..aaf5d61 100644 +--- a/pc-bios/s390-ccw/menu.c ++++ b/pc-bios/s390-ccw/menu.c +@@ -158,7 +158,7 @@ static void boot_menu_prompt(bool retry) + } + } + +-static int get_boot_index(int entries) ++static int get_boot_index(bool *valid_entries) + { + int boot_index; + bool retry = false; +@@ -168,7 +168,8 @@ static int get_boot_index(int entries) + boot_menu_prompt(retry); + boot_index = get_index(); + retry = true; +- } while (boot_index < 0 || boot_index >= entries); ++ } while (boot_index < 0 || boot_index >= MAX_BOOT_ENTRIES || ++ !valid_entries[boot_index]); + + sclp_print("\nBooting entry #"); + sclp_print(uitoa(boot_index, tmp, sizeof(tmp))); +@@ -176,7 +177,8 @@ static int get_boot_index(int entries) + return boot_index; + } + +-static void zipl_println(const char *data, size_t len) ++/* Returns the entry number that was printed */ ++static int zipl_print_entry(const char *data, size_t len) + { + char buf[len + 2]; + +@@ -185,12 +187,15 @@ static void zipl_println(const char *data, size_t len) + buf[len + 1] = '\0'; + + sclp_print(buf); ++ ++ return buf[0] == ' ' ? atoui(buf + 1) : atoui(buf); + } + + int menu_get_zipl_boot_index(const char *menu_data) + { + size_t len; +- int entries; ++ int entry; ++ bool valid_entries[MAX_BOOT_ENTRIES] = {false}; + uint16_t zipl_flag = *(uint16_t *)(menu_data - ZIPL_FLAG_OFFSET); + uint16_t zipl_timeout = *(uint16_t *)(menu_data - ZIPL_TIMEOUT_OFFSET); + +@@ -202,19 +207,25 @@ int menu_get_zipl_boot_index(const char *menu_data) + timeout = zipl_timeout * 1000; + } + +- /* Print and count all menu items, including the banner */ +- for (entries = 0; *menu_data; entries++) { ++ /* Print banner */ ++ sclp_print("s390-ccw zIPL Boot Menu\n\n"); ++ menu_data += strlen(menu_data) + 1; ++ ++ /* Print entries */ ++ while (*menu_data) { + len = strlen(menu_data); +- zipl_println(menu_data, len); ++ entry = zipl_print_entry(menu_data, len); + menu_data += len + 1; + +- if (entries < 2) { ++ valid_entries[entry] = true; ++ ++ if (entry == 0) { + sclp_print("\n"); + } + } + + sclp_print("\n"); +- return get_boot_index(entries - 1); /* subtract 1 to exclude banner */ ++ return get_boot_index(valid_entries); + } + + +-- +1.8.3.1 + diff --git a/SOURCES/0030-pc-bios-s390-ccw-fix-non-sequential-boot-entries-enu.patch b/SOURCES/0030-pc-bios-s390-ccw-fix-non-sequential-boot-entries-enu.patch new file mode 100644 index 0000000..2577549 --- /dev/null +++ b/SOURCES/0030-pc-bios-s390-ccw-fix-non-sequential-boot-entries-enu.patch @@ -0,0 +1,135 @@ +From f6d3898264a5083ebe7bcc663eab353bfa6ba1f4 Mon Sep 17 00:00:00 2001 +From: Collin Walling +Date: Tue, 8 May 2018 09:01:16 +0000 +Subject: pc-bios/s390-ccw: fix non-sequential boot entries (enum) + +zIPL boot menu entries can be non-sequential. Let's account +for this issue for the s390 enumerated boot menu. Since we +can no longer print a range of available entries to the +user, we have to present a list of each available entry. + +An example of this menu: + + s390-ccw Enumerated Boot Menu. + + [0] default + + [1] + [2] + [7] + [8] + [9] + [11] + [12] + + Please choose: + +Signed-off-by: Collin Walling +Reported-by: Vasily Gorbik +Reviewed-by: Thomas Huth +Reviewed-by: Janosch Frank +Signed-off-by: Thomas Huth +(cherry picked from commit 622b39178057289a1c8c1b5148f513e658e90ea1) +--- + pc-bios/s390-ccw/bootmap.c | 12 +++++++----- + pc-bios/s390-ccw/menu.c | 29 ++++++++++++++++++++--------- + pc-bios/s390-ccw/s390-ccw.h | 2 +- + 3 files changed, 28 insertions(+), 15 deletions(-) + +diff --git a/pc-bios/s390-ccw/bootmap.c b/pc-bios/s390-ccw/bootmap.c +index b767fa2..e41e715 100644 +--- a/pc-bios/s390-ccw/bootmap.c ++++ b/pc-bios/s390-ccw/bootmap.c +@@ -565,6 +565,8 @@ static void ipl_scsi(void) + int program_table_entries = 0; + BootMapTable *prog_table = (void *)sec; + unsigned int loadparm = get_loadparm_index(); ++ bool valid_entries[MAX_BOOT_ENTRIES] = {false}; ++ size_t i; + + /* Grab the MBR */ + memset(sec, FREE_SPACE_FILLER, sizeof(sec)); +@@ -585,18 +587,18 @@ static void ipl_scsi(void) + read_block(mbr->pt.blockno, sec, "Error reading Program Table"); + IPL_assert(magic_match(sec, ZIPL_MAGIC), "No zIPL magic in PT"); + +- while (program_table_entries < MAX_BOOT_ENTRIES) { +- if (!prog_table->entry[program_table_entries].scsi.blockno) { +- break; ++ for (i = 0; i < MAX_BOOT_ENTRIES; i++) { ++ if (prog_table->entry[i].scsi.blockno) { ++ valid_entries[i] = true; ++ program_table_entries++; + } +- program_table_entries++; + } + + debug_print_int("program table entries", program_table_entries); + IPL_assert(program_table_entries != 0, "Empty Program Table"); + + if (menu_is_enabled_enum()) { +- loadparm = menu_get_enum_boot_index(program_table_entries); ++ loadparm = menu_get_enum_boot_index(valid_entries); + } + + debug_print_int("loadparm", loadparm); +diff --git a/pc-bios/s390-ccw/menu.c b/pc-bios/s390-ccw/menu.c +index aaf5d61..82a4ae6 100644 +--- a/pc-bios/s390-ccw/menu.c ++++ b/pc-bios/s390-ccw/menu.c +@@ -228,19 +228,30 @@ int menu_get_zipl_boot_index(const char *menu_data) + return get_boot_index(valid_entries); + } + +- +-int menu_get_enum_boot_index(int entries) ++int menu_get_enum_boot_index(bool *valid_entries) + { +- char tmp[4]; ++ char tmp[3]; ++ int i; + +- sclp_print("s390x Enumerated Boot Menu.\n\n"); ++ sclp_print("s390-ccw Enumerated Boot Menu.\n\n"); + +- sclp_print(uitoa(entries, tmp, sizeof(tmp))); +- sclp_print(" entries detected. Select from boot index 0 to "); +- sclp_print(uitoa(entries - 1, tmp, sizeof(tmp))); +- sclp_print(".\n\n"); ++ for (i = 0; i < MAX_BOOT_ENTRIES; i++) { ++ if (valid_entries[i]) { ++ if (i < 10) { ++ sclp_print(" "); ++ } ++ sclp_print("["); ++ sclp_print(uitoa(i, tmp, sizeof(tmp))); ++ sclp_print("]"); ++ if (i == 0) { ++ sclp_print(" default\n"); ++ } ++ sclp_print("\n"); ++ } ++ } + +- return get_boot_index(entries); ++ sclp_print("\n"); ++ return get_boot_index(valid_entries); + } + + void menu_set_parms(uint8_t boot_menu_flag, uint32_t boot_menu_timeout) +diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h +index 2c9e601..a1bdb4c 100644 +--- a/pc-bios/s390-ccw/s390-ccw.h ++++ b/pc-bios/s390-ccw/s390-ccw.h +@@ -91,7 +91,7 @@ void zipl_load(void); + void menu_set_parms(uint8_t boot_menu_flag, uint32_t boot_menu_timeout); + int menu_get_zipl_boot_index(const char *menu_data); + bool menu_is_enabled_zipl(void); +-int menu_get_enum_boot_index(int entries); ++int menu_get_enum_boot_index(bool *valid_entries); + bool menu_is_enabled_enum(void); + + #define MAX_BOOT_ENTRIES 31 +-- +1.8.3.1 + diff --git a/SOURCES/0031-pc-rhel7.6.0-machine-types.patch b/SOURCES/0031-pc-rhel7.6.0-machine-types.patch new file mode 100644 index 0000000..eb8720a --- /dev/null +++ b/SOURCES/0031-pc-rhel7.6.0-machine-types.patch @@ -0,0 +1,112 @@ +From 188fa8896734043c11798495072b7f98111c5d94 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Wed, 25 Apr 2018 13:30:35 +0000 +Subject: pc: rhel7.6.0 machine-types + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1557051 +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1559791 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=15893027 + +The rhel7.6.0 machine-type was going to be added much later +because RHEL-7.6 is not ready yet. However, adding a new +machine-type is the only way to change the default NIC to e1000e. +This patch adds pc-*-rhel7.6.0 machine-types. + +Signed-off-by: Eduardo Habkost +--- + hw/i386/pc_piix.c | 21 ++++++++++++++++++--- + hw/i386/pc_q35.c | 18 ++++++++++++++++-- + 2 files changed, 34 insertions(+), 5 deletions(-) + +diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c +index e5add39..0ff1e2d 100644 +--- a/hw/i386/pc_piix.c ++++ b/hw/i386/pc_piix.c +@@ -1156,6 +1156,21 @@ static void pc_machine_rhel7_options(MachineClass *m) + m->is_default = 1; + } + ++static void pc_init_rhel760(MachineState *machine) ++{ ++ pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ ++ TYPE_I440FX_PCI_DEVICE); ++} ++ ++static void pc_machine_rhel760_options(MachineClass *m) ++{ ++ pc_machine_rhel7_options(m); ++ m->desc = "RHEL 7.6.0 PC (i440FX + PIIX, 1996)"; ++} ++ ++DEFINE_PC_MACHINE(rhel760, "pc-i440fx-rhel7.6.0", pc_init_rhel760, ++ pc_machine_rhel760_options); ++ + static void pc_init_rhel750(MachineState *machine) + { + pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +@@ -1164,7 +1179,9 @@ static void pc_init_rhel750(MachineState *machine) + + static void pc_machine_rhel750_options(MachineClass *m) + { +- pc_machine_rhel7_options(m); ++ pc_machine_rhel760_options(m); ++ m->alias = NULL; ++ m->is_default = 0; + m->desc = "RHEL 7.5.0 PC (i440FX + PIIX, 1996)"; + m->auto_enable_numa_with_memhp = false; + SET_MACHINE_COMPAT(m, PC_RHEL7_5_COMPAT); +@@ -1183,8 +1200,6 @@ static void pc_machine_rhel740_options(MachineClass *m) + { + PCMachineClass *pcmc = PC_MACHINE_CLASS(m); + pc_machine_rhel750_options(m); +- m->alias = NULL; +- m->is_default = 0; + m->desc = "RHEL 7.4.0 PC (i440FX + PIIX, 1996)"; + m->numa_auto_assign_ram = numa_legacy_auto_assign_ram; + pcmc->pc_rom_ro = false; +diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c +index ffc461d..e1fd23e 100644 +--- a/hw/i386/pc_q35.c ++++ b/hw/i386/pc_q35.c +@@ -424,6 +424,20 @@ static void pc_q35_machine_rhel7_options(MachineClass *m) + SET_MACHINE_COMPAT(m, PC_RHEL_COMPAT); + } + ++static void pc_q35_init_rhel760(MachineState *machine) ++{ ++ pc_q35_init(machine); ++} ++ ++static void pc_q35_machine_rhel760_options(MachineClass *m) ++{ ++ pc_q35_machine_rhel7_options(m); ++ m->desc = "RHEL-7.6.0 PC (Q35 + ICH9, 2009)"; ++} ++ ++DEFINE_PC_MACHINE(q35_rhel760, "pc-q35-rhel7.6.0", pc_q35_init_rhel760, ++ pc_q35_machine_rhel760_options); ++ + static void pc_q35_init_rhel750(MachineState *machine) + { + pc_q35_init(machine); +@@ -432,7 +446,8 @@ static void pc_q35_init_rhel750(MachineState *machine) + static void pc_q35_machine_rhel750_options(MachineClass *m) + { + PCMachineClass *pcmc = PC_MACHINE_CLASS(m); +- pc_q35_machine_rhel7_options(m); ++ pc_q35_machine_rhel760_options(m); ++ m->alias = NULL; + m->desc = "RHEL-7.5.0 PC (Q35 + ICH9, 2009)"; + m->auto_enable_numa_with_memhp = false; + pcmc->default_nic_model = "e1000"; +@@ -451,7 +466,6 @@ static void pc_q35_machine_rhel740_options(MachineClass *m) + { + PCMachineClass *pcmc = PC_MACHINE_CLASS(m); + pc_q35_machine_rhel750_options(m); +- m->alias = NULL; + m->desc = "RHEL-7.4.0 PC (Q35 + ICH9, 2009)"; + m->numa_auto_assign_ram = numa_legacy_auto_assign_ram; + pcmc->pc_rom_ro = false; +-- +1.8.3.1 + diff --git a/SOURCES/0032-Remove-rhel6-machine-types.patch b/SOURCES/0032-Remove-rhel6-machine-types.patch new file mode 100644 index 0000000..27cbe56 --- /dev/null +++ b/SOURCES/0032-Remove-rhel6-machine-types.patch @@ -0,0 +1,755 @@ +From 88b450562f14b4b246f88c31d0bdd48e47f3afce Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 26 Apr 2018 02:54:03 +0000 +Subject: Remove rhel6* machine types + +As we do not support RHEL 6 compatibility on RHEL 8 we remove rhel6* +machine types. + +Types were originally added for BZ 983991 (Provide RHEL-6 machine types) and were +updated multipletimes during the RHEL 7 development to keep the compatibility. As +all machine types changes are located in pc_piix.c file there are only tests to +be fixed beside this file (and one comment removal in pc.h). + +Signed-off-by: Miroslav Rezanina +--- + hw/i386/pc_piix.c | 696 --------------------------------------------------- + include/hw/i386/pc.h | 2 - + tests/qom-test.c | 4 +- + 3 files changed, 1 insertion(+), 701 deletions(-) + +diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c +index 0ff1e2d..229d551 100644 +--- a/hw/i386/pc_piix.c ++++ b/hw/i386/pc_piix.c +@@ -1324,699 +1324,3 @@ static void pc_machine_rhel700_options(MachineClass *m) + + DEFINE_PC_MACHINE(rhel700, "pc-i440fx-rhel7.0.0", pc_init_rhel700, + pc_machine_rhel700_options); +- +-#define PC_RHEL6_6_COMPAT \ +- {\ +- .driver = "scsi-hd",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "scsi-cd",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "scsi-disk",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "ide-hd",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "ide-cd",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "ide-drive",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "virtio-blk-pci",\ +- .property = "discard_granularity",\ +- .value = stringify(0),\ +- },{\ +- .driver = "virtio-serial-pci",\ +- .property = "vectors",\ +- /* DEV_NVECTORS_UNSPECIFIED as a uint32_t string */\ +- .value = stringify(0xFFFFFFFF),\ +- },{\ +- .driver = "486-" TYPE_X86_CPU,\ +- .property = "model",\ +- .value = stringify(0),\ +- },{\ +- .driver = "usb-tablet",\ +- .property = "usb_version",\ +- .value = stringify(1),\ +- },{\ +- .driver = "virtio-net-pci",\ +- .property = "mq",\ +- .value = "off",\ +- },{\ +- .driver = "VGA",\ +- .property = "mmio",\ +- .value = "off",\ +- },{\ +- .driver = "virtio-blk-pci",\ +- .property = "config-wce",\ +- .value = "off",\ +- },{\ +- .driver = TYPE_ISA_FDC,\ +- .property = "check_media_rate",\ +- .value = "off",\ +- },{\ +- .driver = "virtio-balloon-pci",\ +- .property = "class",\ +- .value = stringify(PCI_CLASS_MEMORY_RAM),\ +- },{\ +- .driver = TYPE_PCI_DEVICE,\ +- .property = "command_serr_enable",\ +- .value = "off",\ +- },{\ +- .driver = "AC97",\ +- .property = "use_broken_id",\ +- .value = stringify(1),\ +- },{\ +- .driver = "intel-hda",\ +- .property = "msi",\ +- .value = "off",\ +- },{\ +- .driver = "qemu32-" TYPE_X86_CPU,\ +- .property = "min-xlevel",\ +- .value = stringify(0),\ +- },{\ +- .driver = "486-" TYPE_X86_CPU,\ +- .property = "min-level",\ +- .value = stringify(0),\ +- },{\ +- .driver = "qemu32-" TYPE_X86_CPU,\ +- .property = "model",\ +- .value = stringify(3),\ +- },{\ +- .driver = "usb-ccid",\ +- .property = "serial",\ +- .value = "1",\ +- },{\ +- .driver = "virtio-net-pci",\ +- .property = "any_layout",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "pentium" "-" TYPE_X86_CPU,\ +- .property = "apic",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "pentium2" "-" TYPE_X86_CPU,\ +- .property = "apic",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "pentium3" "-" TYPE_X86_CPU,\ +- .property = "apic",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "Conroe" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Penryn" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Nehalem" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Nehalem-IBRS" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "pclmulqdq",\ +- .value = "off",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "pclmulqdq",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "fxsr",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "fxsr",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "mmx",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "mmx",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "pat",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "pat",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "cmov",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "cmov",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "pge",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "pge",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "apic",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "cx8",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "cx8",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "mce",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "mce",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "pae",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "pae",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "msr",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "msr",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "tsc",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "tsc",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "pse",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "pse",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "de",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "de",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Westmere" "-" TYPE_X86_CPU,\ +- .property = "fpu",\ +- .value = "on",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Westmere-IBRS" "-" TYPE_X86_CPU,\ +- .property = "fpu",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Broadwell" "-" TYPE_X86_CPU,\ +- .property = "rdtscp",\ +- .value = "off",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ +- .property = "rdtscp",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "Broadwell" "-" TYPE_X86_CPU,\ +- .property = "smap",\ +- .value = "off",\ +- },\ +- { /* PC_RHEL6_6_COMPAT (copied from the entry above) */ \ +- .driver = "Broadwell-IBRS" "-" TYPE_X86_CPU,\ +- .property = "smap",\ +- .value = "off",\ +- },\ +- {\ +- .driver = TYPE_X86_CPU,\ +- .property = "rdtscp",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "Opteron_G1" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Opteron_G2" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Opteron_G3" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "on",\ +- },\ +- {\ +- .driver = "Opteron_G4" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "Opteron_G5" "-" TYPE_X86_CPU,\ +- .property = "x2apic",\ +- .value = "off",\ +- },\ +- {\ +- .driver = TYPE_X86_CPU,\ +- .property = "3dnow",\ +- .value = "off",\ +- },\ +- {\ +- .driver = TYPE_X86_CPU,\ +- .property = "3dnowext",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "virtio-net-pci",\ +- .property = "__com.redhat_rhel6_ctrl_guest_workaround", \ +- .value = "on",\ +- }, +- +-static void pc_compat_rhel660(MachineState *machine) +-{ +- PCMachineState *pcms = PC_MACHINE(machine); +- PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(pcms); +- +- pc_compat_rhel700(machine); +- if (!machine->cpu_type) { +- machine->cpu_type = "cpu64-rhel6"; +- } +- +- x86_cpu_change_kvm_default("kvm-pv-unhalt", NULL); +- +- pcmc->gigabyte_align = false; +- shadow_bios_after_incoming = true; +- ich9_uhci123_irqpin_override = true; +-} +- +-static void pc_init_rhel660(MachineState *machine) +-{ +- pc_compat_rhel660(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel660_options(MachineClass *m) +-{ +- PCMachineClass *pcmc = PC_MACHINE_CLASS(m); +- pc_machine_rhel700_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.6.0 PC"; +- m->rom_file_has_mr = false; +- m->default_machine_opts = "firmware=bios.bin"; +- pcmc->has_acpi_build = false; +- SET_MACHINE_COMPAT(m, PC_RHEL6_6_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel660, "rhel6.6.0", pc_init_rhel660, +- pc_machine_rhel660_options); +- +-#define PC_RHEL6_5_COMPAT \ +- {\ +- .driver = TYPE_USB_DEVICE,\ +- .property = "msos-desc",\ +- .value = "no",\ +- }, +- +-static void pc_compat_rhel650(MachineState *machine) +-{ +- pc_compat_rhel660(machine); +-} +- +-static void pc_init_rhel650(MachineState *machine) +-{ +- pc_compat_rhel650(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel650_options(MachineClass *m) +-{ +- pc_machine_rhel660_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.5.0 PC"; +- SET_MACHINE_COMPAT(m, PC_RHEL6_5_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel650, "rhel6.5.0", pc_init_rhel650, +- pc_machine_rhel650_options); +- +-#define PC_RHEL6_4_COMPAT \ +- {\ +- .driver = "virtio-scsi-pci",\ +- .property = "vectors",\ +- .value = stringify(2),\ +- },{\ +- .driver = "hda-micro",\ +- .property = "mixer",\ +- .value = "off",\ +- },{\ +- .driver = "hda-duplex",\ +- .property = "mixer",\ +- .value = "off",\ +- },{\ +- .driver = "hda-output",\ +- .property = "mixer",\ +- .value = "off",\ +- },{\ +- .driver = "virtio-net-pci",\ +- .property = "ctrl_mac_addr",\ +- .value = "off",\ +- },\ +- {\ +- .driver = TYPE_X86_CPU,\ +- .property = "sep",\ +- .value = "off",\ +- },\ +- {\ +- .driver = "virtio-net-pci",\ +- .property = "__com.redhat_rhel6_ctrl_guest_workaround", \ +- .value = "off",\ +- }, +- +-static void pc_compat_rhel640(MachineState *machine) +-{ +- pc_compat_rhel650(machine); +-} +- +-static void pc_init_rhel640(MachineState *machine) +-{ +- pc_compat_rhel640(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel640_options(MachineClass *m) +-{ +- pc_machine_rhel650_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.4.0 PC"; +- SET_MACHINE_COMPAT(m, PC_RHEL6_4_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel640, "rhel6.4.0", pc_init_rhel640, +- pc_machine_rhel640_options); +- +-#define PC_RHEL6_3_COMPAT \ +- {\ +- .driver = "Conroe-" TYPE_X86_CPU,\ +- .property = "min-level",\ +- .value = stringify(2),\ +- },{\ +- .driver = "Penryn-" TYPE_X86_CPU,\ +- .property = "min-level",\ +- .value = stringify(2),\ +- },{\ +- .driver = "Nehalem-" TYPE_X86_CPU,\ +- .property = "min-level",\ +- .value = stringify(2),\ +- },{\ +- .driver = "e1000",\ +- .property = "autonegotiation",\ +- .value = "off",\ +- },{\ +- .driver = "qxl",\ +- .property = "revision",\ +- .value = stringify(3),\ +- },{\ +- .driver = "qxl-vga",\ +- .property = "revision",\ +- .value = stringify(3),\ +- },{\ +- .driver = "virtio-scsi-pci",\ +- .property = "hotplug",\ +- .value = "off",\ +- },{\ +- .driver = "virtio-scsi-pci",\ +- .property = "param_change",\ +- .value = "off",\ +- },{\ +- .driver = TYPE_X86_CPU,\ +- .property = "pmu",\ +- .value = "on",\ +- },{\ +- .driver = "usb-hub",\ +- .property = "serial",\ +- .value = "314159",\ +- },{\ +- .driver = "usb-storage",\ +- .property = "serial",\ +- .value = "1",\ +- },\ +- {\ +- .driver = "SandyBridge" "-" TYPE_X86_CPU,\ +- .property = "tsc-deadline",\ +- .value = "off",\ +- },\ +- { /* PC_RHEL6_3_COMPAT (copied from the entry above) */ \ +- .driver = "SandyBridge-IBRS" "-" TYPE_X86_CPU,\ +- .property = "tsc-deadline",\ +- .value = "off",\ +- }, +- +-static void pc_compat_rhel630(MachineState *machine) +-{ +- pc_compat_rhel640(machine); +- x86_cpu_change_kvm_default("kvm-pv-eoi",NULL); +- enable_compat_apic_id_mode(); +-} +- +-static void pc_init_rhel630(MachineState *machine) +-{ +- pc_compat_rhel630(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel630_options(MachineClass *m) +-{ +- pc_machine_rhel640_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.3.0 PC"; +- SET_MACHINE_COMPAT(m, PC_RHEL6_3_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel630, "rhel6.3.0", pc_init_rhel630, +- pc_machine_rhel630_options); +- +- +-#define PC_RHEL6_2_COMPAT \ +- {\ +- .driver = TYPE_X86_CPU,\ +- .property = "pmu",\ +- .value = "off",\ +- }, +- +-static void pc_compat_rhel620(MachineState *machine) +-{ +- pc_compat_rhel630(machine); +-} +- +-static void pc_init_rhel620(MachineState *machine) +-{ +- pc_compat_rhel620(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel620_options(MachineClass *m) +-{ +- pc_machine_rhel630_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.2.0 PC"; +- SET_MACHINE_COMPAT(m, PC_RHEL6_2_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel620, "rhel6.2.0", pc_init_rhel620, +- pc_machine_rhel620_options); +- +-/* +- * NOTE: We don't have the event_idx compat entry for the +- * virtio-balloon-pci driver because RHEL6 doesn't disable +- * it either due to a bug (see RHBZ 1029539 fo more info) +- */ +-#define PC_RHEL6_1_COMPAT \ +- {\ +- .driver = "PIIX4_PM",\ +- .property = "disable_s3",\ +- .value = "0",\ +- },{\ +- .driver = "PIIX4_PM",\ +- .property = "disable_s4",\ +- .value = "0",\ +- },{\ +- .driver = "qxl",\ +- .property = "revision",\ +- .value = stringify(2),\ +- },{\ +- .driver = "qxl-vga",\ +- .property = "revision",\ +- .value = stringify(2),\ +- },{\ +- .driver = "virtio-blk-pci",\ +- .property = "event_idx",\ +- .value = "off",\ +- },{\ +- .driver = "virtio-serial-pci",\ +- .property = "event_idx",\ +- .value = "off",\ +- },{\ +- .driver = "virtio-net-pci",\ +- .property = "event_idx",\ +- .value = "off",\ +- },{\ +- .driver = "usb-kbd",\ +- .property = "serial",\ +- .value = "1",\ +- },{\ +- .driver = "usb-mouse",\ +- .property = "serial",\ +- .value = "1",\ +- },{\ +- .driver = "usb-tablet",\ +- .property = "serial",\ +- .value = "1",\ +- }, +- +-static void pc_compat_rhel610(MachineState *machine) +-{ +- pc_compat_rhel620(machine); +-} +- +-static void pc_init_rhel610(MachineState *machine) +-{ +- pc_compat_rhel610(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel610_options(MachineClass *m) +-{ +- pc_machine_rhel620_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.1.0 PC"; +- SET_MACHINE_COMPAT(m, PC_RHEL6_1_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel610, "rhel6.1.0", pc_init_rhel610, +- pc_machine_rhel610_options); +- +-#define PC_RHEL6_0_COMPAT \ +- {\ +- .driver = "qxl",\ +- .property = "revision",\ +- .value = stringify(1),\ +- },{\ +- .driver = "qxl-vga",\ +- .property = "revision",\ +- .value = stringify(1),\ +- },{\ +- .driver = "VGA",\ +- .property = "rombar",\ +- .value = stringify(0),\ +- }, +- +-static void pc_compat_rhel600(MachineState *machine) +-{ +- pc_compat_rhel610(machine); +-} +- +-static void pc_init_rhel600(MachineState *machine) +-{ +- pc_compat_rhel600(machine); +- pc_init1(machine, TYPE_I440FX_PCI_HOST_BRIDGE, \ +- TYPE_I440FX_PCI_DEVICE);} +- +-static void pc_machine_rhel600_options(MachineClass *m) +-{ +- pc_machine_rhel610_options(m); +- m->family = "pc_piix_Z"; +- m->desc = "RHEL 6.0.0 PC"; +- SET_MACHINE_COMPAT(m, PC_RHEL6_0_COMPAT); +-} +- +-DEFINE_PC_MACHINE(rhel600, "rhel6.0.0", pc_init_rhel600, +- pc_machine_rhel600_options); +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index e94424f..ae84db4 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -1394,8 +1394,6 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + * The PC_RHEL_*_COMPAT serve the same purpose for RHEL-7 machine + * types as the PC_COMPAT_* do for upstream types. + * PC_RHEL_7_*_COMPAT apply both to i440fx and q35 types. +- * PC_RHEL6_*_COMPAT apply to i440fx types only, and therefore live +- * in pc_piix.c. + */ + + /* +diff --git a/tests/qom-test.c b/tests/qom-test.c +index db0d3ab..2fc2670 100644 +--- a/tests/qom-test.c ++++ b/tests/qom-test.c +@@ -16,9 +16,7 @@ + #include "libqtest.h" + + static const char *blacklist_x86[] = { +- "xenfv", "xenpv", "isapc", +- "rhel6.6.0", "rhel6.5.0", "rhel6.4.0", "rhel6.3.0", +- "rhel6.2.0", "rhel6.1.0", "rhel6.0.0", NULL ++ "xenfv", "xenpv", "isapc", NULL + }; + + static const struct { +-- +1.8.3.1 + diff --git a/SOURCES/0033-Remove-rhel6_ctrl_guest_workaround.patch b/SOURCES/0033-Remove-rhel6_ctrl_guest_workaround.patch new file mode 100644 index 0000000..6fe31e7 --- /dev/null +++ b/SOURCES/0033-Remove-rhel6_ctrl_guest_workaround.patch @@ -0,0 +1,81 @@ +From 8a50b1caa7e56dc2b9a2f4dc8bc9c63e9a064085 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 26 Apr 2018 02:54:04 +0000 +Subject: Remove rhel6_ctrl_guest_workaround + +As we are not support RHEL 6 compatibility on RHEL 8 removing hack +to handle missing ctrl-guest-offload in RHEL 6 machine types. + +This hack was introduced for BZ 1378334 (windows guests migration from +rhel6.8-z to rhel7.3 with virtio-net-pci fail) in qemu-kvm-rhev for RHEL 7.4 +and was backported to qemu-kvm-rhev for RHEL 7.3 as commit 9a30ebb5 (and +propagated to RHEL 7.4 with rebase to 2.9.0). + +Singed-off-by: Miroslav Rezanina +--- + hw/virtio/virtio.c | 22 +--------------------- + include/hw/virtio/virtio.h | 1 - + 2 files changed, 1 insertion(+), 22 deletions(-) + +diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c +index 4bcb4f4..006d3d1 100644 +--- a/hw/virtio/virtio.c ++++ b/hw/virtio/virtio.c +@@ -24,7 +24,6 @@ + #include "hw/virtio/virtio-access.h" + #include "sysemu/dma.h" + +-#include "standard-headers/linux/virtio_net.h" + /* + * The alignment to use between consumer and producer parts of vring. + * x86 pagesize again. This is the default, used by transports like PCI +@@ -1992,24 +1991,7 @@ const VMStateInfo virtio_vmstate_info = { + static int virtio_set_features_nocheck(VirtIODevice *vdev, uint64_t val) + { + VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev); +- bool bad; +- uint64_t ctrl_guest_mask = 1ull << VIRTIO_NET_F_CTRL_GUEST_OFFLOADS; +- +- if (vdev->rhel6_ctrl_guest_workaround && (val & ctrl_guest_mask) && +- !(vdev->host_features & ctrl_guest_mask)) { +- /* +- * This works around a mistake in the definition of the rhel6.[56].0 +- * machinetypes, ctrl-guest-offload was not set in qemu-kvm-rhev for +- * those machine types, but is set on the rhel6 qemu-kvm-rhev build. +- * If an incoming rhel6 guest uses it then we need to allow it. +- * Note: There's a small race where a guest read the flag but didn't +- * declare it's useage yet. +- */ +- fprintf(stderr, "RHEL6 ctrl_guest_offload workaround\n"); +- vdev->host_features |= ctrl_guest_mask; +- } +- +- bad = (val & ~(vdev->host_features)) != 0; ++ bool bad = (val & ~(vdev->host_features)) != 0; + + val &= vdev->host_features; + if (k->set_features) { +@@ -2584,8 +2566,6 @@ static void virtio_device_instance_finalize(Object *obj) + + static Property virtio_properties[] = { + DEFINE_VIRTIO_COMMON_FEATURES(VirtIODevice, host_features), +- DEFINE_PROP_BOOL("__com.redhat_rhel6_ctrl_guest_workaround", VirtIODevice, +- rhel6_ctrl_guest_workaround, false), + DEFINE_PROP_END_OF_LIST(), + }; + +diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h +index 41e13d2..098bdaa 100644 +--- a/include/hw/virtio/virtio.h ++++ b/include/hw/virtio/virtio.h +@@ -95,7 +95,6 @@ struct VirtIODevice + uint8_t device_endian; + bool use_guest_notifier_mask; + AddressSpace *dma_as; +- bool rhel6_ctrl_guest_workaround; + QLIST_HEAD(, VirtQueue) *vector_queues; + }; + +-- +1.8.3.1 + diff --git a/SOURCES/0034-Remove-SeaBIOS-shadowing.patch b/SOURCES/0034-Remove-SeaBIOS-shadowing.patch new file mode 100644 index 0000000..791b5b1 --- /dev/null +++ b/SOURCES/0034-Remove-SeaBIOS-shadowing.patch @@ -0,0 +1,242 @@ +From 71562f446db550489bf4ba79e634a8b55e74d83f Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 26 Apr 2018 02:54:06 +0000 +Subject: Remove SeaBIOS shadowing + +As we do not support RHEL 6 compatibility on RHEL 8 we can remove +hacks that shadow SeaBIOS for incomming RHEL-6 host + +This feature was added in qemu-kvm RHEL 7.1 for BZ 1027565 (fail to +reboot guest after migration from RHEL6.5 host to RHEL7.0 host) as +commit 9f136b4ed4ec and was backported to RHEL 7.0 as 0day fix (BZ 1091322, +commit df9e9e9c56c7). + +In addition, fix was provided for qemu-kvm-rhev in RHEL 7.1 (BZ 1103579, +commit cada12245ab9). + +For qemu-kvm, use of this hack was extended for BZ 1176283 +([migration]migrationfailed when configure guest with OVMF bios + machine + type=rhel6.5.0) in RHEL 7.2 by commit c3f813d2f. + +For qemu-kvm-rhev, use of this hack was extened for BZ 1170093 (guest NUMA +failed to migrate when machine is rhel6.5.0) for RHEL 7.1 by commit 8e8107cb3 +and BZ 1175099 ([migration]migration failed when configure guest with OVMF +bios + machine type=rhel6.5.0) for RHEL 7.2 by commit 8b220c0e. + +In addition, during rebase to 2.10, handling was moved to separate module +and stub version was provided. + +Signed-off-by: Miroslav Rezanina +--- + hw/i386/Makefile.objs | 1 - + hw/i386/pc_sysfw.c | 16 ------------- + hw/i386/shadow-bios.c | 64 ------------------------------------------------- + include/sysemu/sysemu.h | 2 -- + migration/savevm.c | 7 ------ + numa.c | 13 ---------- + stubs/Makefile.objs | 1 - + stubs/shadow-bios.c | 7 ------ + 8 files changed, 111 deletions(-) + delete mode 100644 hw/i386/shadow-bios.c + delete mode 100644 stubs/shadow-bios.c + +diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs +index 8c25538..fa87a14 100644 +--- a/hw/i386/Makefile.objs ++++ b/hw/i386/Makefile.objs +@@ -10,4 +10,3 @@ obj-$(CONFIG_VMMOUSE) += vmmouse.o + + obj-y += kvmvapic.o + obj-y += acpi-build.o +-obj-y += shadow-bios.o +diff --git a/hw/i386/pc_sysfw.c b/hw/i386/pc_sysfw.c +index 2a6de35..73ac783 100644 +--- a/hw/i386/pc_sysfw.c ++++ b/hw/i386/pc_sysfw.c +@@ -207,13 +207,6 @@ static void old_pc_system_rom_init(MemoryRegion *rom_memory, bool isapc_ram_fw) + (bios_size % 65536) != 0) { + goto bios_error; + } +- if (shadow_bios_after_incoming && bios_size != 128 * 1024) { +- MachineClass *mc; +- +- mc = MACHINE_GET_CLASS(current_machine); +- error_report("machine %s only supports a 128KB BIOS image", mc->name); +- exit(1); +- } + bios = g_malloc(sizeof(*bios)); + memory_region_init_ram(bios, NULL, "pc.bios", bios_size, &error_fatal); + if (!isapc_ram_fw) { +@@ -261,15 +254,6 @@ void pc_system_firmware_init(MemoryRegion *rom_memory, bool isapc_ram_fw) + return; + } + +- if (shadow_bios_after_incoming) { +- MachineClass *mc; +- +- mc = MACHINE_GET_CLASS(current_machine); +- error_report("flash-based firmware is not supported by machine %s", +- mc->name); +- exit(1); +- } +- + if (kvm_enabled() && !kvm_readonly_mem_enabled()) { + /* Older KVM cannot execute from device memory. So, flash memory + * cannot be used unless the readonly memory kvm capability is present. */ +diff --git a/hw/i386/shadow-bios.c b/hw/i386/shadow-bios.c +deleted file mode 100644 +index 65a4cb8..0000000 +--- a/hw/i386/shadow-bios.c ++++ /dev/null +@@ -1,64 +0,0 @@ +-#include "qemu/osdep.h" +-#include "sysemu/sysemu.h" +-#include "target/i386/cpu.h" +-#include "exec/ram_addr.h" +-#include "qemu/cutils.h" +- +-void shadow_bios(void) +-{ +- RAMBlock *block, *ram, *oprom, *bios; +- size_t one_meg, oprom_size, bios_size; +- uint8_t *cd_seg_host, *ef_seg_host; +- +- ram = NULL; +- oprom = NULL; +- bios = NULL; +- rcu_read_lock(); +- QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { +- if (strcmp("pc.ram", block->idstr) == 0) { +- assert(ram == NULL); +- ram = block; +- } else if (strcmp("pc.rom", block->idstr) == 0) { +- assert(oprom == NULL); +- oprom = block; +- } else if (strcmp("pc.bios", block->idstr) == 0) { +- assert(bios == NULL); +- bios = block; +- } +- } +- assert(ram != NULL); +- assert(oprom != NULL); +- assert(bios != NULL); +- assert(memory_region_is_ram(ram->mr)); +- assert(memory_region_is_ram(oprom->mr)); +- assert(memory_region_is_ram(bios->mr)); +- assert(int128_eq(ram->mr->size, int128_make64(ram->used_length))); +- assert(int128_eq(oprom->mr->size, int128_make64(oprom->used_length))); +- assert(int128_eq(bios->mr->size, int128_make64(bios->used_length))); +- +- one_meg = 1024 * 1024; +- oprom_size = 128 * 1024; +- bios_size = 128 * 1024; +- assert(ram->used_length >= one_meg); +- assert(oprom->used_length == oprom_size); +- assert(bios->used_length == bios_size); +- +- ef_seg_host = memory_region_get_ram_ptr(ram->mr) + (one_meg - bios_size); +- cd_seg_host = ef_seg_host - oprom_size; +- +- /* This is a crude hack, but we must distinguish a rhel6.x.0 machtype guest +- * coming in from a RHEL-6 emulator (where shadowing has had no effect on +- * "pc.ram") from a similar guest coming in from a RHEL-7 emulator (where +- * shadowing has worked). In the latter case we must not trample the live +- * SeaBIOS variables in "pc.ram". +- */ +- if (buffer_is_zero(ef_seg_host, bios_size)) { +- fprintf(stderr, "copying E and F segments from pc.bios to pc.ram\n"); +- memcpy(ef_seg_host, memory_region_get_ram_ptr(bios->mr), bios_size); +- } +- if (buffer_is_zero(cd_seg_host, oprom_size)) { +- fprintf(stderr, "copying C and D segments from pc.rom to pc.ram\n"); +- memcpy(cd_seg_host, memory_region_get_ram_ptr(oprom->mr), oprom_size); +- } +- rcu_read_unlock(); +-} +diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h +index 5832c38..2b42151 100644 +--- a/include/sysemu/sysemu.h ++++ b/include/sysemu/sysemu.h +@@ -94,8 +94,6 @@ void qemu_add_machine_init_done_notifier(Notifier *notify); + void qemu_remove_machine_init_done_notifier(Notifier *notify); + + void qemu_announce_self(void); +-extern bool shadow_bios_after_incoming; +-void shadow_bios(void); + + extern int autostart; + +diff --git a/migration/savevm.c b/migration/savevm.c +index 56c9feb..6c539d1 100644 +--- a/migration/savevm.c ++++ b/migration/savevm.c +@@ -84,7 +84,6 @@ enum qemu_vm_cmd { + MIG_CMD_PACKAGED, /* Send a wrapped stream within this stream */ + MIG_CMD_MAX + }; +-bool shadow_bios_after_incoming; + + #define MAX_VM_CMD_PACKAGED_SIZE UINT32_MAX + static struct mig_cmd_args { +@@ -2206,12 +2205,6 @@ int qemu_loadvm_state(QEMUFile *f) + } + + qemu_loadvm_state_cleanup(); +- /* Supplement SeaBIOS's shadowing now, because it was useless when the +- * incoming VM started on the RHEL-6 emulator. +- */ +- if (shadow_bios_after_incoming) { +- shadow_bios(); +- } + + cpu_synchronize_all_post_init(); + +diff --git a/numa.c b/numa.c +index daf10d8..1116c90 100644 +--- a/numa.c ++++ b/numa.c +@@ -493,19 +493,6 @@ void memory_region_allocate_system_memory(MemoryRegion *mr, Object *owner, + return; + } + +- /* The shadow_bios_after_incoming hack at savevm.c:shadow_bios() is not +- * able to handle the multiple memory blocks added when using NUMA +- * memdevs. We can disallow -numa memdev= when using rhel6.* machine-types +- * because RHEL-6 didn't support the NUMA memdev option. +- */ +- if (shadow_bios_after_incoming) { +- MachineClass *mc; +- mc = MACHINE_GET_CLASS(current_machine); +- error_report("-numa memdev is not supported by machine %s", +- mc->name); +- exit(1); +- } +- + memory_region_init(mr, owner, name, ram_size); + for (i = 0; i < nb_numa_nodes; i++) { + uint64_t size = numa_info[i].node_mem; +diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs +index 8f111c5..dfdfca7 100644 +--- a/stubs/Makefile.objs ++++ b/stubs/Makefile.objs +@@ -44,4 +44,3 @@ stub-obj-y += xen-hvm.o + stub-obj-y += pci-host-piix.o + stub-obj-y += ram-block.o + stub-obj-y += ide-isa.o +-stub-obj-y += shadow-bios.o +diff --git a/stubs/shadow-bios.c b/stubs/shadow-bios.c +deleted file mode 100644 +index c77cd7a..0000000 +--- a/stubs/shadow-bios.c ++++ /dev/null +@@ -1,7 +0,0 @@ +-#include "qemu/osdep.h" +-#include "sysemu/sysemu.h" +- +-void shadow_bios(void) +-{ +- abort(); +-} +-- +1.8.3.1 + diff --git a/SOURCES/0035-Remove-ich9_uhci123_irqpin_override.patch b/SOURCES/0035-Remove-ich9_uhci123_irqpin_override.patch new file mode 100644 index 0000000..3e48687 --- /dev/null +++ b/SOURCES/0035-Remove-ich9_uhci123_irqpin_override.patch @@ -0,0 +1,67 @@ +From 40a88676cdb22e844dce24c1745b6004a8cf7806 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 26 Apr 2018 02:54:07 +0000 +Subject: Remove ich9_uhci123_irqpin_override + +As we do not support RHEL 6 compatibility we remove this hack. + +This hack was introduced in RHEL 7.1 for BZ 1085701 and 1103581 (Guest +hits call trace migrate from RHEL6.5 to RHEL7.0 host with -M 6.1 +& balloon & uhci device) by commits 8061ffe65490 and 42a193d925b3, +and was backported to RHEL 7.0 as 0day fix (BZ 1090981, +commit 48addb5b5b3b). + +Signed-off-by: Miroslav Rezanina +--- + hw/usb/hcd-uhci.c | 13 +------------ + include/hw/usb.h | 3 --- + 2 files changed, 1 insertion(+), 15 deletions(-) + +diff --git a/hw/usb/hcd-uhci.c b/hw/usb/hcd-uhci.c +index 86d6ab8..9d7b9df 100644 +--- a/hw/usb/hcd-uhci.c ++++ b/hw/usb/hcd-uhci.c +@@ -152,8 +152,6 @@ typedef struct UHCI_QH { + uint32_t el_link; + } UHCI_QH; + +-bool ich9_uhci123_irqpin_override; +- + static void uhci_async_cancel(UHCIAsync *async); + static void uhci_queue_fill(UHCIQueue *q, UHCI_TD *td); + static void uhci_resume(void *opaque); +@@ -1222,16 +1220,7 @@ static void usb_uhci_common_realize(PCIDevice *dev, Error **errp) + /* TODO: reset value should be 0. */ + pci_conf[USB_SBRN] = USB_RELEASE_1; // release number + +- if (ich9_uhci123_irqpin_override && +- u->info.vendor_id == PCI_VENDOR_ID_INTEL && +- (u->info.device_id == PCI_DEVICE_ID_INTEL_82801I_UHCI1 || +- u->info.device_id == PCI_DEVICE_ID_INTEL_82801I_UHCI2 || +- u->info.device_id == PCI_DEVICE_ID_INTEL_82801I_UHCI3)) { +- fprintf(stderr, "RHEL-6 compat: %s: irq_pin = 3\n", u->info.name); +- irq_pin = 3; +- } else { +- irq_pin = u->info.irq_pin; +- } ++ irq_pin = u->info.irq_pin; + pci_config_set_interrupt_pin(pci_conf, irq_pin + 1); + + if (s->masterbus) { +diff --git a/include/hw/usb.h b/include/hw/usb.h +index 5b3fb1f..b943ec9 100644 +--- a/include/hw/usb.h ++++ b/include/hw/usb.h +@@ -607,9 +607,6 @@ int usb_get_quirks(uint16_t vendor_id, uint16_t product_id, + uint8_t interface_protocol); + + +-/* hcd-uhci.c -- RHEL-6 machine type compatibility */ +-extern bool ich9_uhci123_irqpin_override; +- + /* hcd-xhci.c -- rhel7.0.0 machine type compatibility */ + extern bool migrate_cve_2014_5263_xhci_fields; + +-- +1.8.3.1 + diff --git a/SOURCES/0036-s390x-css-disabled-subchannels-cannot-be-status-pend.patch b/SOURCES/0036-s390x-css-disabled-subchannels-cannot-be-status-pend.patch new file mode 100644 index 0000000..8dc4a79 --- /dev/null +++ b/SOURCES/0036-s390x-css-disabled-subchannels-cannot-be-status-pend.patch @@ -0,0 +1,50 @@ +From db20a3f422ca948784aa74375d1977b7d0a1c7ed Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Tue, 15 May 2018 07:33:45 +0000 +Subject: s390x/css: disabled subchannels cannot be status pending + +The 3270 code will try to post an attention interrupt when the +3270 emulator (e.g. x3270) attaches. If the guest has not yet +enabled the subchannel for the 3270 device, we will present a spurious +cc 1 (status pending) when it uses msch on it later on, e.g. when +trying to enable the subchannel. + +To fix this, just don't do anything in css_conditional_io_interrupt() +if the subchannel is not enabled. The 3270 code will work fine with +that, and the other user of this function (virtio-ccw) never +attempts to post an interrupt for a disabled device to begin with. + +CC: qemu-stable@nongnu.org +Reported-by: Thomas Huth +Tested-by: Thomas Huth +Acked-by: Christian Borntraeger +Acked-by: Halil Pasic +Reviewed-by: David Hildenbrand +Signed-off-by: Cornelia Huck +(cherry picked from commit 6e9c893ecd00afd5344c35d0d0ded50eaa0938f6) +--- + hw/s390x/css.c | 8 ++++++++ + 1 file changed, 8 insertions(+) + +diff --git a/hw/s390x/css.c b/hw/s390x/css.c +index 301bf17..56c3fa8 100644 +--- a/hw/s390x/css.c ++++ b/hw/s390x/css.c +@@ -617,6 +617,14 @@ void css_inject_io_interrupt(SubchDev *sch) + void css_conditional_io_interrupt(SubchDev *sch) + { + /* ++ * If the subchannel is not enabled, it is not made status pending ++ * (see PoP p. 16-17, "Status Control"). ++ */ ++ if (!(sch->curr_status.pmcw.flags & PMCW_FLAGS_MASK_ENA)) { ++ return; ++ } ++ ++ /* + * If the subchannel is not currently status pending, make it pending + * with alert status. + */ +-- +1.8.3.1 + diff --git a/SOURCES/0037-virtio-ccw-common-reset-handler.patch b/SOURCES/0037-virtio-ccw-common-reset-handler.patch new file mode 100644 index 0000000..4dfd7f7 --- /dev/null +++ b/SOURCES/0037-virtio-ccw-common-reset-handler.patch @@ -0,0 +1,129 @@ +From 4772dbd9b905b7b304f24fe3d2e4ca8ba0a18816 Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Tue, 15 May 2018 07:33:46 +0000 +Subject: virtio-ccw: common reset handler + +All the different virtio ccw devices use the same reset handler, +so let's move setting it into the base virtio ccw device class. + +CC: qemu-stable@nongnu.org +Reviewed-by: Thomas Huth +Reviewed-by: David Hildenbrand +Reviewed-by: Halil Pasic +Signed-off-by: Cornelia Huck +(cherry picked from commit 0c53057adb04d254bc09511880670c92ab185fc6) +--- + hw/s390x/virtio-ccw.c | 13 +------------ + 1 file changed, 1 insertion(+), 12 deletions(-) + +diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c +index 8720e46..2db8cc6 100644 +--- a/hw/s390x/virtio-ccw.c ++++ b/hw/s390x/virtio-ccw.c +@@ -1348,7 +1348,6 @@ static void virtio_ccw_net_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_net_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_net_properties; + set_bit(DEVICE_CATEGORY_NETWORK, dc->categories); + } +@@ -1376,7 +1375,6 @@ static void virtio_ccw_blk_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_blk_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_blk_properties; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + } +@@ -1404,7 +1402,6 @@ static void virtio_ccw_serial_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_serial_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_serial_properties; + set_bit(DEVICE_CATEGORY_INPUT, dc->categories); + } +@@ -1432,7 +1429,6 @@ static void virtio_ccw_balloon_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_balloon_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_balloon_properties; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + } +@@ -1460,7 +1456,6 @@ static void virtio_ccw_scsi_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_scsi_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_scsi_properties; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + } +@@ -1487,7 +1482,6 @@ static void vhost_ccw_scsi_class_init(ObjectClass *klass, void *data) + + k->realize = vhost_ccw_scsi_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = vhost_ccw_scsi_properties; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + } +@@ -1524,7 +1518,6 @@ static void virtio_ccw_rng_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_rng_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_rng_properties; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + } +@@ -1564,7 +1557,6 @@ static void virtio_ccw_crypto_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_crypto_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_crypto_properties; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + } +@@ -1603,7 +1595,6 @@ static void virtio_ccw_gpu_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_gpu_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_gpu_properties; + dc->hotpluggable = false; + set_bit(DEVICE_CATEGORY_DISPLAY, dc->categories); +@@ -1632,7 +1623,6 @@ static void virtio_ccw_input_class_init(ObjectClass *klass, void *data) + + k->realize = virtio_ccw_input_realize; + k->unrealize = virtio_ccw_unrealize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_input_properties; + set_bit(DEVICE_CATEGORY_INPUT, dc->categories); + } +@@ -1736,6 +1726,7 @@ static void virtio_ccw_device_class_init(ObjectClass *klass, void *data) + dc->realize = virtio_ccw_busdev_realize; + dc->unrealize = virtio_ccw_busdev_unrealize; + dc->bus_type = TYPE_VIRTUAL_CSS_BUS; ++ dc->reset = virtio_ccw_reset; + } + + static const TypeInfo virtio_ccw_device_info = { +@@ -1812,7 +1803,6 @@ static void virtio_ccw_9p_class_init(ObjectClass *klass, void *data) + + k->unrealize = virtio_ccw_unrealize; + k->realize = virtio_ccw_9p_realize; +- dc->reset = virtio_ccw_reset; + dc->props = virtio_ccw_9p_properties; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + } +@@ -1862,7 +1852,6 @@ static void vhost_vsock_ccw_class_init(ObjectClass *klass, void *data) + k->unrealize = virtio_ccw_unrealize; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + dc->props = vhost_vsock_ccw_properties; +- dc->reset = virtio_ccw_reset; + } + + static void vhost_vsock_ccw_instance_init(Object *obj) +-- +1.8.3.1 + diff --git a/SOURCES/0038-s390x-ccw-make-sure-all-ccw-devices-are-properly-res.patch b/SOURCES/0038-s390x-ccw-make-sure-all-ccw-devices-are-properly-res.patch new file mode 100644 index 0000000..703e0dd --- /dev/null +++ b/SOURCES/0038-s390x-ccw-make-sure-all-ccw-devices-are-properly-res.patch @@ -0,0 +1,105 @@ +From 39b8d397fe34ae375e33371ee58894d13667560b Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Tue, 15 May 2018 07:33:47 +0000 +Subject: s390x/ccw: make sure all ccw devices are properly reset + +Thomas reported that the subchannel for a 3270 device that ended up +in a broken state (status pending even though not enabled) did not +get out of that state even after a reboot (which involves a subsytem +reset). The reason for this is that the 3270 device did not define +a reset handler. + +Let's fix this by introducing a base reset handler (set up for all +ccw devices) that resets the subchannel and have virtio-ccw call +its virtio-specific reset procedure in addition to that. + +CC: qemu-stable@nongnu.org +Reported-by: Thomas Huth +Suggested-by: Christian Borntraeger +Reviewed-by: Thomas Huth +Tested-by: Thomas Huth +Acked-by: Christian Borntraeger +Reviewed-by: Halil Pasic +Signed-off-by: Cornelia Huck +(cherry picked from commit 838fb84f83c84f00d15b1bede5e080b495644458) +--- + hw/s390x/ccw-device.c | 8 ++++++++ + hw/s390x/virtio-ccw.c | 9 ++++++--- + hw/s390x/virtio-ccw.h | 1 + + 3 files changed, 15 insertions(+), 3 deletions(-) + +diff --git a/hw/s390x/ccw-device.c b/hw/s390x/ccw-device.c +index f9bfa15..7cd73df 100644 +--- a/hw/s390x/ccw-device.c ++++ b/hw/s390x/ccw-device.c +@@ -40,6 +40,13 @@ static Property ccw_device_properties[] = { + DEFINE_PROP_END_OF_LIST(), + }; + ++static void ccw_device_reset(DeviceState *d) ++{ ++ CcwDevice *ccw_dev = CCW_DEVICE(d); ++ ++ css_reset_sch(ccw_dev->sch); ++} ++ + static void ccw_device_class_init(ObjectClass *klass, void *data) + { + DeviceClass *dc = DEVICE_CLASS(klass); +@@ -48,6 +55,7 @@ static void ccw_device_class_init(ObjectClass *klass, void *data) + k->realize = ccw_device_realize; + k->refill_ids = ccw_device_refill_ids; + dc->props = ccw_device_properties; ++ dc->reset = ccw_device_reset; + } + + const VMStateDescription vmstate_ccw_dev = { +diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c +index 2db8cc6..dfedd84 100644 +--- a/hw/s390x/virtio-ccw.c ++++ b/hw/s390x/virtio-ccw.c +@@ -1061,10 +1061,12 @@ static void virtio_ccw_reset(DeviceState *d) + { + VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d); + VirtIODevice *vdev = virtio_bus_get_device(&dev->bus); +- CcwDevice *ccw_dev = CCW_DEVICE(d); ++ VirtIOCCWDeviceClass *vdc = VIRTIO_CCW_DEVICE_GET_CLASS(dev); + + virtio_ccw_reset_virtio(dev, vdev); +- css_reset_sch(ccw_dev->sch); ++ if (vdc->parent_reset) { ++ vdc->parent_reset(d); ++ } + } + + static void virtio_ccw_vmstate_change(DeviceState *d, bool running) +@@ -1721,12 +1723,13 @@ static void virtio_ccw_device_class_init(ObjectClass *klass, void *data) + { + DeviceClass *dc = DEVICE_CLASS(klass); + CCWDeviceClass *k = CCW_DEVICE_CLASS(dc); ++ VirtIOCCWDeviceClass *vdc = VIRTIO_CCW_DEVICE_CLASS(klass); + + k->unplug = virtio_ccw_busdev_unplug; + dc->realize = virtio_ccw_busdev_realize; + dc->unrealize = virtio_ccw_busdev_unrealize; + dc->bus_type = TYPE_VIRTUAL_CSS_BUS; +- dc->reset = virtio_ccw_reset; ++ device_class_set_parent_reset(dc, virtio_ccw_reset, &vdc->parent_reset); + } + + static const TypeInfo virtio_ccw_device_info = { +diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h +index 2fc5130..3453aa1 100644 +--- a/hw/s390x/virtio-ccw.h ++++ b/hw/s390x/virtio-ccw.h +@@ -77,6 +77,7 @@ typedef struct VirtIOCCWDeviceClass { + CCWDeviceClass parent_class; + void (*realize)(VirtioCcwDevice *dev, Error **errp); + void (*unrealize)(VirtioCcwDevice *dev, Error **errp); ++ void (*parent_reset)(DeviceState *dev); + } VirtIOCCWDeviceClass; + + /* Performance improves when virtqueue kick processing is decoupled from the +-- +1.8.3.1 + diff --git a/SOURCES/0039-s390x-Re-enable-CONFIG_TERMINAL3270.patch b/SOURCES/0039-s390x-Re-enable-CONFIG_TERMINAL3270.patch new file mode 100644 index 0000000..89685bd --- /dev/null +++ b/SOURCES/0039-s390x-Re-enable-CONFIG_TERMINAL3270.patch @@ -0,0 +1,32 @@ +From 0d4f38c339fa1e200a01fdf8b17a0a5c22d07ae2 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Tue, 15 May 2018 07:33:48 +0000 +Subject: s390x: Re-enable CONFIG_TERMINAL3270 + +Upstream-status: n/a (downstream only config change) + +This is required to be able to connect to the guest via a 3270 +terminal. + +Signed-off-by: Thomas Huth +--- + default-configs/s390x-softmmu.mak | 3 +-- + 1 file changed, 1 insertion(+), 2 deletions(-) + +diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak +index 649bf2c..17e871a 100644 +--- a/default-configs/s390x-softmmu.mak ++++ b/default-configs/s390x-softmmu.mak +@@ -4,8 +4,7 @@ CONFIG_PCI=y + #CONFIG_VHOST_USER_BLK=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) + CONFIG_VIRTIO=y + CONFIG_SCLPCONSOLE=y +-# Disabled for Red Hat Enterprise Linux: +-# CONFIG_TERMINAL3270=y ++CONFIG_TERMINAL3270=y + CONFIG_S390_FLIC=y + CONFIG_S390_FLIC_KVM=$(CONFIG_KVM) + # Disabled for Red Hat Enterprise Linux: +-- +1.8.3.1 + diff --git a/SOURCES/0041-redhat-define-pseries-rhel7.6.0-machine-types.patch b/SOURCES/0041-redhat-define-pseries-rhel7.6.0-machine-types.patch new file mode 100644 index 0000000..0180fc6 --- /dev/null +++ b/SOURCES/0041-redhat-define-pseries-rhel7.6.0-machine-types.patch @@ -0,0 +1,70 @@ +From 7574808ac154ac9ddf8264bf14e775fab96d0cac Mon Sep 17 00:00:00 2001 +From: Laurent Vivier +Date: Thu, 7 Jun 2018 12:55:12 +0000 +Subject: redhat: define pseries-rhel7.6.0 machine types + +BREW: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=16632825 +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1585651 +BRANCH: rhel8/master-2.12.0 +UPSTREAM: downsream only +Tested: minimal, as the change is the same as for RHEL7.6 + Tested ping-pong migration between two + pseries-rhel7.6.0/qemu-kvm-2.12.0 and with + pseries-rhel7.5.0/qemu-kvm-rhev-2.10 + +Signed-off-by: Laurent Vivier +--- + hw/ppc/spapr.c | 26 ++++++++++++++++++++++++-- + 1 file changed, 24 insertions(+), 2 deletions(-) + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index c751111..c3f08b3 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -4352,19 +4352,41 @@ DEFINE_SPAPR_MACHINE(2_1, "2.1", false); + #endif + + /* ++ * pseries-rhel7.6.0 ++ */ ++ ++static void spapr_machine_rhel760_instance_options(MachineState *machine) ++{ ++} ++ ++static void spapr_machine_rhel760_class_options(MachineClass *mc) ++{ ++ /* Defaults for the latest behaviour inherited from the base class */ ++} ++ ++DEFINE_SPAPR_MACHINE(rhel760, "rhel7.6.0", true); ++ ++/* + * pseries-rhel7.5.0 ++ * like SPAPR_COMPAT_2_11 and SPAPR_COMPAT_2_10 ++ * SPAPR_CAP_HTM already enabled in 7.4 ++ * + */ ++#define SPAPR_COMPAT_RHEL7_5 \ ++ HW_COMPAT_RHEL7_5 \ + + static void spapr_machine_rhel750_instance_options(MachineState *machine) + { ++ spapr_machine_rhel760_instance_options(machine); + } + + static void spapr_machine_rhel750_class_options(MachineClass *mc) + { +- /* Defaults for the latest behaviour inherited from the base class */ ++ spapr_machine_rhel760_class_options(mc); ++ SET_MACHINE_COMPAT(mc, SPAPR_COMPAT_RHEL7_5); + } + +-DEFINE_SPAPR_MACHINE(rhel750, "rhel7.5.0", true); ++DEFINE_SPAPR_MACHINE(rhel750, "rhel7.5.0", false); + + /* + * pseries-rhel7.5.0-sxxm +-- +1.8.3.1 + diff --git a/SOURCES/0042-s390x-cpumodels-add-z14-Model-ZR1.patch b/SOURCES/0042-s390x-cpumodels-add-z14-Model-ZR1.patch new file mode 100644 index 0000000..cc28f47 --- /dev/null +++ b/SOURCES/0042-s390x-cpumodels-add-z14-Model-ZR1.patch @@ -0,0 +1,32 @@ +From f53aa3f10b0c22093917fc076e3ddcb41398f12a Mon Sep 17 00:00:00 2001 +From: Christian Borntraeger +Date: Wed, 20 Jun 2018 10:58:31 +0000 +Subject: s390x/cpumodels: add z14 Model ZR1 + +Introduce the new z14 Model ZR1 cpu model. Mostly identical to z14, only +the cpu type differs (3906 vs. 3907) + +Signed-off-by: Christian Borntraeger +Message-Id: <20180613081819.147178-1-borntraeger@de.ibm.com> +Reviewed-by: David Hildenbrand +Signed-off-by: Cornelia Huck +(cherry picked from commit 23ad956bff98d949057156ea3f68a9763c2dda0e) +--- + target/s390x/cpu_models.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c +index c4016e0..24e689c 100644 +--- a/target/s390x/cpu_models.c ++++ b/target/s390x/cpu_models.c +@@ -79,6 +79,7 @@ static S390CPUDef s390_cpu_defs[] = { + CPUDEF_INIT(0x2964, 13, 2, 47, 0x08000000U, "z13.2", "IBM z13 GA2"), + CPUDEF_INIT(0x2965, 13, 2, 47, 0x08000000U, "z13s", "IBM z13s GA1"), + CPUDEF_INIT(0x3906, 14, 1, 47, 0x08000000U, "z14", "IBM z14 GA1"), ++ CPUDEF_INIT(0x3907, 14, 1, 47, 0x08000000U, "z14ZR1", "IBM z14 Model ZR1 GA1"), + }; + + #define QEMU_MAX_CPU_TYPE 0x2827 +-- +1.8.3.1 + diff --git a/SOURCES/81-kvm-rhel.rules b/SOURCES/81-kvm-rhel.rules new file mode 100644 index 0000000..787cad6 --- /dev/null +++ b/SOURCES/81-kvm-rhel.rules @@ -0,0 +1 @@ +DEVPATH=="*/kvm", ACTION=="change", RUN+="/lib/udev/udev-kvm-check $env{COUNT} $env{EVENT}" diff --git a/SOURCES/85-kvm.preset b/SOURCES/85-kvm.preset new file mode 100644 index 0000000..8024052 --- /dev/null +++ b/SOURCES/85-kvm.preset @@ -0,0 +1,5 @@ +# Enable kvm-setup by default. This can have odd side effects on +# PowerNV systems that aren't intended as KVM hosts, but at present we +# only support RHEL on PowerNV for the purpose of being a RHEV host. + +enable kvm-setup.service diff --git a/SOURCES/95-kvm-memlock.conf b/SOURCES/95-kvm-memlock.conf new file mode 100644 index 0000000..fc59dbe --- /dev/null +++ b/SOURCES/95-kvm-memlock.conf @@ -0,0 +1,10 @@ +# The KVM HV implementation on Power can require a significant amount +# of unswappable memory (about half of which also needs to be host +# physically contiguous) to hold the guest's Hash Page Table (HPT) - +# roughly 1/64th of the guest's RAM size, minimum 16MiB. +# +# These limits allow unprivileged users to start smallish VMs, such as +# those used by libguestfs. +# +* hard memlock 65536 +* soft memlock 65536 diff --git a/SOURCES/99-qemu-guest-agent.rules b/SOURCES/99-qemu-guest-agent.rules new file mode 100644 index 0000000..8a290ab --- /dev/null +++ b/SOURCES/99-qemu-guest-agent.rules @@ -0,0 +1,2 @@ +SUBSYSTEM=="virtio-ports", ATTR{name}=="org.qemu.guest_agent.0", \ + TAG+="systemd" ENV{SYSTEMD_WANTS}="qemu-guest-agent.service" diff --git a/SOURCES/README.tests b/SOURCES/README.tests new file mode 100644 index 0000000..9932773 --- /dev/null +++ b/SOURCES/README.tests @@ -0,0 +1,39 @@ +qemu-kvm-tests README +===================== + +The qemu-kvm-tests rpm contains tests that can be used to verify the +functionality of the installed qemu-kvm package + +When installed, the files from this rpm will be arranged in the following +directory structure + +tests-src/ +├── README +├── scripts +│   ├── qemu.py +│   └── qmp +└── tests + ├── acceptance + ├── Makefile.include + └── qemu-iotests + +The tests/ directory within the tests-src/ directory is setup to remain a copy +of a subset of the tests/ directory from the QEMU source tree + +The avocado_qemu tests and qemu-iotests, along with files required for the +execution of the avocado_qemu tests (scripts/qemu.py and scripts/qmp/) will be +installed in a new location - /usr/lib64/qemu-kvm/tests-src/ + +avocado_qemu tests: +The avocado_qemu tests can be executed by running the following avocado command: +avocado run -p qemu_bin=/usr/libexec/qemu-kvm /usr/lib64/qemu-kvm/tests/acceptance/ +Avocado needs to be installed separately using either pip or from source as +Avocado is not being packaged for RHEL-8. + +qemu-iotests: +symlinks to corresponding binaries need to be created for QEMU_PROG, +QEMU_IO_PROG, QEMU_IMG_PROG, and QEMU_NBD_PROG before the iotests can be +executed. + +The primary purpose of this package is to make these tests available to be +executed as gating tests for the virt module in the RHEL-8 OSCI environment. diff --git a/SOURCES/bridge.conf b/SOURCES/bridge.conf new file mode 100644 index 0000000..a573665 --- /dev/null +++ b/SOURCES/bridge.conf @@ -0,0 +1 @@ +allow virbr0 diff --git a/SOURCES/ksm.service b/SOURCES/ksm.service new file mode 100644 index 0000000..35c6f1d --- /dev/null +++ b/SOURCES/ksm.service @@ -0,0 +1,13 @@ +[Unit] +Description=Kernel Samepage Merging +ConditionPathExists=/sys/kernel/mm/ksm + +[Service] +Type=oneshot +RemainAfterExit=yes +EnvironmentFile=-/etc/sysconfig/ksm +ExecStart=/usr/libexec/ksmctl start +ExecStop=/usr/libexec/ksmctl stop + +[Install] +WantedBy=multi-user.target diff --git a/SOURCES/ksm.sysconfig b/SOURCES/ksm.sysconfig new file mode 100644 index 0000000..d99656d --- /dev/null +++ b/SOURCES/ksm.sysconfig @@ -0,0 +1,4 @@ +# The maximum number of unswappable kernel pages +# which may be allocated by ksm (0 for unlimited) +# If unset, defaults to half of total memory +# KSM_MAX_KERNEL_PAGES= diff --git a/SOURCES/ksmctl.c b/SOURCES/ksmctl.c new file mode 100644 index 0000000..af39591 --- /dev/null +++ b/SOURCES/ksmctl.c @@ -0,0 +1,77 @@ +/* Start/stop KSM, for systemd. + * Copyright (C) 2009, 2011 Red Hat, Inc. + * Written by Paolo Bonzini . + * Based on the original sysvinit script by Dan Kenigsberg + * This file is distributed under the GNU General Public License, version 2 + * or later. */ + +#include +#include +#include +#include +#include +#include + +#define KSM_MAX_KERNEL_PAGES_FILE "/sys/kernel/mm/ksm/max_kernel_pages" +#define KSM_RUN_FILE "/sys/kernel/mm/ksm/run" + +char *program_name; + +int usage(void) +{ + fprintf(stderr, "Usage: %s {start|stop}\n", program_name); + return 1; +} + +int write_value(uint64_t value, char *filename) +{ + FILE *fp; + if (!(fp = fopen(filename, "w")) || + fprintf(fp, "%llu\n", (unsigned long long) value) == EOF || + fflush(fp) == EOF || + fclose(fp) == EOF) + return 1; + + return 0; +} + +uint64_t ksm_max_kernel_pages() +{ + char *var = getenv("KSM_MAX_KERNEL_PAGES"); + char *endptr; + uint64_t value; + if (var && *var) { + value = strtoll(var, &endptr, 0); + if (value < LLONG_MAX && !*endptr) + return value; + } + /* Unless KSM_MAX_KERNEL_PAGES is set, let KSM munch up to half of + * total memory. */ + return sysconf(_SC_PHYS_PAGES) / 2; +} + +int start(void) +{ + if (access(KSM_MAX_KERNEL_PAGES_FILE, R_OK) >= 0) + write_value(ksm_max_kernel_pages(), KSM_MAX_KERNEL_PAGES_FILE); + return write_value(1, KSM_RUN_FILE); +} + +int stop(void) +{ + return write_value(0, KSM_RUN_FILE); +} + +int main(int argc, char **argv) +{ + program_name = argv[0]; + if (argc < 2) { + return usage(); + } else if (!strcmp(argv[1], "start")) { + return start(); + } else if (!strcmp(argv[1], "stop")) { + return stop(); + } else { + return usage(); + } +} diff --git a/SOURCES/ksmtuned b/SOURCES/ksmtuned new file mode 100644 index 0000000..7bc5743 --- /dev/null +++ b/SOURCES/ksmtuned @@ -0,0 +1,139 @@ +#!/bin/bash +# +# Copyright 2009 Red Hat, Inc. and/or its affiliates. +# Released under the GPL +# +# Author: Dan Kenigsberg +# +# ksmtuned - a simple script that controls whether (and with what vigor) ksm +# should search for duplicated pages. +# +# starts ksm when memory commited to qemu processes exceeds a threshold, and +# make ksm work harder and harder untill memory load falls below that +# threshold. +# +# send SIGUSR1 to this process right after a new qemu process is started, or +# following its death, to retune ksm accordingly +# +# needs testing and ironing. contact danken@redhat.com if something breaks. + +if [ -f /etc/ksmtuned.conf ]; then + . /etc/ksmtuned.conf +fi + +debug() { + if [ -n "$DEBUG" ]; then + s="`/bin/date`: $*" + [ -n "$LOGFILE" ] && echo "$s" >> "$LOGFILE" || echo "$s" + fi +} + + +KSM_MONITOR_INTERVAL=${KSM_MONITOR_INTERVAL:-60} +KSM_NPAGES_BOOST=${KSM_NPAGES_BOOST:-300} +KSM_NPAGES_DECAY=${KSM_NPAGES_DECAY:--50} + +KSM_NPAGES_MIN=${KSM_NPAGES_MIN:-64} +KSM_NPAGES_MAX=${KSM_NPAGES_MAX:-1250} +# millisecond sleep between ksm scans for 16Gb server. Smaller servers sleep +# more, bigger sleep less. +KSM_SLEEP_MSEC=${KSM_SLEEP_MSEC:-10} + +KSM_THRES_COEF=${KSM_THRES_COEF:-20} +KSM_THRES_CONST=${KSM_THRES_CONST:-2048} + +total=`awk '/^MemTotal:/ {print $2}' /proc/meminfo` +debug total $total + +npages=0 +sleep=$[KSM_SLEEP_MSEC * 16 * 1024 * 1024 / total] +[ $sleep -le 10 ] && sleep=10 +debug sleep $sleep +thres=$[total * KSM_THRES_COEF / 100] +if [ $KSM_THRES_CONST -gt $thres ]; then + thres=$KSM_THRES_CONST +fi +debug thres $thres + +KSMCTL () { + case x$1 in + xstop) + echo 0 > /sys/kernel/mm/ksm/run + ;; + xstart) + echo $2 > /sys/kernel/mm/ksm/pages_to_scan + echo $3 > /sys/kernel/mm/ksm/sleep_millisecs + echo 1 > /sys/kernel/mm/ksm/run + ;; + esac +} + +committed_memory () { + # calculate how much memory is committed to running qemu processes + local pidlist + pidlist=$(pgrep -d ' ' -- '^qemu(-(kvm|system-.+)|:.{1,11})$') + if [ -n "$pidlist" ]; then + ps -p "$pidlist" -o rsz= + fi | awk '{ sum += $1 }; END { print 0+sum }' +} + +free_memory () { + awk '/^(MemFree|Buffers|Cached):/ {free += $2}; END {print free}' \ + /proc/meminfo +} + +increase_npages() { + local delta + delta=${1:-0} + npages=$[npages + delta] + if [ $npages -lt $KSM_NPAGES_MIN ]; then + npages=$KSM_NPAGES_MIN + elif [ $npages -gt $KSM_NPAGES_MAX ]; then + npages=$KSM_NPAGES_MAX + fi + echo $npages +} + + +adjust () { + local free committed + free=`free_memory` + committed=`committed_memory` + debug committed $committed free $free + if [ $[committed + thres] -lt $total -a $free -gt $thres ]; then + KSMCTL stop + debug "$[committed + thres] < $total and free > $thres, stop ksm" + return 1 + fi + debug "$[committed + thres] > $total, start ksm" + if [ $free -lt $thres ]; then + npages=`increase_npages $KSM_NPAGES_BOOST` + debug "$free < $thres, boost" + else + npages=`increase_npages $KSM_NPAGES_DECAY` + debug "$free > $thres, decay" + fi + KSMCTL start $npages $sleep + debug "KSMCTL start $npages $sleep" + return 0 +} + +function nothing () { + : +} + +loop () { + trap nothing SIGUSR1 + while true + do + sleep $KSM_MONITOR_INTERVAL & + wait $! + adjust + done +} + +PIDFILE=${PIDFILE-/var/run/ksmtune.pid} +if touch "$PIDFILE"; then + loop & + echo $! > "$PIDFILE" +fi diff --git a/SOURCES/ksmtuned.conf b/SOURCES/ksmtuned.conf new file mode 100644 index 0000000..fc4518c --- /dev/null +++ b/SOURCES/ksmtuned.conf @@ -0,0 +1,21 @@ +# Configuration file for ksmtuned. + +# How long ksmtuned should sleep between tuning adjustments +# KSM_MONITOR_INTERVAL=60 + +# Millisecond sleep between ksm scans for 16Gb server. +# Smaller servers sleep more, bigger sleep less. +# KSM_SLEEP_MSEC=10 + +# KSM_NPAGES_BOOST=300 +# KSM_NPAGES_DECAY=-50 +# KSM_NPAGES_MIN=64 +# KSM_NPAGES_MAX=1250 + +# KSM_THRES_COEF=20 +# KSM_THRES_CONST=2048 + +# uncomment the following if you want ksmtuned debug info + +# LOGFILE=/var/log/ksmtuned +# DEBUG=1 diff --git a/SOURCES/ksmtuned.service b/SOURCES/ksmtuned.service new file mode 100644 index 0000000..39febcc --- /dev/null +++ b/SOURCES/ksmtuned.service @@ -0,0 +1,12 @@ +[Unit] +Description=Kernel Samepage Merging (KSM) Tuning Daemon +After=ksm.service +Requires=ksm.service + +[Service] +ExecStart=/usr/sbin/ksmtuned +ExecReload=/bin/kill -USR1 $MAINPID +Type=forking + +[Install] +WantedBy=multi-user.target diff --git a/SOURCES/kvm-AArch64-Add-virt-rhel7.6-machine-type.patch b/SOURCES/kvm-AArch64-Add-virt-rhel7.6-machine-type.patch new file mode 100644 index 0000000..cd1720a --- /dev/null +++ b/SOURCES/kvm-AArch64-Add-virt-rhel7.6-machine-type.patch @@ -0,0 +1,51 @@ +From 9cb37fdbeafdbdc28cf224fd7905a7d678961505 Mon Sep 17 00:00:00 2001 +From: Wei Huang +Date: Wed, 28 Mar 2018 18:58:55 +0200 +Subject: [PATCH 001/268] AArch64: Add virt-rhel7.6 machine type + +RH-Author: Wei Huang +Message-id: <20180328185856.20056-2-wei@redhat.com> +Patchwork-id: 79427 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/2] AArch64: Add virt-rhel7.6 machine type +Bugzilla: 1558723 +RH-Acked-by: Andrew Jones +RH-Acked-by: Auger Eric +RH-Acked-by: Laszlo Ersek + +This patch adds "virt-rhel7.6" machine type. Because RHEL 7.5 virt-arm +was a preview product, we remove the "virt-rhel7.5" type to avoid the +legacy support burden. + +Signed-off-by: Wei Huang +Signed-off-by: Miroslav Rezanina +--- + hw/arm/virt.c | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/hw/arm/virt.c b/hw/arm/virt.c +index 806eb1e..a4d0f52 100644 +--- a/hw/arm/virt.c ++++ b/hw/arm/virt.c +@@ -1820,7 +1820,7 @@ static void rhel_machine_init(void) + } + type_init(rhel_machine_init); + +-static void rhel750_virt_instance_init(Object *obj) ++static void rhel760_virt_instance_init(Object *obj) + { + VirtMachineState *vms = VIRT_MACHINE(obj); + VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms); +@@ -1862,8 +1862,8 @@ static void rhel750_virt_instance_init(Object *obj) + vms->irqmap=a15irqmap; + } + +-static void rhel750_virt_options(MachineClass *mc) ++static void rhel760_virt_options(MachineClass *mc) + { + SET_MACHINE_COMPAT(mc, ARM_RHEL_COMPAT); + } +-DEFINE_RHEL_MACHINE_AS_LATEST(7, 5, 0) ++DEFINE_RHEL_MACHINE_AS_LATEST(7, 6, 0) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Acceptance-tests-add-Linux-kernel-boot-and-console-c.patch b/SOURCES/kvm-Acceptance-tests-add-Linux-kernel-boot-and-console-c.patch new file mode 100644 index 0000000..809c4a2 --- /dev/null +++ b/SOURCES/kvm-Acceptance-tests-add-Linux-kernel-boot-and-console-c.patch @@ -0,0 +1,105 @@ +From a326b17336ae12d9fa492ea34b9b1b08150262d0 Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:39 +0000 +Subject: [PATCH 11/13] Acceptance tests: add Linux kernel boot and console + checking test +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: +Patchwork-id: 83433 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 5/7] Acceptance tests: add Linux kernel boot and console checking test +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +This test boots a Linux kernel, and checks that the given command +line was effective in two ways: + + * It makes the kernel use the set "console device" as a console + * The kernel records the command line as expected in the console + +Given that way too many error conditions may occur, and detecting the +kernel boot progress status may not be trivial, this test relies on a +timeout to handle unexpected situations. Also, it's *not* tagged as a +quick test for obvious reasons. + +It may be useful, while interactively running/debugging this test, or +tests similar to this one, to show some of the logging channels. +Example: + + $ avocado --show=QMP,console run boot_linux_console.py + +Signed-off-by: Cleber Rosa +Message-Id: <20180530184156.15634-6-crosa@redhat.com> +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Eduardo Habkost +(cherry picked from commit c1cc73f407b890c4e7ab5bf520c0637e0364e92a) +Signed-off-by: Yash Mankad +Signed-off-by: Danilo C. L. de Paula +--- + tests/acceptance/boot_linux_console.py | 47 ++++++++++++++++++++++++++++++++++ + 1 file changed, 47 insertions(+) + create mode 100644 tests/acceptance/boot_linux_console.py + +diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py +new file mode 100644 +index 0000000..98324f7 +--- /dev/null ++++ b/tests/acceptance/boot_linux_console.py +@@ -0,0 +1,47 @@ ++# Functional test that boots a Linux kernel and checks the console ++# ++# Copyright (c) 2018 Red Hat, Inc. ++# ++# Author: ++# Cleber Rosa ++# ++# This work is licensed under the terms of the GNU GPL, version 2 or ++# later. See the COPYING file in the top-level directory. ++ ++import logging ++ ++from avocado_qemu import Test ++ ++ ++class BootLinuxConsole(Test): ++ """ ++ Boots a x86_64 Linux kernel and checks that the console is operational ++ and the kernel command line is properly passed from QEMU to the kernel ++ ++ :avocado: enable ++ :avocado: tags=x86_64 ++ """ ++ ++ timeout = 60 ++ ++ def test(self): ++ kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' ++ 'Everything/x86_64/os/images/pxeboot/vmlinuz') ++ kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' ++ kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) ++ ++ self.vm.set_machine('pc') ++ self.vm.set_console() ++ kernel_command_line = 'console=ttyS0' ++ self.vm.add_args('-kernel', kernel_path, ++ '-append', kernel_command_line) ++ self.vm.launch() ++ console = self.vm.console_socket.makefile() ++ console_logger = logging.getLogger('console') ++ while True: ++ msg = console.readline() ++ console_logger.debug(msg.strip()) ++ if 'Kernel command line: %s' % kernel_command_line in msg: ++ break ++ if 'Kernel panic - not syncing' in msg: ++ self.fail("Kernel panic reached") +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Acceptance-tests-add-make-rule-for-running-them.patch b/SOURCES/kvm-Acceptance-tests-add-make-rule-for-running-them.patch new file mode 100644 index 0000000..30730ae --- /dev/null +++ b/SOURCES/kvm-Acceptance-tests-add-make-rule-for-running-them.patch @@ -0,0 +1,180 @@ +From ae8198a11e507c4f4f701aa92c3ae531d140e547 Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:41 +0000 +Subject: [PATCH 13/13] Acceptance tests: add make rule for running them +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: <9527fefa2d8d1b27d4a647cf8355236b61fb028b.1544573601.git.ymankad@redhat.com> +Patchwork-id: 83439 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 7/7] Acceptance tests: add make rule for running them +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +The acceptance (aka functional, aka Avocado-based) tests are +Python files located in "tests/acceptance" that need to be run +with the Avocado libs and test runner. + +Let's provide a convenient way for QEMU developers to run them, +by making use of the tests-venv with the required setup. + +Also, while the Avocado test runner will take care of creating a +location to save test results to, it was understood that it's better +if the results are kept within the build tree. + +Signed-off-by: Cleber Rosa +Acked-by: Stefan Hajnoczi +Acked-by: Wainer dos Santos Moschetta +Reviewed-by: Caio Carrara +Message-Id: <20181018153134.8493-3-crosa@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit a56931eef343c7564e35bcc05eaed2a469a1b1b8) +Signed-off-by: Yash Mankad +Signed-off-by: Danilo C. L. de Paula +--- + docs/devel/testing.rst | 43 ++++++++++++++++++++++++++++++++++++++----- + tests/Makefile.include | 21 +++++++++++++++++++-- + tests/requirements.txt | 1 + + 3 files changed, 58 insertions(+), 7 deletions(-) + +diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst +index f33e5a8..db08a80 100644 +--- a/docs/devel/testing.rst ++++ b/docs/devel/testing.rst +@@ -524,10 +524,39 @@ Tests based on ``avocado_qemu.Test`` can easily: + - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test + - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html + +-Installation +------------- ++Running tests ++------------- ++ ++You can run the acceptance tests simply by executing: ++ ++.. code:: ++ ++ make check-acceptance ++ ++This involves the automatic creation of Python virtual environment ++within the build tree (at ``tests/venv``) which will have all the ++right dependencies, and will save tests results also within the ++build tree (at ``tests/results``). + +-To install Avocado and its dependencies, run: ++Note: the build environment must be using a Python 3 stack, and have ++the ``venv`` and ``pip`` packages installed. If necessary, make sure ++``configure`` is called with ``--python=`` and that those modules are ++available. On Debian and Ubuntu based systems, depending on the ++specific version, they may be on packages named ``python3-venv`` and ++``python3-pip``. ++ ++The scripts installed inside the virtual environment may be used ++without an "activation". For instance, the Avocado test runner ++may be invoked by running: ++ ++ .. code:: ++ ++ tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/ ++ ++Manual Installation ++------------------- ++ ++To manually install Avocado and its dependencies, run: + + .. code:: + +@@ -668,11 +697,15 @@ The exact QEMU binary to be used on QEMUMachine. + Uninstalling Avocado + -------------------- + +-If you've followed the installation instructions above, you can easily +-uninstall Avocado. Start by listing the packages you have installed:: ++If you've followed the manual installation instructions above, you can ++easily uninstall Avocado. Start by listing the packages you have ++installed:: + + pip list --user + + And remove any package you want with:: + + pip uninstall ++ ++If you've used ``make check-acceptance``, the Python virtual environment where ++Avocado is installed will be cleaned up as part of ``make check-clean``. +diff --git a/tests/Makefile.include b/tests/Makefile.include +index 99a9dcd..1177ca3 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -10,6 +10,7 @@ check-help: + @echo " $(MAKE) check-speed Run qobject speed tests" + @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" + @echo " $(MAKE) check-block Run block tests" ++ @echo " $(MAKE) check-acceptance Run all acceptance (functional) tests" + @echo " $(MAKE) check-report.html Generates an HTML test report" + @echo " $(MAKE) check-venv Creates a Python venv for tests" + @echo " $(MAKE) check-clean Clean the tests" +@@ -956,10 +957,15 @@ check-decodetree: + + # Python venv for running tests + +-.PHONY: check-venv ++.PHONY: check-venv check-acceptance + + TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv + TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt ++TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results ++# Controls the output generated by Avocado when running tests. ++# Any number of command separated loggers are accepted. For more ++# information please refer to "avocado --help". ++AVOCADO_SHOW=none + + $(shell $(PYTHON) -c 'import sys; assert sys.version_info >= (3,0)' >/dev/null 2>&1) + ifeq ($(.SHELLSTATUS),0) +@@ -976,8 +982,19 @@ $(TESTS_VENV_DIR): + $(error "venv directory for tests requires Python 3") + endif + ++$(TESTS_RESULTS_DIR): ++ $(call quiet-command, mkdir -p $@, \ ++ MKDIR, $@) ++ + check-venv: $(TESTS_VENV_DIR) + ++check-acceptance: check-venv $(TESTS_RESULTS_DIR) ++ $(call quiet-command, \ ++ $(TESTS_VENV_DIR)/bin/python -m avocado \ ++ --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ ++ --failfast=on $(SRC_PATH)/tests/acceptance, \ ++ "AVOCADO", "tests/acceptance") ++ + # Consolidated targets + + .PHONY: check-qapi-schema check-qtest check-unit check check-clean +@@ -992,7 +1009,7 @@ check-clean: + rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) + rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) + rm -f tests/test-qapi-gen-timestamp +- rm -rf $(TESTS_VENV_DIR) ++ rm -rf $(TESTS_VENV_DIR) $(TESTS_RESULTS_DIR) + + clean: check-clean + +diff --git a/tests/requirements.txt b/tests/requirements.txt +index d39f9d1..64c6e27 100644 +--- a/tests/requirements.txt ++++ b/tests/requirements.txt +@@ -1,3 +1,4 @@ + # Add Python module requirements, one per line, to be installed + # in the tests/venv Python virtual environment. For more info, + # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 ++avocado-framework==65.0 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Acceptance-tests-add-quick-VNC-tests.patch b/SOURCES/kvm-Acceptance-tests-add-quick-VNC-tests.patch new file mode 100644 index 0000000..eaf020e --- /dev/null +++ b/SOURCES/kvm-Acceptance-tests-add-quick-VNC-tests.patch @@ -0,0 +1,104 @@ +From f737591acbcb84db4da620b94970bc3ac4e3b655 Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:37 +0000 +Subject: [PATCH 09/13] Acceptance tests: add quick VNC tests +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: <77eca16322a6a90444210ee11d64875df2746029.1544573601.git.ymankad@redhat.com> +Patchwork-id: 83434 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 3/7] Acceptance tests: add quick VNC tests +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +This patch adds a few simple behavior tests for VNC. + +Signed-off-by: Cleber Rosa +Reviewed-by: Stefan Hajnoczi +Message-Id: <20180530184156.15634-4-crosa@redhat.com> +Reviewed-by: Philippe Mathieu-Daudé +Tested-by: Philippe Mathieu-Daudé +Signed-off-by: Eduardo Habkost +(cherry picked from commit 7b1bd11cff0915a1266c34bdfb66d70f6372340d) +Signed-off-by: Yash Mankad +Signed-off-by: Danilo C. L. de Paula +--- + tests/acceptance/vnc.py | 60 +++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 60 insertions(+) + create mode 100644 tests/acceptance/vnc.py + +diff --git a/tests/acceptance/vnc.py b/tests/acceptance/vnc.py +new file mode 100644 +index 0000000..b1ef9d7 +--- /dev/null ++++ b/tests/acceptance/vnc.py +@@ -0,0 +1,60 @@ ++# Simple functional tests for VNC functionality ++# ++# Copyright (c) 2018 Red Hat, Inc. ++# ++# Author: ++# Cleber Rosa ++# ++# This work is licensed under the terms of the GNU GPL, version 2 or ++# later. See the COPYING file in the top-level directory. ++ ++from avocado_qemu import Test ++ ++ ++class Vnc(Test): ++ """ ++ :avocado: enable ++ :avocado: tags=vnc,quick ++ """ ++ def test_no_vnc(self): ++ self.vm.add_args('-nodefaults', '-S') ++ self.vm.launch() ++ self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled']) ++ ++ def test_no_vnc_change_password(self): ++ self.vm.add_args('-nodefaults', '-S') ++ self.vm.launch() ++ self.assertFalse(self.vm.qmp('query-vnc')['return']['enabled']) ++ set_password_response = self.vm.qmp('change', ++ device='vnc', ++ target='password', ++ arg='new_password') ++ self.assertIn('error', set_password_response) ++ self.assertEqual(set_password_response['error']['class'], ++ 'GenericError') ++ self.assertEqual(set_password_response['error']['desc'], ++ 'Could not set password') ++ ++ def test_vnc_change_password_requires_a_password(self): ++ self.vm.add_args('-nodefaults', '-S', '-vnc', ':0') ++ self.vm.launch() ++ self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled']) ++ set_password_response = self.vm.qmp('change', ++ device='vnc', ++ target='password', ++ arg='new_password') ++ self.assertIn('error', set_password_response) ++ self.assertEqual(set_password_response['error']['class'], ++ 'GenericError') ++ self.assertEqual(set_password_response['error']['desc'], ++ 'Could not set password') ++ ++ def test_vnc_change_password(self): ++ self.vm.add_args('-nodefaults', '-S', '-vnc', ':0,password') ++ self.vm.launch() ++ self.assertTrue(self.vm.qmp('query-vnc')['return']['enabled']) ++ set_password_response = self.vm.qmp('change', ++ device='vnc', ++ target='password', ++ arg='new_password') ++ self.assertEqual(set_password_response['return'], {}) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Add-functional-acceptance-tests-infrastructure.patch b/SOURCES/kvm-Add-functional-acceptance-tests-infrastructure.patch new file mode 100644 index 0000000..08023c9 --- /dev/null +++ b/SOURCES/kvm-Add-functional-acceptance-tests-infrastructure.patch @@ -0,0 +1,359 @@ +From 687a123ab2165fa3adf9e3469577c22008125270 Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:35 +0000 +Subject: [PATCH 07/13] Add functional/acceptance tests infrastructure +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: +Patchwork-id: 83432 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 1/7] Add functional/acceptance tests infrastructure +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +This patch adds the very minimum infrastructure necessary for writing +and running functional/acceptance tests, including: + + * Documentation + * The avocado_qemu.Test base test class + * One example tests (version.py) + +Additional functionality is expected to be added along the tests that +require them. + +Signed-off-by: Cleber Rosa +Message-Id: <20180530184156.15634-2-crosa@redhat.com> +Reviewed-by: Philippe Mathieu-Daudé +Tested-by: Philippe Mathieu-Daudé +[ehabkost: fix typo on testing.rst] +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Eduardo Habkost +(cherry picked from commit c3d7e8c90db208b1d876f8d6458c2dfca169137f) +Signed-off-by: Yash Mankad + +Signed-off-by: Danilo C. L. de Paula +--- + docs/devel/testing.rst | 192 ++++++++++++++++++++++++++++++ + tests/acceptance/README.rst | 10 ++ + tests/acceptance/avocado_qemu/__init__.py | 54 +++++++++ + tests/acceptance/version.py | 24 ++++ + 4 files changed, 280 insertions(+) + create mode 100644 tests/acceptance/README.rst + create mode 100644 tests/acceptance/avocado_qemu/__init__.py + create mode 100644 tests/acceptance/version.py + +diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst +index 0ca1a2d..f33e5a8 100644 +--- a/docs/devel/testing.rst ++++ b/docs/devel/testing.rst +@@ -484,3 +484,195 @@ supported. To start the fuzzer, run + + Alternatively, some command different from "qemu-img info" can be tested, by + changing the ``-c`` option. ++ ++Acceptance tests using the Avocado Framework ++============================================ ++ ++The ``tests/acceptance`` directory hosts functional tests, also known ++as acceptance level tests. They're usually higher level tests, and ++may interact with external resources and with various guest operating ++systems. ++ ++These tests are written using the Avocado Testing Framework (which must ++be installed separately) in conjunction with a the ``avocado_qemu.Test`` ++class, implemented at ``tests/acceptance/avocado_qemu``. ++ ++Tests based on ``avocado_qemu.Test`` can easily: ++ ++ * Customize the command line arguments given to the convenience ++ ``self.vm`` attribute (a QEMUMachine instance) ++ ++ * Interact with the QEMU monitor, send QMP commands and check ++ their results ++ ++ * Interact with the guest OS, using the convenience console device ++ (which may be useful to assert the effectiveness and correctness of ++ command line arguments or QMP commands) ++ ++ * Interact with external data files that accompany the test itself ++ (see ``self.get_data()``) ++ ++ * Download (and cache) remote data files, such as firmware and kernel ++ images ++ ++ * Have access to a library of guest OS images (by means of the ++ ``avocado.utils.vmimage`` library) ++ ++ * Make use of various other test related utilities available at the ++ test class itself and at the utility library: ++ ++ - http://avocado-framework.readthedocs.io/en/latest/api/test/avocado.html#avocado.Test ++ - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html ++ ++Installation ++------------ ++ ++To install Avocado and its dependencies, run: ++ ++.. code:: ++ ++ pip install --user avocado-framework ++ ++Alternatively, follow the instructions on this link: ++ ++ http://avocado-framework.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado ++ ++Overview ++-------- ++ ++This directory provides the ``avocado_qemu`` Python module, containing ++the ``avocado_qemu.Test`` class. Here's a simple usage example: ++ ++.. code:: ++ ++ from avocado_qemu import Test ++ ++ ++ class Version(Test): ++ """ ++ :avocado: enable ++ :avocado: tags=quick ++ """ ++ def test_qmp_human_info_version(self): ++ self.vm.launch() ++ res = self.vm.command('human-monitor-command', ++ command_line='info version') ++ self.assertRegexpMatches(res, r'^(\d+\.\d+\.\d)') ++ ++To execute your test, run: ++ ++.. code:: ++ ++ avocado run version.py ++ ++Tests may be classified according to a convention by using docstring ++directives such as ``:avocado: tags=TAG1,TAG2``. To run all tests ++in the current directory, tagged as "quick", run: ++ ++.. code:: ++ ++ avocado run -t quick . ++ ++The ``avocado_qemu.Test`` base test class ++----------------------------------------- ++ ++The ``avocado_qemu.Test`` class has a number of characteristics that ++are worth being mentioned right away. ++ ++First of all, it attempts to give each test a ready to use QEMUMachine ++instance, available at ``self.vm``. Because many tests will tweak the ++QEMU command line, launching the QEMUMachine (by using ``self.vm.launch()``) ++is left to the test writer. ++ ++At test "tear down", ``avocado_qemu.Test`` handles the QEMUMachine ++shutdown. ++ ++QEMUMachine ++~~~~~~~~~~~ ++ ++The QEMUMachine API is already widely used in the Python iotests, ++device-crash-test and other Python scripts. It's a wrapper around the ++execution of a QEMU binary, giving its users: ++ ++ * the ability to set command line arguments to be given to the QEMU ++ binary ++ ++ * a ready to use QMP connection and interface, which can be used to ++ send commands and inspect its results, as well as asynchronous ++ events ++ ++ * convenience methods to set commonly used command line arguments in ++ a more succinct and intuitive way ++ ++QEMU binary selection ++~~~~~~~~~~~~~~~~~~~~~ ++ ++The QEMU binary used for the ``self.vm`` QEMUMachine instance will ++primarily depend on the value of the ``qemu_bin`` parameter. If it's ++not explicitly set, its default value will be the result of a dynamic ++probe in the same source tree. A suitable binary will be one that ++targets the architecture matching host machine. ++ ++Based on this description, test writers will usually rely on one of ++the following approaches: ++ ++1) Set ``qemu_bin``, and use the given binary ++ ++2) Do not set ``qemu_bin``, and use a QEMU binary named like ++ "${arch}-softmmu/qemu-system-${arch}", either in the current ++ working directory, or in the current source tree. ++ ++The resulting ``qemu_bin`` value will be preserved in the ++``avocado_qemu.Test`` as an attribute with the same name. ++ ++Attribute reference ++------------------- ++ ++Besides the attributes and methods that are part of the base ++``avocado.Test`` class, the following attributes are available on any ++``avocado_qemu.Test`` instance. ++ ++vm ++~~ ++ ++A QEMUMachine instance, initially configured according to the given ++``qemu_bin`` parameter. ++ ++qemu_bin ++~~~~~~~~ ++ ++The preserved value of the ``qemu_bin`` parameter or the result of the ++dynamic probe for a QEMU binary in the current working directory or ++source tree. ++ ++Parameter reference ++------------------- ++ ++To understand how Avocado parameters are accessed by tests, and how ++they can be passed to tests, please refer to:: ++ ++ http://avocado-framework.readthedocs.io/en/latest/WritingTests.html#accessing-test-parameters ++ ++Parameter values can be easily seen in the log files, and will look ++like the following: ++ ++.. code:: ++ ++ PARAMS (key=qemu_bin, path=*, default=x86_64-softmmu/qemu-system-x86_64) => 'x86_64-softmmu/qemu-system-x86_64 ++ ++qemu_bin ++~~~~~~~~ ++ ++The exact QEMU binary to be used on QEMUMachine. ++ ++Uninstalling Avocado ++-------------------- ++ ++If you've followed the installation instructions above, you can easily ++uninstall Avocado. Start by listing the packages you have installed:: ++ ++ pip list --user ++ ++And remove any package you want with:: ++ ++ pip uninstall +diff --git a/tests/acceptance/README.rst b/tests/acceptance/README.rst +new file mode 100644 +index 0000000..89260fa +--- /dev/null ++++ b/tests/acceptance/README.rst +@@ -0,0 +1,10 @@ ++============================================ ++Acceptance tests using the Avocado Framework ++============================================ ++ ++This directory contains functional tests, also known as acceptance ++level tests. They're usually higher level, and may interact with ++external resources and with various guest operating systems. ++ ++For more information, please refer to ``docs/devel/testing.rst``, ++section "Acceptance tests using the Avocado Framework". +diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py +new file mode 100644 +index 0000000..1e54fd5 +--- /dev/null ++++ b/tests/acceptance/avocado_qemu/__init__.py +@@ -0,0 +1,54 @@ ++# Test class and utilities for functional tests ++# ++# Copyright (c) 2018 Red Hat, Inc. ++# ++# Author: ++# Cleber Rosa ++# ++# This work is licensed under the terms of the GNU GPL, version 2 or ++# later. See the COPYING file in the top-level directory. ++ ++import os ++import sys ++ ++import avocado ++ ++SRC_ROOT_DIR = os.path.dirname(os.path.dirname(os.path.dirname(__file__))) ++SRC_ROOT_DIR = os.path.abspath(os.path.dirname(SRC_ROOT_DIR)) ++sys.path.append(os.path.join(SRC_ROOT_DIR, 'scripts')) ++ ++from qemu import QEMUMachine ++ ++def is_readable_executable_file(path): ++ return os.path.isfile(path) and os.access(path, os.R_OK | os.X_OK) ++ ++ ++def pick_default_qemu_bin(): ++ """ ++ Picks the path of a QEMU binary, starting either in the current working ++ directory or in the source tree root directory. ++ """ ++ arch = os.uname()[4] ++ qemu_bin_relative_path = os.path.join("%s-softmmu" % arch, ++ "qemu-system-%s" % arch) ++ if is_readable_executable_file(qemu_bin_relative_path): ++ return qemu_bin_relative_path ++ ++ qemu_bin_from_src_dir_path = os.path.join(SRC_ROOT_DIR, ++ qemu_bin_relative_path) ++ if is_readable_executable_file(qemu_bin_from_src_dir_path): ++ return qemu_bin_from_src_dir_path ++ ++ ++class Test(avocado.Test): ++ def setUp(self): ++ self.vm = None ++ self.qemu_bin = self.params.get('qemu_bin', ++ default=pick_default_qemu_bin()) ++ if self.qemu_bin is None: ++ self.cancel("No QEMU binary defined or found in the source tree") ++ self.vm = QEMUMachine(self.qemu_bin) ++ ++ def tearDown(self): ++ if self.vm is not None: ++ self.vm.shutdown() +diff --git a/tests/acceptance/version.py b/tests/acceptance/version.py +new file mode 100644 +index 0000000..13b0a74 +--- /dev/null ++++ b/tests/acceptance/version.py +@@ -0,0 +1,24 @@ ++# Version check example test ++# ++# Copyright (c) 2018 Red Hat, Inc. ++# ++# Author: ++# Cleber Rosa ++# ++# This work is licensed under the terms of the GNU GPL, version 2 or ++# later. See the COPYING file in the top-level directory. ++ ++ ++from avocado_qemu import Test ++ ++ ++class Version(Test): ++ """ ++ :avocado: enable ++ :avocado: tags=quick ++ """ ++ def test_qmp_human_info_version(self): ++ self.vm.launch() ++ res = self.vm.command('human-monitor-command', ++ command_line='info version') ++ self.assertRegexpMatches(res, r'^(\d+\.\d+\.\d)') +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Bootstrap-Python-venv-for-tests.patch b/SOURCES/kvm-Bootstrap-Python-venv-for-tests.patch new file mode 100644 index 0000000..968b30d --- /dev/null +++ b/SOURCES/kvm-Bootstrap-Python-venv-for-tests.patch @@ -0,0 +1,108 @@ +From 5c0a6bb69135e0fa83a1e063dfe878e5e98c1785 Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:40 +0000 +Subject: [PATCH 12/13] Bootstrap Python venv for tests +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: <8e00545539681a5de548c444e7752894b12bc8ec.1544573601.git.ymankad@redhat.com> +Patchwork-id: 83436 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 6/7] Bootstrap Python venv for tests +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +A number of QEMU tests are written in Python, and may benefit +from an untainted Python venv. + +By using make rules, tests that depend on specific Python libs +can set that rule as a requirement, along with rules that require +the presence or installation of specific libraries. + +The tests/requirements.txt is supposed to contain the Python +requirements that should be added to the venv created by check-venv. + +Signed-off-by: Cleber Rosa +Tested-by: Philippe Mathieu-Daudé +Acked-by: Stefan Hajnoczi +Acked-by: Wainer dos Santos Moschetta +Reviewed-by: Caio Carrara +Message-Id: <20181018153134.8493-2-crosa@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit 213137217a60eca18e9b55817f00dfdd6eaff74a) +Signed-off-by: Yash Mankad +Signed-off-by: Danilo C. L. de Paula +--- + tests/Makefile.include | 26 ++++++++++++++++++++++++++ + tests/requirements.txt | 3 +++ + 2 files changed, 29 insertions(+) + create mode 100644 tests/requirements.txt + +diff --git a/tests/Makefile.include b/tests/Makefile.include +index 3ed8531..99a9dcd 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -11,6 +11,7 @@ check-help: + @echo " $(MAKE) check-qapi-schema Run QAPI schema tests" + @echo " $(MAKE) check-block Run block tests" + @echo " $(MAKE) check-report.html Generates an HTML test report" ++ @echo " $(MAKE) check-venv Creates a Python venv for tests" + @echo " $(MAKE) check-clean Clean the tests" + @echo + @echo "Please note that HTML reports do not regenerate if the unit tests" +@@ -953,6 +954,30 @@ check-decodetree: + ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \ + TEST, decodetree.py) + ++# Python venv for running tests ++ ++.PHONY: check-venv ++ ++TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv ++TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt ++ ++$(shell $(PYTHON) -c 'import sys; assert sys.version_info >= (3,0)' >/dev/null 2>&1) ++ifeq ($(.SHELLSTATUS),0) ++$(TESTS_VENV_DIR): $(TESTS_VENV_REQ) ++ $(call quiet-command, \ ++ $(PYTHON) -m venv --system-site-packages $@, \ ++ VENV, $@) ++ $(call quiet-command, \ ++ $(TESTS_VENV_DIR)/bin/python -m pip -q install -r $(TESTS_VENV_REQ), \ ++ PIP, $(TESTS_VENV_REQ)) ++ $(call quiet-command, touch $@) ++else ++$(TESTS_VENV_DIR): ++ $(error "venv directory for tests requires Python 3") ++endif ++ ++check-venv: $(TESTS_VENV_DIR) ++ + # Consolidated targets + + .PHONY: check-qapi-schema check-qtest check-unit check check-clean +@@ -967,6 +992,7 @@ check-clean: + rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) + rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) + rm -f tests/test-qapi-gen-timestamp ++ rm -rf $(TESTS_VENV_DIR) + + clean: check-clean + +diff --git a/tests/requirements.txt b/tests/requirements.txt +new file mode 100644 +index 0000000..d39f9d1 +--- /dev/null ++++ b/tests/requirements.txt +@@ -0,0 +1,3 @@ ++# Add Python module requirements, one per line, to be installed ++# in the tests/venv Python virtual environment. For more info, ++# refer to: https://pip.pypa.io/en/stable/user_guide/#id1 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Declare-cirrus-vga-as-deprecated.patch b/SOURCES/kvm-Declare-cirrus-vga-as-deprecated.patch new file mode 100644 index 0000000..3195628 --- /dev/null +++ b/SOURCES/kvm-Declare-cirrus-vga-as-deprecated.patch @@ -0,0 +1,50 @@ +From 4b889f33761a4447998b16846bfb983519def96d Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Wed, 28 Nov 2018 07:23:59 +0000 +Subject: [PATCH 07/16] Declare cirrus-vga as deprecated + +RH-Author: Thomas Huth +Message-id: <1543389839-8995-1-git-send-email-thuth@redhat.com> +Patchwork-id: 83178 +O-Subject: [RHEL8 qemu-kvm PATCH v2] Declare cirrus-vga as deprecated +Bugzilla: 1651994 +RH-Acked-by: Gerd Hoffmann +RH-Acked-by: Markus Armbruster +RH-Acked-by: Laszlo Ersek + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1651994 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=19300792 +Upstream: n/a (downstream only) +Branch: rhel8/master-2.12.0 +Branch: rhel8/master-3.0.0 + +The "cirrus" device in qemu-kvm is considered as a legacy device, which +also had a lot of security issues in the past. KVM guest should preferably +use "stdvga", "virtio-vga" or "qxl" as graphics card nowadays instead. +To avoid that we have to carry along the legacy "cirrus" device in +downstream qemu-kvm forever, let's mark it as deprecated in RHEL8 now, +so that we can finally remove it in RHEL9. + +Signed-off-by: Thomas Huth +Signed-off-by: Danilo C. L. de Paula +--- + hw/display/cirrus_vga.c | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c +index 014268a..29d6055 100644 +--- a/hw/display/cirrus_vga.c ++++ b/hw/display/cirrus_vga.c +@@ -3098,6 +3098,9 @@ static void pci_cirrus_vga_realize(PCIDevice *dev, Error **errp) + PCIDeviceClass *pc = PCI_DEVICE_GET_CLASS(dev); + int16_t device_id = pc->device_id; + ++ warn_report("'cirrus-vga' is deprecated, " ++ "please use a different VGA card instead"); ++ + /* follow real hardware, cirrus card emulated has 4 MB video memory. + Also accept 8 MB/16 MB for backward compatibility. */ + if (s->vga.vram_size_mb != 4 && s->vga.vram_size_mb != 8 && +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-AT24Cx-i2c-eeprom.patch b/SOURCES/kvm-Disable-AT24Cx-i2c-eeprom.patch new file mode 100644 index 0000000..ca6cec7 --- /dev/null +++ b/SOURCES/kvm-Disable-AT24Cx-i2c-eeprom.patch @@ -0,0 +1,40 @@ +From d68f80c725a05ddf0a2a997ba35832e8f32c7fe9 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 7 Jun 2018 07:43:11 +0200 +Subject: [PATCH 259/268] Disable AT24Cx i2c eeprom + +RH-Author: Miroslav Rezanina +Message-id: +Patchwork-id: 80598 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 3/7] Disable AT24Cx i2c eeprom +Bugzilla: 1586357 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Markus Armbruster + +From: Miroslav Rezanina + +We do not want to support new eeprom_at24c device. It is protected +by CONFIG_I2C option but we can't remove it as it cover other supported +devices. Manually remove module with the device. + +Signed-off-by: Miroslav Rezanina +--- + hw/nvram/Makefile.objs | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/nvram/Makefile.objs b/hw/nvram/Makefile.objs +index a912d25..cbc8bba 100644 +--- a/hw/nvram/Makefile.objs ++++ b/hw/nvram/Makefile.objs +@@ -1,6 +1,6 @@ + common-obj-$(CONFIG_DS1225Y) += ds1225y.o + common-obj-y += eeprom93xx.o +-common-obj-$(CONFIG_I2C) += eeprom_at24c.o ++#common-obj-$(CONFIG_I2C) += eeprom_at24c.o + common-obj-y += fw_cfg.o + common-obj-y += chrp_nvram.o + common-obj-$(CONFIG_MAC_NVRAM) += mac_nvram.o +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-CAN-bus-devices.patch b/SOURCES/kvm-Disable-CAN-bus-devices.patch new file mode 100644 index 0000000..8492cc8 --- /dev/null +++ b/SOURCES/kvm-Disable-CAN-bus-devices.patch @@ -0,0 +1,40 @@ +From 3f953e91fb0d690f33dd6e0d4b257cbb97c410cc Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 7 Jun 2018 07:43:12 +0200 +Subject: [PATCH 260/268] Disable CAN bus devices + +RH-Author: Miroslav Rezanina +Message-id: <706a21b4d25946165513ff99619c0fed0852dc50.1528355911.git.mrezanin@redhat.com> +Patchwork-id: 80593 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 4/7] Disable CAN bus devices +Bugzilla: 1586357 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Markus Armbruster + +From: Miroslav Rezanina + +We do not want to support new CAN devices (kvaser_pci, mioe3680_pci +and pcm3680_pci). + +Signed-off-by: Miroslav Rezanina +--- + default-configs/pci.mak | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/default-configs/pci.mak b/default-configs/pci.mak +index 4c8c296..25fc382 100644 +--- a/default-configs/pci.mak ++++ b/default-configs/pci.mak +@@ -33,7 +33,7 @@ CONFIG_SERIAL_ISA=y + CONFIG_SERIAL_PCI=y + CONFIG_CAN_BUS=y + CONFIG_CAN_SJA1000=y +-CONFIG_CAN_PCI=y ++#CONFIG_CAN_PCI=y + #CONFIG_IPACK=y + CONFIG_WDT_IB6300ESB=y + CONFIG_PCI_TESTDEV=y +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-CONFIG_CAN_BUS-and-CONFIG_CAN_SJA1000.patch b/SOURCES/kvm-Disable-CONFIG_CAN_BUS-and-CONFIG_CAN_SJA1000.patch new file mode 100644 index 0000000..f21f30b --- /dev/null +++ b/SOURCES/kvm-Disable-CONFIG_CAN_BUS-and-CONFIG_CAN_SJA1000.patch @@ -0,0 +1,55 @@ +From 9caf292924a26cbfb7e46d232a0f32f706254866 Mon Sep 17 00:00:00 2001 +From: Danilo de Paula +Date: Fri, 7 Dec 2018 14:16:59 +0000 +Subject: [PATCH 2/2] Disable CONFIG_CAN_BUS and CONFIG_CAN_SJA1000 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Danilo de Paula +Message-id: <20181207141659.3485-1-ddepaula@redhat.com> +Patchwork-id: 83304 +O-Subject: [RHEL-8.0/rhel qemu-kvm PATCH] Disable CONFIG_CAN_BUS and CONFIG_CAN_SJA1000 +Bugzilla: 1640042 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Stefan Hajnoczi + +BZ: 1640042 +Branch: rhel8/master-2.12.0 +Branch: rhel8/master-3.1.0-rc2 +Upstream: Downstream only +Brew: None + +We currently compile qemu-kvm with CONFIG_CAN_SJA1000=y and +CONFIG_CAN_BUS=y in default-configs/pci.mak. +As far as I can see, this has currently no effect, +since we already disabled the other CONFIG_CAN_* switches there +(which control the inclusion of the emulated CAN devices). + +Since we do not have any plans to support CAN in downstream, we should also disable +CONFIG_CAN_BUS and CONFIG_CAN_SJA1000 in our downstream configuration. + +Signed-off-by: Danilo C. L. de Paula +--- + default-configs/pci.mak | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/default-configs/pci.mak b/default-configs/pci.mak +index 763c375..2277308 100644 +--- a/default-configs/pci.mak ++++ b/default-configs/pci.mak +@@ -31,8 +31,8 @@ CONFIG_AHCI=y + CONFIG_SERIAL=y + CONFIG_SERIAL_ISA=y + CONFIG_SERIAL_PCI=y +-CONFIG_CAN_BUS=y +-CONFIG_CAN_SJA1000=y ++#CONFIG_CAN_BUS=y ++#CONFIG_CAN_SJA1000=y + #CONFIG_CAN_PCI=y + #CONFIG_IPACK=y + CONFIG_WDT_IB6300ESB=y +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-CONFIG_IPMI-and-CONFIG_I2C-for-ppc64.patch b/SOURCES/kvm-Disable-CONFIG_IPMI-and-CONFIG_I2C-for-ppc64.patch new file mode 100644 index 0000000..1cb33e8 --- /dev/null +++ b/SOURCES/kvm-Disable-CONFIG_IPMI-and-CONFIG_I2C-for-ppc64.patch @@ -0,0 +1,50 @@ +From 3eef52a0caff23b537e88009d79ec1725ccebe06 Mon Sep 17 00:00:00 2001 +From: Danilo de Paula +Date: Fri, 7 Dec 2018 14:12:00 +0000 +Subject: [PATCH 1/2] Disable CONFIG_IPMI and CONFIG_I2C for ppc64 + +RH-Author: Danilo de Paula +Message-id: <20181207141200.30857-1-ddepaula@redhat.com> +Patchwork-id: 83303 +O-Subject: [RHEL-8.0/rhel qemu-kvm PATCH] Disable CONFIG_IPMI and CONFIG_I2C for ppc64 +Bugzilla: 1640044 +RH-Acked-by: Thomas Huth +RH-Acked-by: David Gibson +RH-Acked-by: Eduardo Habkost + +BZ: 1640044 +Branch: rhel8/master-2.12.0 +Branch: rhel8/master-3.1.0-rc2 +Upstream: Downstream only +Brew: none + +Our downstream qemu-kvm only uses the para-virtualized "pseries" machine +type. +There is no need to include I2C or IPMI. + +Signed-off-by: Danilo C. L. de Paula +--- + default-configs/ppc64-softmmu.mak | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/default-configs/ppc64-softmmu.mak b/default-configs/ppc64-softmmu.mak +index 0ee8f6c..cece3dc 100644 +--- a/default-configs/ppc64-softmmu.mak ++++ b/default-configs/ppc64-softmmu.mak +@@ -15,11 +15,11 @@ CONFIG_USB_OHCI=y + CONFIG_VGA=y + CONFIG_VGA_PCI=y + CONFIG_SERIAL=y +-CONFIG_I2C=y ++#CONFIG_I2C=y + + # For PowerNV + #CONFIG_POWERNV=y +-CONFIG_IPMI=y ++#CONFIG_IPMI=y + #CONFIG_IPMI_LOCAL=y + #CONFIG_IPMI_EXTERN=y + #CONFIG_ISA_IPMI_BT=y +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-aarch64-devices-reappeared-after-2.12-rebase.patch b/SOURCES/kvm-Disable-aarch64-devices-reappeared-after-2.12-rebase.patch new file mode 100644 index 0000000..133fad2 --- /dev/null +++ b/SOURCES/kvm-Disable-aarch64-devices-reappeared-after-2.12-rebase.patch @@ -0,0 +1,43 @@ +From c670fa1f55724d096e28c9ab929ff4cb7d935d31 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 7 Jun 2018 07:43:09 +0200 +Subject: [PATCH 257/268] Disable aarch64 devices reappeared after 2.12 rebase + +RH-Author: Miroslav Rezanina +Message-id: +Patchwork-id: 80592 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/7] Disable aarch64 devices reappeared after 2.12 rebase +Bugzilla: 1586357 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Laurent Vivier + +From: Miroslav Rezanina + +Up to QEMU 2.11 we disabled aarch64 devices by removing files from +Makefile. +As default_config options were added, we use them to disable devices +since 2.12. However, we can't turn CONFIG_ARM_V7M (turning it off will +break build) so we have to manually remove armv7m module from Makefile. + +Signed-off-by: Miroslav Rezanina +--- + hw/arm/Makefile.objs | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/arm/Makefile.objs b/hw/arm/Makefile.objs +index 2885e3e..3e8b167 100644 +--- a/hw/arm/Makefile.objs ++++ b/hw/arm/Makefile.objs +@@ -16,7 +16,7 @@ obj-$(CONFIG_STRONGARM) += collie.o + obj-$(CONFIG_VERSATILE) += vexpress.o versatilepb.o + obj-$(CONFIG_ZYNQ) += xilinx_zynq.o + +-obj-$(CONFIG_ARM_V7M) += armv7m.o ++#obj-$(CONFIG_ARM_V7M) += armv7m.o + obj-$(CONFIG_EXYNOS4) += exynos4210.o + obj-$(CONFIG_PXA2XX) += pxa2xx.o pxa2xx_gpio.o pxa2xx_pic.o + obj-$(CONFIG_DIGIC) += digic.o +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-ivshmem.patch b/SOURCES/kvm-Disable-ivshmem.patch new file mode 100644 index 0000000..2a5186a --- /dev/null +++ b/SOURCES/kvm-Disable-ivshmem.patch @@ -0,0 +1,36 @@ +From 67c5a8ce8ef97d9b08cfcbe70e05da9ca91dd62e Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Wed, 10 Oct 2018 04:58:19 +0100 +Subject: [PATCH 2/5] Disable ivshmem + +RH-Author: Markus Armbruster +Message-id: <20181010045819.32729-3-armbru@redhat.com> +Patchwork-id: 82526 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/2] Disable ivshmem +Bugzilla: 1621817 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi + +Signed-off-by: Markus Armbruster +Signed-off-by: Danilo C. L. de Paula +--- + default-configs/pci.mak | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/default-configs/pci.mak b/default-configs/pci.mak +index 25fc382..763c375 100644 +--- a/default-configs/pci.mak ++++ b/default-configs/pci.mak +@@ -43,7 +43,7 @@ CONFIG_PCI_TESTDEV=y + CONFIG_EDU=y + CONFIG_VGA=y + CONFIG_VGA_PCI=y +-CONFIG_IVSHMEM_DEVICE=$(CONFIG_IVSHMEM) ++#CONFIG_IVSHMEM_DEVICE=$(CONFIG_IVSHMEM) + #CONFIG_ROCKER=y + #CONFIG_VHOST_USER_SCSI=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) + #CONFIG_VHOST_USER_BLK=$(call land,$(CONFIG_VHOST_USER),$(CONFIG_LINUX)) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-new-pvrdma-device.patch b/SOURCES/kvm-Disable-new-pvrdma-device.patch new file mode 100644 index 0000000..cd75628 --- /dev/null +++ b/SOURCES/kvm-Disable-new-pvrdma-device.patch @@ -0,0 +1,39 @@ +From 747643ced0f1950360a2af103bb2490849aa3abf Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 7 Jun 2018 07:43:14 +0200 +Subject: [PATCH 262/268] Disable new pvrdma device + +RH-Author: Miroslav Rezanina +Message-id: <5d290e25879e3ac42c247ff3c3a524001ceb0be1.1528355911.git.mrezanin@redhat.com> +Patchwork-id: 80597 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 6/7] Disable new pvrdma device +Bugzilla: 1586357 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Markus Armbruster + +From: Miroslav Rezanina + +New pvrdma device was introduced in rebase to QEMU 2.12. We do not want to +support this device. + +Signed-off-by: Miroslav Rezanina +--- + hw/rdma/Makefile.objs | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/hw/rdma/Makefile.objs b/hw/rdma/Makefile.objs +index 3504c39..02ca2a9 100644 +--- a/hw/rdma/Makefile.objs ++++ b/hw/rdma/Makefile.objs +@@ -1,5 +1,6 @@ + ifeq ($(CONFIG_RDMA),y) + obj-$(CONFIG_PCI) += rdma_utils.o rdma_backend.o rdma_rm.o + obj-$(CONFIG_PCI) += vmw/pvrdma_dev_ring.o vmw/pvrdma_cmd.o \ +- vmw/pvrdma_qp_ops.o vmw/pvrdma_main.o ++ vmw/pvrdma_qp_ops.o ++#obj-$(CONFIG_PCI) += vmw/pvrdma_main.o + endif +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-new-superio-devices.patch b/SOURCES/kvm-Disable-new-superio-devices.patch new file mode 100644 index 0000000..e5cbf32 --- /dev/null +++ b/SOURCES/kvm-Disable-new-superio-devices.patch @@ -0,0 +1,38 @@ +From 284c3931c86e875e86c41a838a786d4732f95b5b Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 7 Jun 2018 07:43:13 +0200 +Subject: [PATCH 261/268] Disable new superio devices + +RH-Author: Miroslav Rezanina +Message-id: <51483d3f0091abc80d52485ab076581d4ca914c5.1528355911.git.mrezanin@redhat.com> +Patchwork-id: 80595 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 5/7] Disable new superio devices +Bugzilla: 1586357 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Markus Armbruster + +From: Miroslav Rezanina + +We do not want to support new superio devices (fdc37m81x-superio and +smc37c669-superio). + +Signed-off-by: Miroslav Rezanina +--- + hw/isa/Makefile.objs | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/isa/Makefile.objs b/hw/isa/Makefile.objs +index 83e06f6..7de4f44 100644 +--- a/hw/isa/Makefile.objs ++++ b/hw/isa/Makefile.objs +@@ -1,5 +1,5 @@ + common-obj-$(CONFIG_ISA_BUS) += isa-bus.o +-common-obj-$(CONFIG_ISA_BUS) += isa-superio.o smc37c669-superio.o ++#common-obj-$(CONFIG_ISA_BUS) += isa-superio.o smc37c669-superio.o + common-obj-$(CONFIG_APM) += apm.o + common-obj-$(CONFIG_I82378) += i82378.o + common-obj-$(CONFIG_PC87312) += pc87312.o +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Disable-split-irq-device.patch b/SOURCES/kvm-Disable-split-irq-device.patch new file mode 100644 index 0000000..b7bb21a --- /dev/null +++ b/SOURCES/kvm-Disable-split-irq-device.patch @@ -0,0 +1,40 @@ +From 3ebdb9532749bbc04458f868e7c08680a236007c Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 7 Jun 2018 07:43:10 +0200 +Subject: [PATCH 258/268] Disable split-irq device + +RH-Author: Miroslav Rezanina +Message-id: <63c6336171256528761f3b4aa22c058d472b048b.1528355911.git.mrezanin@redhat.com> +Patchwork-id: 80596 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/7] Disable split-irq device +Bugzilla: 1586357 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Markus Armbruster + +From: Miroslav Rezanina + +There's new split-irq device introduced in QEMU 2.12. We do not want to +support this device. + +Signed-off-by: Miroslav Rezanina +--- + hw/core/Makefile.objs | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/core/Makefile.objs b/hw/core/Makefile.objs +index 2b4491f..e967fb2 100644 +--- a/hw/core/Makefile.objs ++++ b/hw/core/Makefile.objs +@@ -20,7 +20,7 @@ common-obj-$(CONFIG_SOFTMMU) += qdev-properties-system.o + # common-obj-$(CONFIG_SOFTMMU) += register.o + # obj-$(CONFIG_SOFTMMU) += generic-loader.o + # common-obj-$(CONFIG_SOFTMMU) += or-irq.o +-common-obj-$(CONFIG_SOFTMMU) += split-irq.o ++#common-obj-$(CONFIG_SOFTMMU) += split-irq.o + common-obj-$(CONFIG_PLATFORM_BUS) += platform-bus.o + + obj-$(CONFIG_SOFTMMU) += null-machine.o +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Do-not-build-bluetooth-support.patch b/SOURCES/kvm-Do-not-build-bluetooth-support.patch new file mode 100644 index 0000000..0c626ae --- /dev/null +++ b/SOURCES/kvm-Do-not-build-bluetooth-support.patch @@ -0,0 +1,129 @@ +From b579d3287a75db4b7ee37d25d99b2087f4ed4475 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Fri, 30 Nov 2018 13:58:43 +0000 +Subject: [PATCH 08/16] Do not build bluetooth support + +RH-Author: Miroslav Rezanina +Message-id: <1543586323-7323-1-git-send-email-mrezanin@redhat.com> +Patchwork-id: 83193 +O-Subject: [RHEL-8 qemu-kvm PATCHv2] Do not build bluetooth support +Bugzilla: 1654651 +RH-Acked-by: Thomas Huth +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Gerd Hoffmann + +From: Miroslav Rezanina + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1654651 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=19329246 +Branch: rhel8/master-2.12.0 + +We do not support bt devices however some bt related code is built in. +As this code is deprecated upstream disabling build of this code downstream. + +Signed-off-by: Miroslav Rezanina +--- +v2: + - Removed -bt option from qemu-options.hx + +Signed-off-by: Danilo C. L. de Paula +--- + Makefile.objs | 4 ++-- + hw/bt/Makefile.objs | 4 ++-- + qemu-options.hx | 2 ++ + vl.c | 7 +++++++ + 4 files changed, 13 insertions(+), 4 deletions(-) + +diff --git a/Makefile.objs b/Makefile.objs +index c6c3554..be72238 100644 +--- a/Makefile.objs ++++ b/Makefile.objs +@@ -115,8 +115,8 @@ common-obj-y += replay/ + + common-obj-y += ui/ + common-obj-m += ui/ +-common-obj-y += bt-host.o bt-vhci.o +-bt-host.o-cflags := $(BLUEZ_CFLAGS) ++#common-obj-y += bt-host.o bt-vhci.o ++#bt-host.o-cflags := $(BLUEZ_CFLAGS) + + common-obj-y += dma-helpers.o + common-obj-y += vl.o +diff --git a/hw/bt/Makefile.objs b/hw/bt/Makefile.objs +index 867a7d2..e678e9e 100644 +--- a/hw/bt/Makefile.objs ++++ b/hw/bt/Makefile.objs +@@ -1,3 +1,3 @@ +-common-obj-y += core.o l2cap.o sdp.o hci.o hid.o +-common-obj-y += hci-csr.o ++#common-obj-y += core.o l2cap.o sdp.o hci.o hid.o ++#common-obj-y += hci-csr.o + +diff --git a/qemu-options.hx b/qemu-options.hx +index 4271cd3..2e05112 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -2827,6 +2827,7 @@ STEXI + ETEXI + DEFHEADING() + ++#if 0 + DEFHEADING(Bluetooth(R) options:) + STEXI + @table @option +@@ -2901,6 +2902,7 @@ STEXI + @end table + ETEXI + DEFHEADING() ++#endif + + #ifdef CONFIG_TPM + DEFHEADING(TPM device options:) +diff --git a/vl.c b/vl.c +index f253876..74fa8f2 100644 +--- a/vl.c ++++ b/vl.c +@@ -923,6 +923,7 @@ static void configure_rtc(QemuOpts *opts) + } + } + ++#if 0 // Disabled for Red Hat Enterprise Linux + /***********************************************************/ + /* Bluetooth support */ + static int nb_hcis; +@@ -1044,6 +1045,7 @@ static int bt_parse(const char *opt) + error_report("bad bluetooth parameter '%s'", opt); + return 1; + } ++#endif + + static int parse_sandbox(void *opaque, QemuOpts *opts, Error **errp) + { +@@ -3367,9 +3369,12 @@ int main(int argc, char **argv, char **envp) + exit(1); + break; + #endif ++ ++#if 0 // Disabled for Red Hat Enterprise Linux + case QEMU_OPTION_bt: + add_device_config(DEV_BT, optarg); + break; ++#endif + case QEMU_OPTION_audio_help: + AUD_help (); + exit (0); +@@ -4523,9 +4528,11 @@ int main(int argc, char **argv, char **envp) + exit(1); + } + ++#if 0 // Disabled for Red Hat Enterprise Linux + /* init the bluetooth world */ + if (foreach_device_config(DEV_BT, bt_parse)) + exit(1); ++#endif + + if (!xen_enabled()) { + /* On 32-bit hosts, QEMU is limited by virtual address space */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Fix-annocheck-issues.patch b/SOURCES/kvm-Fix-annocheck-issues.patch new file mode 100644 index 0000000..f220df3 --- /dev/null +++ b/SOURCES/kvm-Fix-annocheck-issues.patch @@ -0,0 +1,53 @@ +From 9997a461dc882720fff3990aeca0725a91f20ac3 Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Wed, 12 Sep 2018 09:43:39 +0100 +Subject: [PATCH 2/4] Fix annocheck issues + +RH-Author: Miroslav Rezanina +Message-id: <1536745419-16795-1-git-send-email-mrezanin@redhat.com> +Patchwork-id: 82139 +O-Subject: [RHEL8/rhel qemu-kvm PATCH] Fix annocheck issues +Bugzilla: 1624164 +RH-Acked-by: Thomas Huth +RH-Acked-by: Danilo de Paula +RH-Acked-by: Laszlo Ersek + +From: Miroslav Rezanina + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1624164 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18283300 +BRANCH: rhel8/master-2.12.0 +Upstream: Do we want have this upstream? + +annocheck reports several issues with qemu-kvm packages. + +Most of them is "Compiled without -D_GLIBCXX_ASSERTIONS.". These issues +comes from capstone submodule as we strip all -W compile flags. We can +add missing flag downstream but I'm not sure this change should be done +upstream too. + +In addition, there's "Not linked with -Wl,-z,now." error for s390-ccw.img +and s390-netboot.img. However, this flag is used for building these files +as build log shows. + +Signed-off-by: Miroslav Rezanina +Signed-off-by: Danilo C. L. de Paula +--- + Makefile | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/Makefile b/Makefile +index 9803f27..da3eedb 100644 +--- a/Makefile ++++ b/Makefile +@@ -502,6 +502,7 @@ CAP_CFLAGS += -DCAPSTONE_HAS_ARM + CAP_CFLAGS += -DCAPSTONE_HAS_ARM64 + CAP_CFLAGS += -DCAPSTONE_HAS_POWERPC + CAP_CFLAGS += -DCAPSTONE_HAS_X86 ++CAP_CFLAGS += -Wp,-D_GLIBCXX_ASSERTIONS + + subdir-capstone: .git-submodule-status + $(call quiet-command,$(MAKE) -C $(SRC_PATH)/capstone CAPSTONE_SHARED=no BUILDDIR="$(BUILD_DIR)/capstone" CC="$(CC)" AR="$(AR)" LD="$(LD)" RANLIB="$(RANLIB)" CFLAGS="$(CAP_CFLAGS)" $(SUBDIR_MAKEFLAGS) $(BUILD_DIR)/capstone/$(LIBCAPSTONE)) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Fix-libusb-1.0.22-deprecated-libusb_set_debug-with-l.patch b/SOURCES/kvm-Fix-libusb-1.0.22-deprecated-libusb_set_debug-with-l.patch new file mode 100644 index 0000000..fc5d11b --- /dev/null +++ b/SOURCES/kvm-Fix-libusb-1.0.22-deprecated-libusb_set_debug-with-l.patch @@ -0,0 +1,67 @@ +From f79fc6d50a69dc3443f028b8f0fa11fe0f94810c Mon Sep 17 00:00:00 2001 +From: Danilo de Paula +Date: Mon, 27 Aug 2018 17:15:07 +0100 +Subject: [PATCH] Fix libusb-1.0.22 deprecated libusb_set_debug with + libusb_set_option + +RH-Author: Danilo de Paula +Message-id: <20180827171507.6372-2-ddepaula@redhat.com> +Patchwork-id: 81932 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 1/1] Fix libusb-1.0.22 deprecated libusb_set_debug with libusb_set_option +Bugzilla: 1622656 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Gerd Hoffmann +RH-Acked-by: Laurent Vivier + +From: John Thomson + +libusb-1.0.22 marked libusb_set_debug deprecated +it is replaced with +libusb_set_option(libusb_context, LIBUSB_OPTION_LOG_LEVEL, libusb_log_level); + +details here: https://github.com/libusb/libusb/commit/539f22e2fd916558d11ab9a66f10f461c5593168 + +Warning here: + + CC hw/usb/host-libusb.o +/builds/xen/src/qemu-xen/hw/usb/host-libusb.c: In function 'usb_host_init': +/builds/xen/src/qemu-xen/hw/usb/host-libusb.c:250:5: error: 'libusb_set_debug' is deprecated: Use libusb_set_option instead [-Werror=deprecated-declarations] + libusb_set_debug(ctx, loglevel); + ^~~~~~~~~~~~~~~~ +In file included from /builds/xen/src/qemu-xen/hw/usb/host-libusb.c:40:0: +/usr/include/libusb-1.0/libusb.h:1300:18: note: declared here + void LIBUSB_CALL libusb_set_debug(libusb_context *ctx, int level); + ^~~~~~~~~~~~~~~~ +cc1: all warnings being treated as errors +make: *** [/builds/xen/src/qemu-xen/rules.mak:66: hw/usb/host-libusb.o] Error 1 +make: Leaving directory '/builds/xen/src/xen/tools/qemu-xen-build' + +Signed-off-by: John Thomson +Message-id: 20180405132046.4968-1-git@johnthomson.fastmail.com.au +Signed-off-by: Gerd Hoffmann + +(cherry picked from commit 9d8fa0df49af16a208fa961c2968fba4daffcc07) +Signed-off-by: Danilo C. L. de Paula +--- + hw/usb/host-libusb.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c +index 0290fb8..f31e9cb 100644 +--- a/hw/usb/host-libusb.c ++++ b/hw/usb/host-libusb.c +@@ -248,7 +248,11 @@ static int usb_host_init(void) + if (rc != 0) { + return -1; + } ++#if LIBUSB_API_VERSION >= 0x01000106 ++ libusb_set_option(ctx, LIBUSB_OPTION_LOG_LEVEL, loglevel); ++#else + libusb_set_debug(ctx, loglevel); ++#endif + #ifdef CONFIG_WIN32 + /* FIXME: add support for Windows. */ + #else +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Fix-x-hv-max-vps-compat-value-for-7.4-machine-type.patch b/SOURCES/kvm-Fix-x-hv-max-vps-compat-value-for-7.4-machine-type.patch new file mode 100644 index 0000000..008f07b --- /dev/null +++ b/SOURCES/kvm-Fix-x-hv-max-vps-compat-value-for-7.4-machine-type.patch @@ -0,0 +1,60 @@ +From 738561e0b91258ad42765d669a62e9f28784fefe Mon Sep 17 00:00:00 2001 +From: Miroslav Rezanina +Date: Thu, 31 May 2018 06:36:35 +0200 +Subject: [PATCH 005/268] Fix x-hv-max-vps compat value for 7.4 machine type + +RH-Author: Miroslav Rezanina +Message-id: <1527748595-28488-1-git-send-email-mrezanin@redhat.com> +Patchwork-id: 80541 +O-Subject: [RHEL qemu-kvm/qemu-kvm-rhev PATCH] Fix x-hv-max-vps compat value for 7.4 machine type +Bugzilla: 1583959 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Laszlo Ersek + +Commit b2f9f4fcaad9c64f4551ab1dbe9e474c3dc6b2b4 increased the limit of vcpus +for windows guest. In addition, it keep old limit for older machine types. +However, due to conflict was compatibility part incorrectly placed and +we keep this limit for 7.3 and older machine types instead of 7.4 and older. + +Moving the chunk to correct spot so we have correct limit. + +Signed-off-by: Miroslav Rezanina +--- + include/hw/i386/pc.h | 11 +++++------ + 1 file changed, 5 insertions(+), 6 deletions(-) + +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index c33ddbb..285e8df 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -1019,9 +1019,13 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .driver = "q35-pcihost",\ + .property = "x-pci-hole64-fix",\ + .value = "off",\ ++ },\ ++ { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_10 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "x-hv-max-vps",\ ++ .value = "0x40",\ + }, + +- + #define PC_RHEL7_3_COMPAT \ + HW_COMPAT_RHEL7_3 \ + { /* PC_RHEL7_3_COMPAT from PC_COMPAT_2_8 */ \ +@@ -1083,11 +1087,6 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .driver = TYPE_X86_CPU,\ + .property = "kvm-no-smi-migration",\ + .value = "on",\ +- },\ +- { /* PC_RHEL7_4_COMPAT from PC_COMPAT_2_10 */ \ +- .driver = TYPE_X86_CPU,\ +- .property = "x-hv-max-vps",\ +- .value = "0x40",\ + }, + + #define PC_RHEL7_2_COMPAT \ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Migration-TLS-Fix-crash-due-to-double-cleanup.patch b/SOURCES/kvm-Migration-TLS-Fix-crash-due-to-double-cleanup.patch new file mode 100644 index 0000000..e943541 --- /dev/null +++ b/SOURCES/kvm-Migration-TLS-Fix-crash-due-to-double-cleanup.patch @@ -0,0 +1,65 @@ +From 699be60852400ad3459992a02c8477c08944ad09 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:18 +0100 +Subject: [PATCH] Migration+TLS: Fix crash due to double cleanup +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-15-dgilbert@redhat.com> +Patchwork-id: 81569 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 14/18] Migration+TLS: Fix crash due to double cleanup +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: "Dr. David Alan Gilbert" + +During a TLS connect we see: + migration_channel_connect calls + migration_tls_channel_connect + (calls after TLS setup) + migration_channel_connect + +My previous error handling fix made migration_channel_connect +call migrate_fd_connect in all cases; unfortunately the above +means it gets called twice and crashes doing double cleanup. + +Fixes: 688a3dcba98 + +Reported-by: Peter Krempa +Signed-off-by: Dr. David Alan Gilbert +Reviewed-by: Daniel P. Berrangé +Message-Id: <20180430185943.35714-1-dgilbert@redhat.com> +Signed-off-by: Juan Quintela +(cherry picked from commit 8b7bf2badac25c0a52aff1b181ad75fdb304dd0c) +Signed-off-by: Danilo C. L. de Paula +--- + migration/channel.c | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/migration/channel.c b/migration/channel.c +index c5eaf0f..7a32b5a 100644 +--- a/migration/channel.c ++++ b/migration/channel.c +@@ -71,6 +71,15 @@ void migration_channel_connect(MigrationState *s, + !object_dynamic_cast(OBJECT(ioc), + TYPE_QIO_CHANNEL_TLS)) { + migration_tls_channel_connect(s, ioc, hostname, &error); ++ ++ if (!error) { ++ /* tls_channel_connect will call back to this ++ * function after the TLS handshake, ++ * so we mustn't call migrate_fd_connect until then ++ */ ++ ++ return; ++ } + } else { + QEMUFile *f = qemu_fopen_channel_output(ioc); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-RHEL-8.0-Add-pseries-rhel7.6.0-sxxm-machine-type.patch b/SOURCES/kvm-RHEL-8.0-Add-pseries-rhel7.6.0-sxxm-machine-type.patch new file mode 100644 index 0000000..394c2c9 --- /dev/null +++ b/SOURCES/kvm-RHEL-8.0-Add-pseries-rhel7.6.0-sxxm-machine-type.patch @@ -0,0 +1,73 @@ +From 6c2f10596f53c29687a64aa78f339e3043850936 Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Wed, 25 Jul 2018 08:36:42 +0100 +Subject: [PATCH 03/14] RHEL-8.0: Add pseries-rhel7.6.0-sxxm machine type + +RH-Author: David Gibson +Message-id: <20180725083642.11004-1-dgibson@redhat.com> +Patchwork-id: 81501 +O-Subject: [RHEL-8.0 qemu-kvm PATCH] RHEL-8.0: Add pseries-rhel7.6.0-sxxm machine type +Bugzilla: 1595501 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych +RH-Acked-by: Thomas Huth + +For the pseries-rhel7.3.0 .. pseries-rhel7.5.0 machine types we have -sxxm +variants, the only difference being that the -sxxm variants have the +Spectre and Meltdown mitigations available to guests by default. + +We'd delayed on adding a similar variant for RHEL 7.6, in the hope that we +might be able to enable the mitigations by default for the ordinary 7.6 +machine type. This requires that updated POWER8 firmware (FW860.51 or +newer) be installed on the host. + +The updated firmware was only released late May, and it's not clear how +quickly and widely it will be deployed. For that reason, plus for +consistency in how things need to be configured across rhel-7.y, we're now +adding a pseries-rhel7.6.0-sxxm machine type. + +Upstream status: downstream only +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1595501 + +Signed-off-by: David Gibson +Signed-off-by: Danilo C. L. de Paula +--- + hw/ppc/spapr.c | 22 ++++++++++++++++++++++ + 1 file changed, 22 insertions(+) + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index 7de3f07..a61dafd 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -4355,6 +4355,28 @@ static void spapr_machine_rhel760_class_options(MachineClass *mc) + DEFINE_SPAPR_MACHINE(rhel760, "rhel7.6.0", true); + + /* ++ * pseries-rhel7.6.0-sxxm ++ * ++ * pseries-rhel7.6.0 with speculative execution exploit mitigations enabled by default ++ */ ++static void spapr_machine_rhel760sxxm_instance_options(MachineState *machine) ++{ ++ spapr_machine_rhel760_instance_options(machine); ++} ++ ++static void spapr_machine_rhel760sxxm_class_options(MachineClass *mc) ++{ ++ sPAPRMachineClass *smc = SPAPR_MACHINE_CLASS(mc); ++ ++ spapr_machine_rhel760_class_options(mc); ++ smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_WORKAROUND; ++ smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_FIXED_CCD; ++} ++ ++DEFINE_SPAPR_MACHINE(rhel760sxxm, "rhel7.6.0-sxxm", false); ++ ++/* + * pseries-rhel7.5.0 + * like SPAPR_COMPAT_2_11 and SPAPR_COMPAT_2_10 + * SPAPR_CAP_HTM already enabled in 7.4 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Re-enable-CONFIG_HYPERV_TESTDEV.patch b/SOURCES/kvm-Re-enable-CONFIG_HYPERV_TESTDEV.patch new file mode 100644 index 0000000..a6ca124 --- /dev/null +++ b/SOURCES/kvm-Re-enable-CONFIG_HYPERV_TESTDEV.patch @@ -0,0 +1,45 @@ +From d51e082e60b3f28d595ebb88e2b4ac17384cecee Mon Sep 17 00:00:00 2001 +From: Vitaly Kuznetsov +Date: Mon, 19 Nov 2018 15:47:06 +0000 +Subject: [PATCH 05/16] Re-enable CONFIG_HYPERV_TESTDEV + +RH-Author: Vitaly Kuznetsov +Message-id: <20181119154706.26183-1-vkuznets@redhat.com> +Patchwork-id: 83046 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH] Re-enable CONFIG_HYPERV_TESTDEV +Bugzilla: 1651195 +RH-Acked-by: Luiz Capitulino +RH-Acked-by: Laurent Vivier +RH-Acked-by: Auger Eric + +BZ: 1651195 +BRANCH: rhel8/master-2.12.0 +UPSTREAM: downstream only +BREW: 19214465 + +Recently we re-enabled 'hv_synic' and 'hv_stimer' enlightenments for +Windows guests. Enable 'hyperv-testdev' device so kvm-unit-tests tests for +synic and stimer can pass. + +Signed-off-by: Vitaly Kuznetsov +Signed-off-by: Danilo C. L. de Paula +--- + default-configs/x86_64-softmmu.mak | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak +index 2675606..854eab1 100644 +--- a/default-configs/x86_64-softmmu.mak ++++ b/default-configs/x86_64-softmmu.mak +@@ -60,7 +60,7 @@ CONFIG_XIO3130=y + CONFIG_IOH3420=y + CONFIG_I82801B11=y + CONFIG_SMBIOS=y +-#CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM) ++CONFIG_HYPERV_TESTDEV=$(CONFIG_KVM) + CONFIG_PXB=y + CONFIG_ACPI_VMGENID=y + CONFIG_FW_CFG_DMA=y +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Re-enable-disabled-Hyper-V-enlightenments.patch b/SOURCES/kvm-Re-enable-disabled-Hyper-V-enlightenments.patch new file mode 100644 index 0000000..c7eaaa0 --- /dev/null +++ b/SOURCES/kvm-Re-enable-disabled-Hyper-V-enlightenments.patch @@ -0,0 +1,51 @@ +From a2721f63de158e00dda6b043ae8465fb31b8a619 Mon Sep 17 00:00:00 2001 +From: Vitaly Kuznetsov +Date: Wed, 5 Sep 2018 09:41:52 +0100 +Subject: [PATCH 1/4] Re-enable disabled Hyper-V enlightenments + +RH-Author: Vitaly Kuznetsov +Message-id: <20180905094152.31940-1-vkuznets@redhat.com> +Patchwork-id: 82051 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH] Re-enable disabled Hyper-V enlightenments +Bugzilla: 1625185 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: vrozenfe +RH-Acked-by: Eduardo Otubo + +BZ: 1625185 +BRANCH:rhel8/master-2.12.0 +UPSTREAM: downstream only +BREW: 18157023 + +Partially revert 0d70915c93d. With the latest Win10 update stimer/synic +enlightenments are a must (see BZ#1610461), vpindex is needed for the +upcoming PV TLB flush/IPI support, reset and runtime are enabled for +consistency with upstream (there's no benefit in disabling them). + +Signed-off-by: Vitaly Kuznetsov +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 2 -- + 1 file changed, 2 deletions(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index e16dba7..0215b20 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -5366,13 +5366,11 @@ static Property x86_cpu_properties[] = { + DEFINE_PROP_BOOL("hv-vapic", X86CPU, hyperv_vapic, false), + DEFINE_PROP_BOOL("hv-time", X86CPU, hyperv_time, false), + DEFINE_PROP_BOOL("hv-crash", X86CPU, hyperv_crash, false), +-#if 0 /* Disabled for Red Hat Enterprise Linux */ + DEFINE_PROP_BOOL("hv-reset", X86CPU, hyperv_reset, false), + DEFINE_PROP_BOOL("hv-vpindex", X86CPU, hyperv_vpindex, false), + DEFINE_PROP_BOOL("hv-runtime", X86CPU, hyperv_runtime, false), + DEFINE_PROP_BOOL("hv-synic", X86CPU, hyperv_synic, false), + DEFINE_PROP_BOOL("hv-stimer", X86CPU, hyperv_stimer, false), +-#endif + DEFINE_PROP_BOOL("hv-frequencies", X86CPU, hyperv_frequencies, false), + DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true), + DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false), +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Revert-hw-acpi-build-build-SRAT-memory-affinity-stru.patch b/SOURCES/kvm-Revert-hw-acpi-build-build-SRAT-memory-affinity-stru.patch new file mode 100644 index 0000000..11377bc --- /dev/null +++ b/SOURCES/kvm-Revert-hw-acpi-build-build-SRAT-memory-affinity-stru.patch @@ -0,0 +1,109 @@ +From c07dfca3b340161294755691f416ab20f37fa7c3 Mon Sep 17 00:00:00 2001 +From: Igor Mammedov +Date: Thu, 4 Oct 2018 10:31:31 +0100 +Subject: [PATCH 01/17] Revert "hw/acpi-build: build SRAT memory affinity + structures for DIMM devices" + +RH-Author: Igor Mammedov +Message-id: <1538649091-70517-1-git-send-email-imammedo@redhat.com> +Patchwork-id: 82373 +O-Subject: [RHEL8/virt-8.0.0 qemu-kvm PATCH] Revert "hw/acpi-build: build SRAT memory affinity structures for DIMM devices" +Bugzilla: 1609235 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Thomas Huth + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1609235 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18630981 +Upstream: RHEL only + +Since upstream commits + (0efd7e108 "pc: acpi: fix memory hotplug regression by reducing stub SRAT entry size") + (dbb6da8ba7 "pc: acpi: revert back to 1 SRAT entry for hotpluggable area") +hasn't been backported to RHEL8, it's sufficient to revert commit + (848a1cc1e8 "hw/acpi-build: build SRAT memory affinity structures for DIMM devices") +for the result to match the current upstream state and fix the bug. + +Signed-off-by: Igor Mammedov +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/acpi-build.c | 56 ++++------------------------------------------------ + 1 file changed, 4 insertions(+), 52 deletions(-) + +diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c +index b309a97..a175a8a 100644 +--- a/hw/i386/acpi-build.c ++++ b/hw/i386/acpi-build.c +@@ -2253,55 +2253,6 @@ build_tpm2(GArray *table_data, BIOSLinker *linker, GArray *tcpalog) + #define HOLE_640K_START (640 * 1024) + #define HOLE_640K_END (1024 * 1024) + +-static void build_srat_hotpluggable_memory(GArray *table_data, uint64_t base, +- uint64_t len, int default_node) +-{ +- MemoryDeviceInfoList *info_list = qmp_pc_dimm_device_list(); +- MemoryDeviceInfoList *info; +- MemoryDeviceInfo *mi; +- PCDIMMDeviceInfo *di; +- uint64_t end = base + len, cur, size; +- bool is_nvdimm; +- AcpiSratMemoryAffinity *numamem; +- MemoryAffinityFlags flags; +- +- for (cur = base, info = info_list; +- cur < end; +- cur += size, info = info->next) { +- numamem = acpi_data_push(table_data, sizeof *numamem); +- +- if (!info) { +- build_srat_memory(numamem, cur, end - cur, default_node, +- MEM_AFFINITY_HOTPLUGGABLE | MEM_AFFINITY_ENABLED); +- break; +- } +- +- mi = info->value; +- is_nvdimm = (mi->type == MEMORY_DEVICE_INFO_KIND_NVDIMM); +- di = !is_nvdimm ? mi->u.dimm.data : mi->u.nvdimm.data; +- +- if (cur < di->addr) { +- build_srat_memory(numamem, cur, di->addr - cur, default_node, +- MEM_AFFINITY_HOTPLUGGABLE | MEM_AFFINITY_ENABLED); +- numamem = acpi_data_push(table_data, sizeof *numamem); +- } +- +- size = di->size; +- +- flags = MEM_AFFINITY_ENABLED; +- if (di->hotpluggable) { +- flags |= MEM_AFFINITY_HOTPLUGGABLE; +- } +- if (is_nvdimm) { +- flags |= MEM_AFFINITY_NON_VOLATILE; +- } +- +- build_srat_memory(numamem, di->addr, size, di->node, flags); +- } +- +- qapi_free_MemoryDeviceInfoList(info_list); +-} +- + static void + build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine) + { +@@ -2413,9 +2364,10 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine) + * providing _PXM method if necessary. + */ + if (hotplugabble_address_space_size) { +- build_srat_hotpluggable_memory(table_data, pcms->hotplug_memory.base, +- hotplugabble_address_space_size, +- pcms->numa_nodes - 1); ++ numamem = acpi_data_push(table_data, sizeof *numamem); ++ build_srat_memory(numamem, pcms->hotplug_memory.base, ++ hotplugabble_address_space_size, pcms->numa_nodes - 1, ++ MEM_AFFINITY_HOTPLUGGABLE | MEM_AFFINITY_ENABLED); + } + + build_header(linker, table_data, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Revert-spapr-Don-t-allow-memory-hotplug-to-memory-le.patch b/SOURCES/kvm-Revert-spapr-Don-t-allow-memory-hotplug-to-memory-le.patch new file mode 100644 index 0000000..6d8928a --- /dev/null +++ b/SOURCES/kvm-Revert-spapr-Don-t-allow-memory-hotplug-to-memory-le.patch @@ -0,0 +1,69 @@ +From c30a2c251e309eba40fd04e36b80016e4858c586 Mon Sep 17 00:00:00 2001 +From: Serhii Popovych +Date: Wed, 11 Jul 2018 17:11:45 +0100 +Subject: [PATCH 2/4] Revert "spapr: Don't allow memory hotplug to memory less + nodes" + +RH-Author: Serhii Popovych +Message-id: <1531329105-80927-3-git-send-email-spopovyc@redhat.com> +Patchwork-id: 81313 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 2/2] Revert "spapr: Don't allow memory hotplug to memory less nodes" +Bugzilla: 1599593 +RH-Acked-by: Laurent Vivier +RH-Acked-by: David Gibson +RH-Acked-by: Thomas Huth + +This reverts commit b556854bd8524c26b8be98ab1bfdf0826831e793. + +Leave change @node type from uint32_t to to int from reverted commit +because node < 0 is always false. + +Note that implementing capability or some trick to detect if guest +kernel does not support hot-add to memory: this returns previous +behavour where memory added to first non-empty node. + +Signed-off-by: Serhii Popovych +Signed-off-by: David Gibson +(cherry picked from commit e47f1d2786c3d01a7894a493aafe0efa6b64453c) +Signed-off-by: Serhii Popovych +Signed-off-by: Danilo C. L. de Paula +--- + hw/ppc/spapr.c | 22 ---------------------- + 1 file changed, 22 deletions(-) + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index f3da93f..ef00937 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -3489,28 +3489,6 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev, + return; + } + +- /* +- * Currently PowerPC kernel doesn't allow hot-adding memory to +- * memory-less node, but instead will silently add the memory +- * to the first node that has some memory. This causes two +- * unexpected behaviours for the user. +- * +- * - Memory gets hotplugged to a different node than what the user +- * specified. +- * - Since pc-dimm subsystem in QEMU still thinks that memory belongs +- * to memory-less node, a reboot will set things accordingly +- * and the previously hotplugged memory now ends in the right node. +- * This appears as if some memory moved from one node to another. +- * +- * So until kernel starts supporting memory hotplug to memory-less +- * nodes, just prevent such attempts upfront in QEMU. +- */ +- if (nb_numa_nodes && !numa_info[node].node_mem) { +- error_setg(errp, "Can't hotplug memory to memory-less node %d", +- node); +- return; +- } +- + spapr_memory_plug(hotplug_dev, dev, node, errp); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) { + spapr_core_plug(hotplug_dev, dev, errp); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Revert-usb-release-the-created-buses.patch b/SOURCES/kvm-Revert-usb-release-the-created-buses.patch new file mode 100644 index 0000000..41480c6 --- /dev/null +++ b/SOURCES/kvm-Revert-usb-release-the-created-buses.patch @@ -0,0 +1,99 @@ +From 432dce181a267465b10d3e4bca025314d85ff76f Mon Sep 17 00:00:00 2001 +From: Serhii Popovych +Date: Mon, 9 Jul 2018 11:31:18 +0200 +Subject: [PATCH 202/268] Revert "usb: release the created buses" +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Serhii Popovych +Message-id: <1531135878-18813-4-git-send-email-spopovyc@redhat.com> +Patchwork-id: 81266 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 3/3] Revert "usb: release the created buses" +Bugzilla: 1556678 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Marc-André Lureau +RH-Acked-by: David Gibson + +From: Marc-André Lureau + +The USB device don't hold the bus. There is no ASAN related reports +anymore. + +This reverts commit cd7bc87868d534f95e928cad98e2a52df7695771. + +Signed-off-by: Marc-André Lureau +Message-id: 20180613172815.32738-3-marcandre.lureau@redhat.com +Signed-off-by: Gerd Hoffmann +(cherry picked from commit 9b5c2fd53feb574036747d0284fd7f73dfedc89c) +Signed-off-by: Serhii Popovych +Signed-off-by: Miroslav Rezanina +--- + hw/usb/dev-storage.c | 16 ---------------- + hw/usb/dev-uas.c | 2 -- + 2 files changed, 18 deletions(-) + +diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c +index 68e2062..9fb00a9 100644 +--- a/hw/usb/dev-storage.c ++++ b/hw/usb/dev-storage.c +@@ -589,13 +589,6 @@ static const struct SCSIBusInfo usb_msd_scsi_info_bot = { + .load_request = usb_msd_load_request, + }; + +-static void usb_msd_unrealize_storage(USBDevice *dev, Error **errp) +-{ +- MSDState *s = USB_STORAGE_DEV(dev); +- +- object_unref(OBJECT(&s->bus)); +-} +- + static void usb_msd_storage_realize(USBDevice *dev, Error **errp) + { + MSDState *s = USB_STORAGE_DEV(dev); +@@ -645,13 +638,6 @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp) + s->scsi_dev = scsi_dev; + } + +-static void usb_msd_bot_unrealize(USBDevice *dev, Error **errp) +-{ +- MSDState *s = USB_STORAGE_DEV(dev); +- +- object_unref(OBJECT(&s->bus)); +-} +- + static void usb_msd_bot_realize(USBDevice *dev, Error **errp) + { + MSDState *s = USB_STORAGE_DEV(dev); +@@ -716,7 +702,6 @@ static void usb_msd_class_storage_initfn(ObjectClass *klass, void *data) + USBDeviceClass *uc = USB_DEVICE_CLASS(klass); + + uc->realize = usb_msd_storage_realize; +- uc->unrealize = usb_msd_unrealize_storage; + dc->props = msd_properties; + } + +@@ -779,7 +764,6 @@ static void usb_msd_class_bot_initfn(ObjectClass *klass, void *data) + USBDeviceClass *uc = USB_DEVICE_CLASS(klass); + + uc->realize = usb_msd_bot_realize; +- uc->unrealize = usb_msd_bot_unrealize; + uc->attached_settable = true; + } + +diff --git a/hw/usb/dev-uas.c b/hw/usb/dev-uas.c +index c218b53..eaf515d 100644 +--- a/hw/usb/dev-uas.c ++++ b/hw/usb/dev-uas.c +@@ -896,8 +896,6 @@ static void usb_uas_unrealize(USBDevice *dev, Error **errp) + UASDevice *uas = USB_UAS(dev); + + qemu_bh_delete(uas->status_bh); +- +- object_unref(OBJECT(&uas->bus)); + } + + static void usb_uas_realize(USBDevice *dev, Error **errp) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Use-4-MB-vram-for-cirrus.patch b/SOURCES/kvm-Use-4-MB-vram-for-cirrus.patch new file mode 100644 index 0000000..5a586ac --- /dev/null +++ b/SOURCES/kvm-Use-4-MB-vram-for-cirrus.patch @@ -0,0 +1,63 @@ +From e924798dc9091c09e4e6968b9357e89aac6a5c03 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 13 Jun 2018 10:40:26 +0200 +Subject: [PATCH 007/268] Use 4 MB vram for cirrus. + +RH-Author: Gerd Hoffmann +Message-id: <20180613104026.4395-3-kraxel@redhat.com> +Patchwork-id: 80652 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/2] Use 4 MB vram for cirrus. +Bugzilla: 1542080 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Daniel P. Berrange + +Follow upstream, which switched from 8 MB to 4 MB in release 2.9, +see commit 73c148130b58709f0f2abfedbae92681d87eb404. + +RHEL traditionally defaults to 16 MB video memory, so we had a patch +changing 8 MB -> 16 MB downstream. The 2.9 rebase kept that patch, now +changing 4 MB -> 16 MB. + +This patch effectively reverts the downstream change and adds a compat +property so the video memory will stay at 16 MB for 7.5 and older +machine types. + +Signed-off-by: Gerd Hoffmann +Signed-off-by: Miroslav Rezanina +--- + hw/display/cirrus_vga.c | 2 +- + include/hw/compat.h | 4 ++++ + 2 files changed, 5 insertions(+), 1 deletion(-) + +diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c +index feacb45..014268a 100644 +--- a/hw/display/cirrus_vga.c ++++ b/hw/display/cirrus_vga.c +@@ -3133,7 +3133,7 @@ static void pci_cirrus_vga_realize(PCIDevice *dev, Error **errp) + + static Property pci_vga_cirrus_properties[] = { + DEFINE_PROP_UINT32("vgamem_mb", struct PCICirrusVGAState, +- cirrus_vga.vga.vram_size_mb, 16), ++ cirrus_vga.vga.vram_size_mb, 4), + DEFINE_PROP_BOOL("blitter", struct PCICirrusVGAState, + cirrus_vga.enable_blitter, true), + DEFINE_PROP_END_OF_LIST(), +diff --git a/include/hw/compat.h b/include/hw/compat.h +index f7b39c5..f4cc6e0 100644 +--- a/include/hw/compat.h ++++ b/include/hw/compat.h +@@ -472,6 +472,10 @@ + .driver = "virtio-tablet-device",\ + .property = "wheel-axis",\ + .value = "false",\ ++ },{ /* HW_COMPAT_RHEL7_5 */ \ ++ .driver = "cirrus-vga",\ ++ .property = "vgamem_mb",\ ++ .value = "16",\ + }, + + #endif /* HW_COMPAT_H */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-Use-inhibit-to-prevent-ballooning-without-synchr.patch b/SOURCES/kvm-Use-inhibit-to-prevent-ballooning-without-synchr.patch new file mode 100644 index 0000000..e7ff6b4 --- /dev/null +++ b/SOURCES/kvm-Use-inhibit-to-prevent-ballooning-without-synchr.patch @@ -0,0 +1,80 @@ +From 486481ac5bddfab22b74bcebcbd27af1f7a604f5 Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:01:34 +0000 +Subject: [PATCH 11/16] kvm: Use inhibit to prevent ballooning without + synchronous mmu + +RH-Author: Alex Williamson +Message-id: <154387449407.27651.8902076548922742582.stgit@gimli.home> +Patchwork-id: 83237 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 2/7] kvm: Use inhibit to prevent ballooning without synchronous mmu +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +Remove KVM specific tests in balloon_page(), instead marking +ballooning as inhibited without KVM_CAP_SYNC_MMU support. + +Reviewed-by: David Hildenbrand +Reviewed-by: Peter Xu +Reviewed-by: Cornelia Huck +Acked-by: Paolo Bonzini +Signed-off-by: Alex Williamson +(cherry picked from commit f59489423ab79852e98d9b3025b7d99ba8da584f) +Signed-off-by: Danilo C. L. de Paula +--- + accel/kvm/kvm-all.c | 4 ++++ + hw/virtio/virtio-balloon.c | 4 +--- + 2 files changed, 5 insertions(+), 3 deletions(-) + +diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c +index 3f1c06e..3a7c8a3 100644 +--- a/accel/kvm/kvm-all.c ++++ b/accel/kvm/kvm-all.c +@@ -39,6 +39,7 @@ + #include "trace.h" + #include "hw/irq.h" + #include "sysemu/sev.h" ++#include "sysemu/balloon.h" + + #include "hw/boards.h" + +@@ -1711,6 +1712,9 @@ static int kvm_init(MachineState *ms) + s->many_ioeventfds = kvm_check_many_ioeventfds(); + + s->sync_mmu = !!kvm_vm_check_extension(kvm_state, KVM_CAP_SYNC_MMU); ++ if (!s->sync_mmu) { ++ qemu_balloon_inhibit(true); ++ } + + return 0; + +diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c +index f456cea..4aab291 100644 +--- a/hw/virtio/virtio-balloon.c ++++ b/hw/virtio/virtio-balloon.c +@@ -21,7 +21,6 @@ + #include "hw/mem/pc-dimm.h" + #include "sysemu/balloon.h" + #include "hw/virtio/virtio-balloon.h" +-#include "sysemu/kvm.h" + #include "exec/address-spaces.h" + #include "qapi/error.h" + #include "qapi/qapi-events-misc.h" +@@ -36,8 +35,7 @@ + + static void balloon_page(void *addr, int deflate) + { +- if (!qemu_balloon_is_inhibited() && (!kvm_enabled() || +- kvm_has_sync_mmu())) { ++ if (!qemu_balloon_is_inhibited()) { + qemu_madvise(addr, BALLOON_PAGE_SIZE, + deflate ? QEMU_MADV_WILLNEED : QEMU_MADV_DONTNEED); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ahci-don-t-schedule-unnecessary-BH.patch b/SOURCES/kvm-ahci-don-t-schedule-unnecessary-BH.patch new file mode 100644 index 0000000..fe1de0c --- /dev/null +++ b/SOURCES/kvm-ahci-don-t-schedule-unnecessary-BH.patch @@ -0,0 +1,46 @@ +From 16e7c9997d4e9682095206ef3c07d713d263143c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 10 Jul 2018 23:06:16 +0200 +Subject: [PATCH 207/268] ahci: don't schedule unnecessary BH + +RH-Author: John Snow +Message-id: <20180710230616.11000-4-jsnow@redhat.com> +Patchwork-id: 81292 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 3/3] ahci: don't schedule unnecessary BH +Bugzilla: 1584914 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Stefan Hajnoczi + +The comment gives us a hint. *Maybe* we still have something to +process. Well, why not check? + +Signed-off-by: John Snow +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Jeff Cody +Message-id: 20180531004323.4611-4-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 42af312adef8afdae11d5f83d12a404b178dbda4) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + hw/ide/ahci.c | 3 +-- + 1 file changed, 1 insertion(+), 2 deletions(-) + +diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c +index a9558e4..380366b 100644 +--- a/hw/ide/ahci.c ++++ b/hw/ide/ahci.c +@@ -1427,8 +1427,7 @@ static void ahci_cmd_done(IDEDMA *dma) + /* update d2h status */ + ahci_write_fis_d2h(ad); + +- if (!ad->check_bh) { +- /* maybe we still have something to process, check later */ ++ if (ad->port_regs.cmd_issue && !ad->check_bh) { + ad->check_bh = qemu_bh_new(ahci_check_cmd_bh, ad); + qemu_bh_schedule(ad->check_bh); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ahci-fix-PxCI-register-race.patch b/SOURCES/kvm-ahci-fix-PxCI-register-race.patch new file mode 100644 index 0000000..aa78142 --- /dev/null +++ b/SOURCES/kvm-ahci-fix-PxCI-register-race.patch @@ -0,0 +1,81 @@ +From 93623a848fba7757a0840b78d7b3874bab4d7a1b Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 10 Jul 2018 23:06:15 +0200 +Subject: [PATCH 206/268] ahci: fix PxCI register race +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: John Snow +Message-id: <20180710230616.11000-3-jsnow@redhat.com> +Patchwork-id: 81293 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/3] ahci: fix PxCI register race +Bugzilla: 1584914 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Stefan Hajnoczi + +Fixes: https://bugs.launchpad.net/qemu/+bug/1769189 + +AHCI presently signals completion prior to the PxCI register being +cleared to indicate completion. If a guest driver attempts to issue +a new command in its IRQ handler, it might be surprised to learn there +is still a command pending. + +In the case of Windows 10's boot driver, it will actually poll the IRQ +register hoping to find out when the command is done running -- which +will never happen, as there isn't a command running. + +Fix this: clear PxCI in ahci_cmd_done and not in the asynchronous BH. +Because it now runs synchronously, we don't need to check if the command +is actually done by spying on the ATA registers. We know it's done. + +CC: qemu-stable +Reported-by: François Guerraz +Tested-by: Bruce Rogers +Signed-off-by: John Snow +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Jeff Cody +Message-id: 20180531004323.4611-3-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 5694c7eacce6b263ad7497cc1bb76aad746cfd4e) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + hw/ide/ahci.c | 13 ++++++------- + 1 file changed, 6 insertions(+), 7 deletions(-) + +diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c +index b7a6f68..a9558e4 100644 +--- a/hw/ide/ahci.c ++++ b/hw/ide/ahci.c +@@ -532,13 +532,6 @@ static void ahci_check_cmd_bh(void *opaque) + qemu_bh_delete(ad->check_bh); + ad->check_bh = NULL; + +- if ((ad->busy_slot != -1) && +- !(ad->port.ifs[0].status & (BUSY_STAT|DRQ_STAT))) { +- /* no longer busy */ +- ad->port_regs.cmd_issue &= ~(1 << ad->busy_slot); +- ad->busy_slot = -1; +- } +- + check_cmd(ad->hba, ad->port_no); + } + +@@ -1425,6 +1418,12 @@ static void ahci_cmd_done(IDEDMA *dma) + + trace_ahci_cmd_done(ad->hba, ad->port_no); + ++ /* no longer busy */ ++ if (ad->busy_slot != -1) { ++ ad->port_regs.cmd_issue &= ~(1 << ad->busy_slot); ++ ad->busy_slot = -1; ++ } ++ + /* update d2h status */ + ahci_write_fis_d2h(ad); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ahci-trim-signatures-on-raise-lower.patch b/SOURCES/kvm-ahci-trim-signatures-on-raise-lower.patch new file mode 100644 index 0000000..1921f27 --- /dev/null +++ b/SOURCES/kvm-ahci-trim-signatures-on-raise-lower.patch @@ -0,0 +1,66 @@ +From f671c509afc918070a550bda393d92bd197c5cab Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 10 Jul 2018 23:06:14 +0200 +Subject: [PATCH 205/268] ahci: trim signatures on raise/lower + +RH-Author: John Snow +Message-id: <20180710230616.11000-2-jsnow@redhat.com> +Patchwork-id: 81295 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/3] ahci: trim signatures on raise/lower +Bugzilla: 1584914 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Stefan Hajnoczi + +These functions work on the AHCI device, not the individual +AHCI devices, so trim the AHCIDevice argument. + +Signed-off-by: John Snow +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Jeff Cody +Message-id: 20180531004323.4611-2-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit dc5a43eda68fff32c7b0b43847332db325b094f3) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + hw/ide/ahci.c | 8 ++++---- + 1 file changed, 4 insertions(+), 4 deletions(-) + +diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c +index e22d7be..b7a6f68 100644 +--- a/hw/ide/ahci.c ++++ b/hw/ide/ahci.c +@@ -131,7 +131,7 @@ static uint32_t ahci_port_read(AHCIState *s, int port, int offset) + return val; + } + +-static void ahci_irq_raise(AHCIState *s, AHCIDevice *dev) ++static void ahci_irq_raise(AHCIState *s) + { + DeviceState *dev_state = s->container; + PCIDevice *pci_dev = (PCIDevice *) object_dynamic_cast(OBJECT(dev_state), +@@ -146,7 +146,7 @@ static void ahci_irq_raise(AHCIState *s, AHCIDevice *dev) + } + } + +-static void ahci_irq_lower(AHCIState *s, AHCIDevice *dev) ++static void ahci_irq_lower(AHCIState *s) + { + DeviceState *dev_state = s->container; + PCIDevice *pci_dev = (PCIDevice *) object_dynamic_cast(OBJECT(dev_state), +@@ -174,9 +174,9 @@ static void ahci_check_irq(AHCIState *s) + trace_ahci_check_irq(s, old_irq, s->control_regs.irqstatus); + if (s->control_regs.irqstatus && + (s->control_regs.ghc & HOST_CTL_IRQ_EN)) { +- ahci_irq_raise(s, NULL); ++ ahci_irq_raise(s); + } else { +- ahci_irq_lower(s, NULL); ++ ahci_irq_lower(s); + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-aio-Do-aio_notify_accept-only-during-blocking-aio_po.patch b/SOURCES/kvm-aio-Do-aio_notify_accept-only-during-blocking-aio_po.patch new file mode 100644 index 0000000..ab4e8f3 --- /dev/null +++ b/SOURCES/kvm-aio-Do-aio_notify_accept-only-during-blocking-aio_po.patch @@ -0,0 +1,124 @@ +From 4809b6fbd13f8fc67daf1e37254d98e8fb9a9f20 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Tue, 9 Oct 2018 08:16:48 +0100 +Subject: [PATCH 04/17] aio: Do aio_notify_accept only during blocking aio_poll + +RH-Author: Fam Zheng +Message-id: <20181009081651.15463-3-famz@redhat.com> +Patchwork-id: 82450 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/5] aio: Do aio_notify_accept only during blocking aio_poll +Bugzilla: 1623085 +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula + +BZ: 1623085 + +An aio_notify() pairs with an aio_notify_accept(). The former should +happen in the main thread or a vCPU thread, and the latter should be +done in the IOThread. + +There is one rare case that the main thread or vCPU thread may "steal" +the aio_notify() event just raised by itself, in bdrv_set_aio_context() +[1]. The sequence is like this: + + main thread IO Thread + =============================================================== + bdrv_drained_begin() + aio_disable_external(ctx) + aio_poll(ctx, true) + ctx->notify_me += 2 + ... + bdrv_drained_end() + ... + aio_notify() + ... + bdrv_set_aio_context() + aio_poll(ctx, false) +[1] aio_notify_accept(ctx) + ppoll() /* Hang! */ + +[1] is problematic. It will clear the ctx->notifier event so that +the blocked ppoll() will not return. + +(For the curious, this bug was noticed when booting a number of VMs +simultaneously in RHV. One or two of the VMs will hit this race +condition, making the VIRTIO device unresponsive to I/O commands. When +it hangs, Seabios is busy waiting for a read request to complete (read +MBR), right after initializing the virtio-blk-pci device, using 100% +guest CPU. See also https://bugzilla.redhat.com/show_bug.cgi?id=1562750 +for the original bug analysis.) + +aio_notify() only injects an event when ctx->notify_me is set, +correspondingly aio_notify_accept() is only useful when ctx->notify_me +_was_ set. Move the call to it into the "blocking" branch. This will +effectively skip [1] and fix the hang. + +Furthermore, blocking aio_poll is only allowed on home thread +(in_aio_context_home_thread), because otherwise two blocking +aio_poll()'s can steal each other's ctx->notifier event and cause +hanging just like described above. + +Cc: qemu-stable@nongnu.org +Suggested-by: Paolo Bonzini +Signed-off-by: Fam Zheng +Message-Id: <20180809132259.18402-3-famz@redhat.com> +Signed-off-by: Fam Zheng +(cherry picked from commit b37548fcd1b8ac2e88e185a395bef851f3fc4e65) +Signed-off-by: Fam Zheng +Signed-off-by: Danilo C. L. de Paula +--- + util/aio-posix.c | 4 ++-- + util/aio-win32.c | 3 ++- + 2 files changed, 4 insertions(+), 3 deletions(-) + +diff --git a/util/aio-posix.c b/util/aio-posix.c +index f650c7c..f05d3a8 100644 +--- a/util/aio-posix.c ++++ b/util/aio-posix.c +@@ -591,6 +591,7 @@ bool aio_poll(AioContext *ctx, bool blocking) + * so disable the optimization now. + */ + if (blocking) { ++ assert(in_aio_context_home_thread(ctx)); + atomic_add(&ctx->notify_me, 2); + } + +@@ -633,6 +634,7 @@ bool aio_poll(AioContext *ctx, bool blocking) + + if (blocking) { + atomic_sub(&ctx->notify_me, 2); ++ aio_notify_accept(ctx); + } + + /* Adjust polling time */ +@@ -676,8 +678,6 @@ bool aio_poll(AioContext *ctx, bool blocking) + } + } + +- aio_notify_accept(ctx); +- + /* if we have any readable fds, dispatch event */ + if (ret > 0) { + for (i = 0; i < npfd; i++) { +diff --git a/util/aio-win32.c b/util/aio-win32.c +index a67b00c..ac5524c 100644 +--- a/util/aio-win32.c ++++ b/util/aio-win32.c +@@ -373,11 +373,12 @@ bool aio_poll(AioContext *ctx, bool blocking) + ret = WaitForMultipleObjects(count, events, FALSE, timeout); + if (blocking) { + assert(first); ++ assert(in_aio_context_home_thread(ctx)); + atomic_sub(&ctx->notify_me, 2); ++ aio_notify_accept(ctx); + } + + if (first) { +- aio_notify_accept(ctx); + progress |= aio_bh_poll(ctx); + first = false; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-aio-posix-Don-t-count-ctx-notifier-as-progress-when-.patch b/SOURCES/kvm-aio-posix-Don-t-count-ctx-notifier-as-progress-when-.patch new file mode 100644 index 0000000..5bbc66c --- /dev/null +++ b/SOURCES/kvm-aio-posix-Don-t-count-ctx-notifier-as-progress-when-.patch @@ -0,0 +1,49 @@ +From 84935bf2482b06b983ccdb953b2cc90960d2ab16 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Tue, 9 Oct 2018 08:16:47 +0100 +Subject: [PATCH 03/17] aio-posix: Don't count ctx->notifier as progress when + polling + +RH-Author: Fam Zheng +Message-id: <20181009081651.15463-2-famz@redhat.com> +Patchwork-id: 82454 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/5] aio-posix: Don't count ctx->notifier as progress when polling +Bugzilla: 1623085 +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula + +BZ: 1623085 + +The same logic exists in fd polling. This change is especially important +to avoid busy loop once we limit aio_notify_accept() to blocking +aio_poll(). + +Cc: qemu-stable@nongnu.org +Signed-off-by: Fam Zheng +Message-Id: <20180809132259.18402-2-famz@redhat.com> +Signed-off-by: Fam Zheng +(cherry picked from commit 70232b5253a3c4e03ed1ac47ef9246a8ac66c6fa) +Signed-off-by: Fam Zheng +Signed-off-by: Danilo C. L. de Paula +--- + util/aio-posix.c | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/util/aio-posix.c b/util/aio-posix.c +index d8f0cb4..f650c7c 100644 +--- a/util/aio-posix.c ++++ b/util/aio-posix.c +@@ -494,7 +494,8 @@ static bool run_poll_handlers_once(AioContext *ctx) + QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { + if (!node->deleted && node->io_poll && + aio_node_check(ctx, node->is_external) && +- node->io_poll(node->opaque)) { ++ node->io_poll(node->opaque) && ++ node->opaque != &ctx->notifier) { + progress = true; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-aio-posix-compute-timeout-before-polling.patch b/SOURCES/kvm-aio-posix-compute-timeout-before-polling.patch new file mode 100644 index 0000000..f751810 --- /dev/null +++ b/SOURCES/kvm-aio-posix-compute-timeout-before-polling.patch @@ -0,0 +1,186 @@ +From 59776df6fa3156a467e889ae00751da4bf6857eb Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Tue, 9 Oct 2018 08:16:50 +0100 +Subject: [PATCH 06/17] aio-posix: compute timeout before polling + +RH-Author: Fam Zheng +Message-id: <20181009081651.15463-5-famz@redhat.com> +Patchwork-id: 82453 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 4/5] aio-posix: compute timeout before polling +Bugzilla: 1632622 +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula + +From: Paolo Bonzini + +BZ: 1632622 + +This is a preparation for the next patch, and also a very small +optimization. Compute the timeout only once, before invoking +try_poll_mode, and adjust it in run_poll_handlers. The adjustment +is the polling time when polling fails, or zero (non-blocking) if +polling succeeds. + +Fixes: 70232b5253a3c4e03ed1ac47ef9246a8ac66c6fa +Signed-off-by: Paolo Bonzini +Message-Id: <20180912171040.1732-3-pbonzini@redhat.com> +Reviewed-by: Fam Zheng +Signed-off-by: Fam Zheng +(cherry picked from commit e30cffa04d52e35996569f1cfac111be19576bde) +Signed-off-by: Fam Zheng +Signed-off-by: Danilo C. L. de Paula +--- + util/aio-posix.c | 59 ++++++++++++++++++++++++++++++++----------------------- + util/trace-events | 4 ++-- + 2 files changed, 36 insertions(+), 27 deletions(-) + +diff --git a/util/aio-posix.c b/util/aio-posix.c +index 1d7cc53..1b17597 100644 +--- a/util/aio-posix.c ++++ b/util/aio-posix.c +@@ -490,7 +490,7 @@ static void add_pollfd(AioHandler *node) + npfd++; + } + +-static bool run_poll_handlers_once(AioContext *ctx) ++static bool run_poll_handlers_once(AioContext *ctx, int64_t *timeout) + { + bool progress = false; + AioHandler *node; +@@ -500,6 +500,7 @@ static bool run_poll_handlers_once(AioContext *ctx) + aio_node_check(ctx, node->is_external) && + node->io_poll(node->opaque) && + node->opaque != &ctx->notifier) { ++ *timeout = 0; + progress = true; + } + +@@ -522,31 +523,38 @@ static bool run_poll_handlers_once(AioContext *ctx) + * + * Returns: true if progress was made, false otherwise + */ +-static bool run_poll_handlers(AioContext *ctx, int64_t max_ns) ++static bool run_poll_handlers(AioContext *ctx, int64_t max_ns, int64_t *timeout) + { + bool progress; +- int64_t end_time; ++ int64_t start_time, elapsed_time; + + assert(ctx->notify_me); + assert(qemu_lockcnt_count(&ctx->list_lock) > 0); + +- trace_run_poll_handlers_begin(ctx, max_ns); +- +- end_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + max_ns; ++ trace_run_poll_handlers_begin(ctx, max_ns, *timeout); + ++ start_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + do { +- progress = run_poll_handlers_once(ctx); +- } while (!progress && qemu_clock_get_ns(QEMU_CLOCK_REALTIME) < end_time ++ progress = run_poll_handlers_once(ctx, timeout); ++ elapsed_time = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - start_time; ++ } while (!progress && elapsed_time < max_ns + && !atomic_read(&ctx->poll_disable_cnt)); + +- trace_run_poll_handlers_end(ctx, progress); ++ /* If time has passed with no successful polling, adjust *timeout to ++ * keep the same ending time. ++ */ ++ if (*timeout != -1) { ++ *timeout -= MIN(*timeout, elapsed_time); ++ } + ++ trace_run_poll_handlers_end(ctx, progress, *timeout); + return progress; + } + + /* try_poll_mode: + * @ctx: the AioContext +- * @blocking: busy polling is only attempted when blocking is true ++ * @timeout: timeout for blocking wait, computed by the caller and updated if ++ * polling succeeds. + * + * ctx->notify_me must be non-zero so this function can detect aio_notify(). + * +@@ -554,19 +562,16 @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns) + * + * Returns: true if progress was made, false otherwise + */ +-static bool try_poll_mode(AioContext *ctx, bool blocking) ++static bool try_poll_mode(AioContext *ctx, int64_t *timeout) + { +- if (blocking && ctx->poll_max_ns && !atomic_read(&ctx->poll_disable_cnt)) { +- /* See qemu_soonest_timeout() uint64_t hack */ +- int64_t max_ns = MIN((uint64_t)aio_compute_timeout(ctx), +- (uint64_t)ctx->poll_ns); ++ /* See qemu_soonest_timeout() uint64_t hack */ ++ int64_t max_ns = MIN((uint64_t)*timeout, (uint64_t)ctx->poll_ns); + +- if (max_ns) { +- poll_set_started(ctx, true); ++ if (max_ns && !atomic_read(&ctx->poll_disable_cnt)) { ++ poll_set_started(ctx, true); + +- if (run_poll_handlers(ctx, max_ns)) { +- return true; +- } ++ if (run_poll_handlers(ctx, max_ns, timeout)) { ++ return true; + } + } + +@@ -575,7 +580,7 @@ static bool try_poll_mode(AioContext *ctx, bool blocking) + /* Even if we don't run busy polling, try polling once in case it can make + * progress and the caller will be able to avoid ppoll(2)/epoll_wait(2). + */ +- return run_poll_handlers_once(ctx); ++ return run_poll_handlers_once(ctx, timeout); + } + + bool aio_poll(AioContext *ctx, bool blocking) +@@ -605,8 +610,14 @@ bool aio_poll(AioContext *ctx, bool blocking) + start = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + } + +- progress = try_poll_mode(ctx, blocking); +- if (!progress) { ++ timeout = blocking ? aio_compute_timeout(ctx) : 0; ++ progress = try_poll_mode(ctx, &timeout); ++ assert(!(timeout && progress)); ++ ++ /* If polling is allowed, non-blocking aio_poll does not need the ++ * system call---a single round of run_poll_handlers_once suffices. ++ */ ++ if (timeout || atomic_read(&ctx->poll_disable_cnt)) { + assert(npfd == 0); + + /* fill pollfds */ +@@ -620,8 +631,6 @@ bool aio_poll(AioContext *ctx, bool blocking) + } + } + +- timeout = blocking ? aio_compute_timeout(ctx) : 0; +- + /* wait until next event */ + if (aio_epoll_check_poll(ctx, pollfds, npfd, timeout)) { + AioHandler epoll_handler; +diff --git a/util/trace-events b/util/trace-events +index 4822434..79569b7 100644 +--- a/util/trace-events ++++ b/util/trace-events +@@ -1,8 +1,8 @@ + # See docs/devel/tracing.txt for syntax documentation. + + # util/aio-posix.c +-run_poll_handlers_begin(void *ctx, int64_t max_ns) "ctx %p max_ns %"PRId64 +-run_poll_handlers_end(void *ctx, bool progress) "ctx %p progress %d" ++run_poll_handlers_begin(void *ctx, int64_t max_ns, int64_t timeout) "ctx %p max_ns %"PRId64 " timeout %"PRId64 ++run_poll_handlers_end(void *ctx, bool progress, int64_t timeout) "ctx %p progress %d new timeout %"PRId64 + poll_shrink(void *ctx, int64_t old, int64_t new) "ctx %p old %"PRId64" new %"PRId64 + poll_grow(void *ctx, int64_t old, int64_t new) "ctx %p old %"PRId64" new %"PRId64 + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-aio-posix-do-skip-system-call-if-ctx-notifier-pollin.patch b/SOURCES/kvm-aio-posix-do-skip-system-call-if-ctx-notifier-pollin.patch new file mode 100644 index 0000000..4610a40 --- /dev/null +++ b/SOURCES/kvm-aio-posix-do-skip-system-call-if-ctx-notifier-pollin.patch @@ -0,0 +1,65 @@ +From bd738e0bbc2fec04c928959e8e7e99af03024782 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Tue, 9 Oct 2018 08:16:51 +0100 +Subject: [PATCH 07/17] aio-posix: do skip system call if ctx->notifier polling + succeeds + +RH-Author: Fam Zheng +Message-id: <20181009081651.15463-6-famz@redhat.com> +Patchwork-id: 82449 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 5/5] aio-posix: do skip system call if ctx->notifier polling succeeds +Bugzilla: 1632622 +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula + +From: Paolo Bonzini + +BZ: 1632622 + +Commit 70232b5253 ("aio-posix: Don't count ctx->notifier as progress when +2018-08-15), by not reporting progress, causes aio_poll to execute the +system call when polling succeeds because of ctx->notifier. This introduces +latency before the call to aio_bh_poll() and negates the advantages of +polling, unfortunately. + +The fix builds on the previous patch, separating the effect of polling on +the timeout from the progress reported to aio_poll(). ctx->notifier +does zero the timeout, causing the caller to skip the system call, +but it does not report progress, so that the bug fix of commit 70232b5253 +still stands. + +Fixes: 70232b5253a3c4e03ed1ac47ef9246a8ac66c6fa +Signed-off-by: Paolo Bonzini +Message-Id: <20180912171040.1732-4-pbonzini@redhat.com> +Reviewed-by: Fam Zheng +Signed-off-by: Fam Zheng +(cherry picked from commit cfeb35d6774b2e936046aa9923217818bd160299) +Signed-off-by: Fam Zheng +Signed-off-by: Danilo C. L. de Paula +--- + util/aio-posix.c | 7 ++++--- + 1 file changed, 4 insertions(+), 3 deletions(-) + +diff --git a/util/aio-posix.c b/util/aio-posix.c +index 1b17597..357de8a 100644 +--- a/util/aio-posix.c ++++ b/util/aio-posix.c +@@ -498,10 +498,11 @@ static bool run_poll_handlers_once(AioContext *ctx, int64_t *timeout) + QLIST_FOREACH_RCU(node, &ctx->aio_handlers, node) { + if (!node->deleted && node->io_poll && + aio_node_check(ctx, node->is_external) && +- node->io_poll(node->opaque) && +- node->opaque != &ctx->notifier) { ++ node->io_poll(node->opaque)) { + *timeout = 0; +- progress = true; ++ if (node->opaque != &ctx->notifier) { ++ progress = true; ++ } + } + + /* Caller handles freeing deleted nodes. Don't do it here. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-aio-posix-fix-concurrent-access-to-poll_disable_cnt.patch b/SOURCES/kvm-aio-posix-fix-concurrent-access-to-poll_disable_cnt.patch new file mode 100644 index 0000000..cb6ff2f --- /dev/null +++ b/SOURCES/kvm-aio-posix-fix-concurrent-access-to-poll_disable_cnt.patch @@ -0,0 +1,122 @@ +From 67669ce3a2c163c467df63abc90e77bd9a856d34 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Tue, 9 Oct 2018 08:16:49 +0100 +Subject: [PATCH 05/17] aio-posix: fix concurrent access to poll_disable_cnt + +RH-Author: Fam Zheng +Message-id: <20181009081651.15463-4-famz@redhat.com> +Patchwork-id: 82452 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 3/5] aio-posix: fix concurrent access to poll_disable_cnt +Bugzilla: 1632622 +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula + +From: Paolo Bonzini + +BZ: 1632622 + +It is valid for an aio_set_fd_handler to happen concurrently with +aio_poll. In that case, poll_disable_cnt can change under the heels +of aio_poll, and the assertion on poll_disable_cnt can fail in +run_poll_handlers. + +Therefore, this patch simply checks the counter on every polling +iteration. There are no particular needs for ordering, since the +polling loop is terminated anyway by aio_notify at the end of +aio_set_fd_handler. + +Signed-off-by: Paolo Bonzini +Message-Id: <20180912171040.1732-2-pbonzini@redhat.com> +Reviewed-by: Fam Zheng +Signed-off-by: Fam Zheng +(cherry picked from commit d7be5dd19c0df7f76e1b42f0c2cbbabefa1974cb) +Signed-off-by: Fam Zheng +Signed-off-by: Danilo C. L. de Paula +--- + util/aio-posix.c | 26 +++++++++++++++----------- + 1 file changed, 15 insertions(+), 11 deletions(-) + +diff --git a/util/aio-posix.c b/util/aio-posix.c +index f05d3a8..1d7cc53 100644 +--- a/util/aio-posix.c ++++ b/util/aio-posix.c +@@ -211,6 +211,7 @@ void aio_set_fd_handler(AioContext *ctx, + AioHandler *node; + bool is_new = false; + bool deleted = false; ++ int poll_disable_change; + + qemu_lockcnt_lock(&ctx->list_lock); + +@@ -244,11 +245,9 @@ void aio_set_fd_handler(AioContext *ctx, + QLIST_REMOVE(node, node); + deleted = true; + } +- +- if (!node->io_poll) { +- ctx->poll_disable_cnt--; +- } ++ poll_disable_change = -!node->io_poll; + } else { ++ poll_disable_change = !io_poll - (node && !node->io_poll); + if (node == NULL) { + /* Alloc and insert if it's not already there */ + node = g_new0(AioHandler, 1); +@@ -257,10 +256,6 @@ void aio_set_fd_handler(AioContext *ctx, + + g_source_add_poll(&ctx->source, &node->pfd); + is_new = true; +- +- ctx->poll_disable_cnt += !io_poll; +- } else { +- ctx->poll_disable_cnt += !io_poll - !node->io_poll; + } + + /* Update handler with latest information */ +@@ -274,6 +269,15 @@ void aio_set_fd_handler(AioContext *ctx, + node->pfd.events |= (io_write ? G_IO_OUT | G_IO_ERR : 0); + } + ++ /* No need to order poll_disable_cnt writes against other updates; ++ * the counter is only used to avoid wasting time and latency on ++ * iterated polling when the system call will be ultimately necessary. ++ * Changing handlers is a rare event, and a little wasted polling until ++ * the aio_notify below is not an issue. ++ */ ++ atomic_set(&ctx->poll_disable_cnt, ++ atomic_read(&ctx->poll_disable_cnt) + poll_disable_change); ++ + aio_epoll_update(ctx, node, is_new); + qemu_lockcnt_unlock(&ctx->list_lock); + aio_notify(ctx); +@@ -525,7 +529,6 @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns) + + assert(ctx->notify_me); + assert(qemu_lockcnt_count(&ctx->list_lock) > 0); +- assert(ctx->poll_disable_cnt == 0); + + trace_run_poll_handlers_begin(ctx, max_ns); + +@@ -533,7 +536,8 @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns) + + do { + progress = run_poll_handlers_once(ctx); +- } while (!progress && qemu_clock_get_ns(QEMU_CLOCK_REALTIME) < end_time); ++ } while (!progress && qemu_clock_get_ns(QEMU_CLOCK_REALTIME) < end_time ++ && !atomic_read(&ctx->poll_disable_cnt)); + + trace_run_poll_handlers_end(ctx, progress); + +@@ -552,7 +556,7 @@ static bool run_poll_handlers(AioContext *ctx, int64_t max_ns) + */ + static bool try_poll_mode(AioContext *ctx, bool blocking) + { +- if (blocking && ctx->poll_max_ns && ctx->poll_disable_cnt == 0) { ++ if (blocking && ctx->poll_max_ns && !atomic_read(&ctx->poll_disable_cnt)) { + /* See qemu_soonest_timeout() uint64_t hack */ + int64_t max_ns = MIN((uint64_t)aio_compute_timeout(ctx), + (uint64_t)ctx->poll_ns); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-aio-wait-Increase-num_waiters-even-in-home-thread.patch b/SOURCES/kvm-aio-wait-Increase-num_waiters-even-in-home-thread.patch new file mode 100644 index 0000000..b09527e --- /dev/null +++ b/SOURCES/kvm-aio-wait-Increase-num_waiters-even-in-home-thread.patch @@ -0,0 +1,64 @@ +From 6d0923cbc54fc8eb983f433ca34a09e2a9982035 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:57 +0100 +Subject: [PATCH 31/49] aio-wait: Increase num_waiters even in home thread + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-19-kwolf@redhat.com> +Patchwork-id: 82609 +O-Subject: [RHEL-8 qemu-kvm PATCH 28/44] aio-wait: Increase num_waiters even in home thread +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Even if AIO_WAIT_WHILE() is called in the home context of the +AioContext, we still want to allow the condition to change depending on +other threads as long as they kick the AioWait. Specfically block jobs +can be running in an I/O thread and should then be able to kick a drain +in the main loop context. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +(cherry picked from commit 486574483aba988c83b20e7d3f1ccd50c4c333d8) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + include/block/aio-wait.h | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h +index c85a62f..600fad1 100644 +--- a/include/block/aio-wait.h ++++ b/include/block/aio-wait.h +@@ -76,6 +76,8 @@ typedef struct { + bool waited_ = false; \ + AioWait *wait_ = (wait); \ + AioContext *ctx_ = (ctx); \ ++ /* Increment wait_->num_waiters before evaluating cond. */ \ ++ atomic_inc(&wait_->num_waiters); \ + if (ctx_ && in_aio_context_home_thread(ctx_)) { \ + while ((cond)) { \ + aio_poll(ctx_, true); \ +@@ -84,8 +86,6 @@ typedef struct { + } else { \ + assert(qemu_get_current_aio_context() == \ + qemu_get_aio_context()); \ +- /* Increment wait_->num_waiters before evaluating cond. */ \ +- atomic_inc(&wait_->num_waiters); \ + while ((cond)) { \ + if (ctx_) { \ + aio_context_release(ctx_); \ +@@ -96,8 +96,8 @@ typedef struct { + } \ + waited_ = true; \ + } \ +- atomic_dec(&wait_->num_waiters); \ + } \ ++ atomic_dec(&wait_->num_waiters); \ + waited_; }) + + /** +-- +1.8.3.1 + diff --git a/SOURCES/kvm-backup-Use-copy-offloading.patch b/SOURCES/kvm-backup-Use-copy-offloading.patch new file mode 100644 index 0000000..231de7f --- /dev/null +++ b/SOURCES/kvm-backup-Use-copy-offloading.patch @@ -0,0 +1,260 @@ +From 1df22a2aa1fc70f0b2b268bbe0c184d95ce74b04 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:52 +0200 +Subject: [PATCH 234/268] backup: Use copy offloading + +RH-Author: John Snow +Message-id: <20180718225511.14878-17-jsnow@redhat.com> +Patchwork-id: 81399 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 16/35] backup: Use copy offloading +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Fam Zheng + +The implementation is similar to the 'qemu-img convert'. In the +beginning of the job, offloaded copy is attempted. If it fails, further +I/O will go through the existing bounce buffer code path. + +Then, as Kevin pointed out, both this and qemu-img convert can benefit +from a local check if one request fails because of, for example, the +offset is beyond EOF, but another may well be accepted by the protocol +layer. This will be implemented separately. + +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Fam Zheng +Message-id: 20180703023758.14422-4-famz@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit 9ded4a0114968e98b41494fc035ba14f84cdf700) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 150 ++++++++++++++++++++++++++++++++++++++--------------- + block/trace-events | 1 + + 2 files changed, 110 insertions(+), 41 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 5661435..d26eeb5 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -45,6 +45,8 @@ typedef struct BackupBlockJob { + QLIST_HEAD(, CowRequest) inflight_reqs; + + HBitmap *copy_bitmap; ++ bool use_copy_range; ++ int64_t copy_range_size; + } BackupBlockJob; + + static const BlockJobDriver backup_job_driver; +@@ -86,19 +88,101 @@ static void cow_request_end(CowRequest *req) + qemu_co_queue_restart_all(&req->wait_queue); + } + ++/* Copy range to target with a bounce buffer and return the bytes copied. If ++ * error occured, return a negative error number */ ++static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, ++ int64_t start, ++ int64_t end, ++ bool is_write_notifier, ++ bool *error_is_read, ++ void **bounce_buffer) ++{ ++ int ret; ++ struct iovec iov; ++ QEMUIOVector qiov; ++ BlockBackend *blk = job->common.blk; ++ int nbytes; ++ ++ hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); ++ nbytes = MIN(job->cluster_size, job->len - start); ++ if (!*bounce_buffer) { ++ *bounce_buffer = blk_blockalign(blk, job->cluster_size); ++ } ++ iov.iov_base = *bounce_buffer; ++ iov.iov_len = nbytes; ++ qemu_iovec_init_external(&qiov, &iov, 1); ++ ++ ret = blk_co_preadv(blk, start, qiov.size, &qiov, ++ is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); ++ if (ret < 0) { ++ trace_backup_do_cow_read_fail(job, start, ret); ++ if (error_is_read) { ++ *error_is_read = true; ++ } ++ goto fail; ++ } ++ ++ if (qemu_iovec_is_zero(&qiov)) { ++ ret = blk_co_pwrite_zeroes(job->target, start, ++ qiov.size, BDRV_REQ_MAY_UNMAP); ++ } else { ++ ret = blk_co_pwritev(job->target, start, ++ qiov.size, &qiov, ++ job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); ++ } ++ if (ret < 0) { ++ trace_backup_do_cow_write_fail(job, start, ret); ++ if (error_is_read) { ++ *error_is_read = false; ++ } ++ goto fail; ++ } ++ ++ return nbytes; ++fail: ++ hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); ++ return ret; ++ ++} ++ ++/* Copy range to target and return the bytes copied. If error occured, return a ++ * negative error number. */ ++static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, ++ int64_t start, ++ int64_t end, ++ bool is_write_notifier) ++{ ++ int ret; ++ int nr_clusters; ++ BlockBackend *blk = job->common.blk; ++ int nbytes; ++ ++ assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); ++ nbytes = MIN(job->copy_range_size, end - start); ++ nr_clusters = DIV_ROUND_UP(nbytes, job->cluster_size); ++ hbitmap_reset(job->copy_bitmap, start / job->cluster_size, ++ nr_clusters); ++ ret = blk_co_copy_range(blk, start, job->target, start, nbytes, ++ is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); ++ if (ret < 0) { ++ trace_backup_do_cow_copy_range_fail(job, start, ret); ++ hbitmap_set(job->copy_bitmap, start / job->cluster_size, ++ nr_clusters); ++ return ret; ++ } ++ ++ return nbytes; ++} ++ + static int coroutine_fn backup_do_cow(BackupBlockJob *job, + int64_t offset, uint64_t bytes, + bool *error_is_read, + bool is_write_notifier) + { +- BlockBackend *blk = job->common.blk; + CowRequest cow_request; +- struct iovec iov; +- QEMUIOVector bounce_qiov; +- void *bounce_buffer = NULL; + int ret = 0; + int64_t start, end; /* bytes */ +- int n; /* bytes */ ++ void *bounce_buffer = NULL; + + qemu_co_rwlock_rdlock(&job->flush_rwlock); + +@@ -110,60 +194,38 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, + wait_for_overlapping_requests(job, start, end); + cow_request_begin(&cow_request, job, start, end); + +- for (; start < end; start += job->cluster_size) { ++ while (start < end) { + if (!hbitmap_get(job->copy_bitmap, start / job->cluster_size)) { + trace_backup_do_cow_skip(job, start); ++ start += job->cluster_size; + continue; /* already copied */ + } +- hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); + + trace_backup_do_cow_process(job, start); + +- n = MIN(job->cluster_size, job->len - start); +- +- if (!bounce_buffer) { +- bounce_buffer = blk_blockalign(blk, job->cluster_size); +- } +- iov.iov_base = bounce_buffer; +- iov.iov_len = n; +- qemu_iovec_init_external(&bounce_qiov, &iov, 1); +- +- ret = blk_co_preadv(blk, start, bounce_qiov.size, &bounce_qiov, +- is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); +- if (ret < 0) { +- trace_backup_do_cow_read_fail(job, start, ret); +- if (error_is_read) { +- *error_is_read = true; ++ if (job->use_copy_range) { ++ ret = backup_cow_with_offload(job, start, end, is_write_notifier); ++ if (ret < 0) { ++ job->use_copy_range = false; + } +- hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); +- goto out; + } +- +- if (buffer_is_zero(iov.iov_base, iov.iov_len)) { +- ret = blk_co_pwrite_zeroes(job->target, start, +- bounce_qiov.size, BDRV_REQ_MAY_UNMAP); +- } else { +- ret = blk_co_pwritev(job->target, start, +- bounce_qiov.size, &bounce_qiov, +- job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); ++ if (!job->use_copy_range) { ++ ret = backup_cow_with_bounce_buffer(job, start, end, is_write_notifier, ++ error_is_read, &bounce_buffer); + } + if (ret < 0) { +- trace_backup_do_cow_write_fail(job, start, ret); +- if (error_is_read) { +- *error_is_read = false; +- } +- hbitmap_set(job->copy_bitmap, start / job->cluster_size, 1); +- goto out; ++ break; + } + + /* Publish progress, guest I/O counts as progress too. Note that the + * offset field is an opaque progress value, it is not a disk offset. + */ +- job->bytes_read += n; +- job_progress_update(&job->common.job, n); ++ start += ret; ++ job->bytes_read += ret; ++ job_progress_update(&job->common.job, ret); ++ ret = 0; + } + +-out: + if (bounce_buffer) { + qemu_vfree(bounce_buffer); + } +@@ -665,6 +727,12 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + } else { + job->cluster_size = MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); + } ++ job->use_copy_range = true; ++ job->copy_range_size = MIN_NON_ZERO(blk_get_max_transfer(job->common.blk), ++ blk_get_max_transfer(job->target)); ++ job->copy_range_size = MAX(job->cluster_size, ++ QEMU_ALIGN_UP(job->copy_range_size, ++ job->cluster_size)); + + /* Required permissions are already taken with target's blk_new() */ + block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, +diff --git a/block/trace-events b/block/trace-events +index 2d59b53..c35287b 100644 +--- a/block/trace-events ++++ b/block/trace-events +@@ -42,6 +42,7 @@ backup_do_cow_skip(void *job, int64_t start) "job %p start %"PRId64 + backup_do_cow_process(void *job, int64_t start) "job %p start %"PRId64 + backup_do_cow_read_fail(void *job, int64_t start, int ret) "job %p start %"PRId64" ret %d" + backup_do_cow_write_fail(void *job, int64_t start, int ret) "job %p start %"PRId64" ret %d" ++backup_do_cow_copy_range_fail(void *job, int64_t start, int ret) "job %p start %"PRId64" ret %d" + + # blockdev.c + qmp_block_job_cancel(void *job) "job %p" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-balloon-Allow-multiple-inhibit-users.patch b/SOURCES/kvm-balloon-Allow-multiple-inhibit-users.patch new file mode 100644 index 0000000..febed46 --- /dev/null +++ b/SOURCES/kvm-balloon-Allow-multiple-inhibit-users.patch @@ -0,0 +1,72 @@ +From 51b3550c677efeb39b53d6cfe90a2b9798be707b Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:01:10 +0000 +Subject: [PATCH 10/16] balloon: Allow multiple inhibit users + +RH-Author: Alex Williamson +Message-id: <154387447040.27651.8134042757905328573.stgit@gimli.home> +Patchwork-id: 83235 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 1/7] balloon: Allow multiple inhibit users +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +A simple true/false internal state does not allow multiple users. Fix +this within the existing interface by converting to a counter, so long +as the counter is elevated, ballooning is inhibited. + +Reviewed-by: David Hildenbrand +Reviewed-by: Peter Xu +Reviewed-by: Cornelia Huck +Signed-off-by: Alex Williamson +(cherry picked from commit 01ccbec7bdf6f89f1b7d46dda05e4c1fd2dd5ade) +Signed-off-by: Danilo C. L. de Paula +--- + balloon.c | 13 ++++++++++--- + 1 file changed, 10 insertions(+), 3 deletions(-) + +diff --git a/balloon.c b/balloon.c +index 6bf0a96..9319879 100644 +--- a/balloon.c ++++ b/balloon.c +@@ -26,6 +26,7 @@ + + #include "qemu/osdep.h" + #include "qemu-common.h" ++#include "qemu/atomic.h" + #include "exec/cpu-common.h" + #include "sysemu/kvm.h" + #include "sysemu/balloon.h" +@@ -37,16 +38,22 @@ + static QEMUBalloonEvent *balloon_event_fn; + static QEMUBalloonStatus *balloon_stat_fn; + static void *balloon_opaque; +-static bool balloon_inhibited; ++static int balloon_inhibit_count; + + bool qemu_balloon_is_inhibited(void) + { +- return balloon_inhibited; ++ return atomic_read(&balloon_inhibit_count) > 0; + } + + void qemu_balloon_inhibit(bool state) + { +- balloon_inhibited = state; ++ if (state) { ++ atomic_inc(&balloon_inhibit_count); ++ } else { ++ atomic_dec(&balloon_inhibit_count); ++ } ++ ++ assert(atomic_read(&balloon_inhibit_count) >= 0); + } + + static bool have_balloon(Error **errp) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-bitmap-Update-count-after-a-merge.patch b/SOURCES/kvm-bitmap-Update-count-after-a-merge.patch new file mode 100644 index 0000000..6e4771f --- /dev/null +++ b/SOURCES/kvm-bitmap-Update-count-after-a-merge.patch @@ -0,0 +1,56 @@ +From 6749a16ae2e3d5aa51012bed0d9a910be8de004e Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:22 +0000 +Subject: [PATCH 28/35] bitmap: Update count after a merge + +RH-Author: John Snow +Message-id: <20181120181828.15132-19-jsnow@redhat.com> +Patchwork-id: 83073 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 18/24] bitmap: Update count after a merge +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Eric Blake + +We need an accurate count of the number of bits set in a bitmap +after a merge. In particular, since the merge operation short-circuits +a merge from an empty source, if you have bitmaps A, B, and C where +B started empty, then merge C into B, and B into A, an inaccurate +count meant that A did not get the contents of C. + +In the worst case, we may falsely regard the bitmap as empty when +it has had new writes merged into it. + +Fixes: be58721db +CC: qemu-stable@nongnu.org +Signed-off-by: Eric Blake +Signed-off-by: John Snow +Reviewed-by: Vladimir Sementsov-Ogievskiy +Message-id: 20181002233314.30159-1-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit d1dde7149e376d72b422a529ec4bf3ed47f3ba30) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + util/hbitmap.c | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/util/hbitmap.c b/util/hbitmap.c +index d5aca51..8d402c5 100644 +--- a/util/hbitmap.c ++++ b/util/hbitmap.c +@@ -759,6 +759,9 @@ bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) + } + } + ++ /* Recompute the dirty count */ ++ result->count = hb_count_between(result, 0, result->size - 1); ++ + return true; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-bloc-qcow2-drop-dirty_bitmaps_loaded-state-variable.patch b/SOURCES/kvm-bloc-qcow2-drop-dirty_bitmaps_loaded-state-variable.patch new file mode 100644 index 0000000..99777c6 --- /dev/null +++ b/SOURCES/kvm-bloc-qcow2-drop-dirty_bitmaps_loaded-state-variable.patch @@ -0,0 +1,84 @@ +From 9672f88c8747e874efb9d8d6f67b8d4d8c474779 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:25 +0000 +Subject: [PATCH 31/35] bloc/qcow2: drop dirty_bitmaps_loaded state variable + +RH-Author: John Snow +Message-id: <20181120181828.15132-22-jsnow@redhat.com> +Patchwork-id: 83067 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 21/24] bloc/qcow2: drop dirty_bitmaps_loaded state variable +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +This variable doesn't work as it should, because it is actually cleared +in qcow2_co_invalidate_cache() by memset(). Drop it, as the following +patch will introduce new behavior. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +Signed-off-by: John Snow +(cherry picked from commit 2ea427effff61efa5d0dc69f9cae126d13879617) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 19 ++----------------- + block/qcow2.h | 1 - + 2 files changed, 2 insertions(+), 18 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 5c5530d..d260cd6 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1149,7 +1149,6 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, + uint64_t ext_end; + uint64_t l1_vm_state_index; + bool update_header = false; +- bool header_updated = false; + + ret = bdrv_pread(bs->file, 0, &header, sizeof(header)); + if (ret < 0) { +@@ -1488,23 +1487,9 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, + s->autoclear_features &= QCOW2_AUTOCLEAR_MASK; + } + +- if (s->dirty_bitmaps_loaded) { +- /* It's some kind of reopen. There are no known cases where we need to +- * reload bitmaps in such a situation, so it's safer to skip them. +- * +- * Moreover, if we have some readonly bitmaps and we are reopening for +- * rw we should reopen bitmaps correspondingly. +- */ +- if (bdrv_has_readonly_bitmaps(bs) && +- !bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE)) +- { +- qcow2_reopen_bitmaps_rw_hint(bs, &header_updated, &local_err); +- } +- } else { +- header_updated = qcow2_load_dirty_bitmaps(bs, &local_err); +- s->dirty_bitmaps_loaded = true; ++ if (qcow2_load_dirty_bitmaps(bs, &local_err)) { ++ update_header = false; + } +- update_header = update_header && !header_updated; + if (local_err != NULL) { + error_propagate(errp, local_err); + ret = -EINVAL; +diff --git a/block/qcow2.h b/block/qcow2.h +index d2c63e4..43163b2 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -299,7 +299,6 @@ typedef struct BDRVQcow2State { + uint32_t nb_bitmaps; + uint64_t bitmap_directory_size; + uint64_t bitmap_directory_offset; +- bool dirty_bitmaps_loaded; + + int flags; + int qcow_version; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Add-BDRV_REQ_WRITE_UNCHANGED-flag.patch b/SOURCES/kvm-block-Add-BDRV_REQ_WRITE_UNCHANGED-flag.patch new file mode 100644 index 0000000..078262d --- /dev/null +++ b/SOURCES/kvm-block-Add-BDRV_REQ_WRITE_UNCHANGED-flag.patch @@ -0,0 +1,70 @@ +From b48a7b70caf3883782f8e90fb58b7f46ef6ace88 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:06 +0200 +Subject: [PATCH 039/268] block: Add BDRV_REQ_WRITE_UNCHANGED flag + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-5-mreitz@redhat.com> +Patchwork-id: 80764 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 04/10] block: Add BDRV_REQ_WRITE_UNCHANGED flag +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +This flag signifies that a write request will not change the visible +disk content. With this flag set, it is sufficient to have the +BLK_PERM_WRITE_UNCHANGED permission instead of BLK_PERM_WRITE. + +Signed-off-by: Max Reitz +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Alberto Garcia +Message-id: 20180421132929.21610-4-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit c6035964f8316b504060618d05b5dd434f18595b) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 6 +++++- + include/block/block.h | 6 +++++- + 2 files changed, 10 insertions(+), 2 deletions(-) + +diff --git a/block/io.c b/block/io.c +index bd9a19a..134b2a4 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -1501,7 +1501,11 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child, + assert(!waited || !req->serialising); + assert(req->overlap_offset <= offset); + assert(offset + bytes <= req->overlap_offset + req->overlap_bytes); +- assert(child->perm & BLK_PERM_WRITE); ++ if (flags & BDRV_REQ_WRITE_UNCHANGED) { ++ assert(child->perm & (BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE)); ++ } else { ++ assert(child->perm & BLK_PERM_WRITE); ++ } + assert(end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE); + + ret = notifier_with_return_list_notify(&bs->before_write_notifiers, req); +diff --git a/include/block/block.h b/include/block/block.h +index 397b5e8..3894edd 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -54,8 +54,12 @@ typedef enum { + BDRV_REQ_FUA = 0x10, + BDRV_REQ_WRITE_COMPRESSED = 0x20, + ++ /* Signifies that this write request will not change the visible disk ++ * content. */ ++ BDRV_REQ_WRITE_UNCHANGED = 0x40, ++ + /* Mask of valid flags */ +- BDRV_REQ_MASK = 0x3f, ++ BDRV_REQ_MASK = 0x7f, + } BdrvRequestFlags; + + typedef struct BlockSizes { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Add-COR-filter-driver.patch b/SOURCES/kvm-block-Add-COR-filter-driver.patch new file mode 100644 index 0000000..06fdfce --- /dev/null +++ b/SOURCES/kvm-block-Add-COR-filter-driver.patch @@ -0,0 +1,260 @@ +From b4442d3284353b0ad54eacbe7e29c8d09dbcf301 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:04 +0200 +Subject: [PATCH 037/268] block: Add COR filter driver + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-3-mreitz@redhat.com> +Patchwork-id: 80762 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 02/10] block: Add COR filter driver +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +This adds a simple copy-on-read filter driver. It relies on the already +existing COR functionality in the central block layer code, which may be +moved here once we no longer need it there. + +Signed-off-by: Max Reitz +Message-id: 20180421132929.21610-2-mreitz@redhat.com +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 6c6f24fd84895d03baa898bbc4324dd4ccc97071) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/Makefile.objs | 2 +- + block/copy-on-read.c | 171 +++++++++++++++++++++++++++++++++++++++++++++++++++ + qapi/block-core.json | 5 +- + 3 files changed, 176 insertions(+), 2 deletions(-) + create mode 100644 block/copy-on-read.c + +diff --git a/block/Makefile.objs b/block/Makefile.objs +index c0693fc..be2cda1 100644 +--- a/block/Makefile.objs ++++ b/block/Makefile.objs +@@ -26,7 +26,7 @@ block-obj-y += accounting.o dirty-bitmap.o + block-obj-y += write-threshold.o + block-obj-y += backup.o + block-obj-$(CONFIG_REPLICATION) += replication.o +-block-obj-y += throttle.o ++block-obj-y += throttle.o copy-on-read.o + + block-obj-y += crypto.o + +diff --git a/block/copy-on-read.c b/block/copy-on-read.c +new file mode 100644 +index 0000000..823ec75 +--- /dev/null ++++ b/block/copy-on-read.c +@@ -0,0 +1,171 @@ ++/* ++ * Copy-on-read filter block driver ++ * ++ * Copyright (c) 2018 Red Hat, Inc. ++ * ++ * Author: ++ * Max Reitz ++ * ++ * This program is free software; you can redistribute it and/or ++ * modify it under the terms of the GNU General Public License as ++ * published by the Free Software Foundation; either version 2 or ++ * (at your option) version 3 of the License. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, see . ++ */ ++ ++#include "qemu/osdep.h" ++#include "block/block_int.h" ++ ++ ++static int cor_open(BlockDriverState *bs, QDict *options, int flags, ++ Error **errp) ++{ ++ bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, false, ++ errp); ++ if (!bs->file) { ++ return -EINVAL; ++ } ++ ++ bs->supported_write_flags = BDRV_REQ_FUA & ++ bs->file->bs->supported_write_flags; ++ ++ bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & ++ bs->file->bs->supported_zero_flags; ++ ++ return 0; ++} ++ ++ ++static void cor_close(BlockDriverState *bs) ++{ ++} ++ ++ ++#define PERM_PASSTHROUGH (BLK_PERM_CONSISTENT_READ \ ++ | BLK_PERM_WRITE \ ++ | BLK_PERM_RESIZE) ++#define PERM_UNCHANGED (BLK_PERM_ALL & ~PERM_PASSTHROUGH) ++ ++static void cor_child_perm(BlockDriverState *bs, BdrvChild *c, ++ const BdrvChildRole *role, ++ BlockReopenQueue *reopen_queue, ++ uint64_t perm, uint64_t shared, ++ uint64_t *nperm, uint64_t *nshared) ++{ ++ if (c == NULL) { ++ *nperm = (perm & PERM_PASSTHROUGH) | BLK_PERM_WRITE_UNCHANGED; ++ *nshared = (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED; ++ return; ++ } ++ ++ *nperm = (perm & PERM_PASSTHROUGH) | ++ (c->perm & PERM_UNCHANGED); ++ *nshared = (shared & PERM_PASSTHROUGH) | ++ (c->shared_perm & PERM_UNCHANGED); ++} ++ ++ ++static int64_t cor_getlength(BlockDriverState *bs) ++{ ++ return bdrv_getlength(bs->file->bs); ++} ++ ++ ++static int cor_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) ++{ ++ return bdrv_truncate(bs->file, offset, prealloc, errp); ++} ++ ++ ++static int coroutine_fn cor_co_preadv(BlockDriverState *bs, ++ uint64_t offset, uint64_t bytes, ++ QEMUIOVector *qiov, int flags) ++{ ++ return bdrv_co_preadv(bs->file, offset, bytes, qiov, ++ flags | BDRV_REQ_COPY_ON_READ); ++} ++ ++ ++static int coroutine_fn cor_co_pwritev(BlockDriverState *bs, ++ uint64_t offset, uint64_t bytes, ++ QEMUIOVector *qiov, int flags) ++{ ++ ++ return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); ++} ++ ++ ++static int coroutine_fn cor_co_pwrite_zeroes(BlockDriverState *bs, ++ int64_t offset, int bytes, ++ BdrvRequestFlags flags) ++{ ++ return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); ++} ++ ++ ++static int coroutine_fn cor_co_pdiscard(BlockDriverState *bs, ++ int64_t offset, int bytes) ++{ ++ return bdrv_co_pdiscard(bs->file->bs, offset, bytes); ++} ++ ++ ++static void cor_eject(BlockDriverState *bs, bool eject_flag) ++{ ++ bdrv_eject(bs->file->bs, eject_flag); ++} ++ ++ ++static void cor_lock_medium(BlockDriverState *bs, bool locked) ++{ ++ bdrv_lock_medium(bs->file->bs, locked); ++} ++ ++ ++static bool cor_recurse_is_first_non_filter(BlockDriverState *bs, ++ BlockDriverState *candidate) ++{ ++ return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate); ++} ++ ++ ++BlockDriver bdrv_copy_on_read = { ++ .format_name = "copy-on-read", ++ ++ .bdrv_open = cor_open, ++ .bdrv_close = cor_close, ++ .bdrv_child_perm = cor_child_perm, ++ ++ .bdrv_getlength = cor_getlength, ++ .bdrv_truncate = cor_truncate, ++ ++ .bdrv_co_preadv = cor_co_preadv, ++ .bdrv_co_pwritev = cor_co_pwritev, ++ .bdrv_co_pwrite_zeroes = cor_co_pwrite_zeroes, ++ .bdrv_co_pdiscard = cor_co_pdiscard, ++ ++ .bdrv_eject = cor_eject, ++ .bdrv_lock_medium = cor_lock_medium, ++ ++ .bdrv_co_block_status = bdrv_co_block_status_from_file, ++ ++ .bdrv_recurse_is_first_non_filter = cor_recurse_is_first_non_filter, ++ ++ .has_variable_length = true, ++ .is_filter = true, ++}; ++ ++static void bdrv_copy_on_read_init(void) ++{ ++ bdrv_register(&bdrv_copy_on_read); ++} ++ ++block_init(bdrv_copy_on_read_init); +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 51eafdd..9e4f1ac 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -2506,11 +2506,12 @@ + # @vxhs: Since 2.10 + # @throttle: Since 2.11 + # @nvme: Since 2.12 ++# @copy-on-read: Since 2.13 + # + # Since: 2.9 + ## + { 'enum': 'BlockdevDriver', +- 'data': [ 'blkdebug', 'blkverify', 'bochs', 'cloop', ++ 'data': [ 'blkdebug', 'blkverify', 'bochs', 'cloop', 'copy-on-read', + 'dmg', 'file', 'ftp', 'ftps', 'gluster', 'host_cdrom', + 'host_device', 'http', 'https', 'iscsi', 'luks', 'nbd', 'nfs', + 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow', 'qcow2', 'qed', +@@ -3541,6 +3542,7 @@ + 'blkverify': 'BlockdevOptionsBlkverify', + 'bochs': 'BlockdevOptionsGenericFormat', + 'cloop': 'BlockdevOptionsGenericFormat', ++ 'copy-on-read':'BlockdevOptionsGenericFormat', + 'dmg': 'BlockdevOptionsGenericFormat', + 'file': 'BlockdevOptionsFile', + 'ftp': 'BlockdevOptionsCurlFtp', +@@ -4068,6 +4070,7 @@ + 'blkverify': 'BlockdevCreateNotSupported', + 'bochs': 'BlockdevCreateNotSupported', + 'cloop': 'BlockdevCreateNotSupported', ++ 'copy-on-read': 'BlockdevCreateNotSupported', + 'dmg': 'BlockdevCreateNotSupported', + 'file': 'BlockdevCreateOptionsFile', + 'ftp': 'BlockdevCreateNotSupported', +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Add-Error-parameter-to-bdrv_amend_options.patch b/SOURCES/kvm-block-Add-Error-parameter-to-bdrv_amend_options.patch new file mode 100644 index 0000000..200dbdf --- /dev/null +++ b/SOURCES/kvm-block-Add-Error-parameter-to-bdrv_amend_options.patch @@ -0,0 +1,465 @@ +From 26f46c414ca053bfe87cebd9d40107fc4fa6bc1b Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:59:38 +0200 +Subject: [PATCH 065/268] block: Add Error parameter to bdrv_amend_options + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-3-mreitz@redhat.com> +Patchwork-id: 80755 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/7] block: Add Error parameter to bdrv_amend_options +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +Looking at the qcow2 code that is riddled with error_report() calls, +this is really how it should have been from the start. + +Along the way, turn the target_version/current_version comparisons at +the beginning of qcow2_downgrade() into assertions (the caller has to +make sure these conditions are met), and rephrase the error message on +using compat=1.1 to get refcount widths other than 16 bits. + +Signed-off-by: Max Reitz +Message-id: 20180509210023.20283-3-mreitz@redhat.com +Reviewed-by: Eric Blake +Reviewed-by: John Snow +Signed-off-by: Max Reitz +(cherry picked from commit d1402b502691142b9cebadd5cb993dc8858e9071) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block.c | 8 ++++-- + block/qcow2.c | 72 ++++++++++++++++++++++++++-------------------- + include/block/block.h | 3 +- + include/block/block_int.h | 3 +- + qemu-img.c | 4 +-- + tests/qemu-iotests/060.out | 4 +-- + tests/qemu-iotests/061.out | 7 ----- + tests/qemu-iotests/080.out | 4 +-- + tests/qemu-iotests/112.out | 5 +--- + 9 files changed, 58 insertions(+), 52 deletions(-) + +diff --git a/block.c b/block.c +index 982d54e..d991a09 100644 +--- a/block.c ++++ b/block.c +@@ -5008,15 +5008,19 @@ void bdrv_remove_aio_context_notifier(BlockDriverState *bs, + } + + int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts, +- BlockDriverAmendStatusCB *status_cb, void *cb_opaque) ++ BlockDriverAmendStatusCB *status_cb, void *cb_opaque, ++ Error **errp) + { + if (!bs->drv) { ++ error_setg(errp, "Node is ejected"); + return -ENOMEDIUM; + } + if (!bs->drv->bdrv_amend_options) { ++ error_setg(errp, "Block driver '%s' does not support option amendment", ++ bs->drv->format_name); + return -ENOTSUP; + } +- return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque); ++ return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque, errp); + } + + /* This function will be called by the bdrv_recurse_is_first_non_filter method +diff --git a/block/qcow2.c b/block/qcow2.c +index 26a6a7f..092db81 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -4039,22 +4039,21 @@ static int qcow2_load_vmstate(BlockDriverState *bs, QEMUIOVector *qiov, + * have to be removed. + */ + static int qcow2_downgrade(BlockDriverState *bs, int target_version, +- BlockDriverAmendStatusCB *status_cb, void *cb_opaque) ++ BlockDriverAmendStatusCB *status_cb, void *cb_opaque, ++ Error **errp) + { + BDRVQcow2State *s = bs->opaque; + int current_version = s->qcow_version; + int ret; + +- if (target_version == current_version) { +- return 0; +- } else if (target_version > current_version) { +- return -EINVAL; +- } else if (target_version != 2) { +- return -EINVAL; +- } ++ /* This is qcow2_downgrade(), not qcow2_upgrade() */ ++ assert(target_version < current_version); ++ ++ /* There are no other versions (now) that you can downgrade to */ ++ assert(target_version == 2); + + if (s->refcount_order != 4) { +- error_report("compat=0.10 requires refcount_bits=16"); ++ error_setg(errp, "compat=0.10 requires refcount_bits=16"); + return -ENOTSUP; + } + +@@ -4062,6 +4061,7 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, + if (s->incompatible_features & QCOW2_INCOMPAT_DIRTY) { + ret = qcow2_mark_clean(bs); + if (ret < 0) { ++ error_setg_errno(errp, -ret, "Failed to make the image clean"); + return ret; + } + } +@@ -4071,6 +4071,8 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, + * best thing to do anyway */ + + if (s->incompatible_features) { ++ error_setg(errp, "Cannot downgrade an image with incompatible features " ++ "%#" PRIx64 " set", s->incompatible_features); + return -ENOTSUP; + } + +@@ -4084,6 +4086,7 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, + + ret = qcow2_expand_zero_clusters(bs, status_cb, cb_opaque); + if (ret < 0) { ++ error_setg_errno(errp, -ret, "Failed to turn zero into data clusters"); + return ret; + } + +@@ -4091,6 +4094,7 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version, + ret = qcow2_update_header(bs); + if (ret < 0) { + s->qcow_version = current_version; ++ error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + return 0; +@@ -4168,7 +4172,8 @@ static void qcow2_amend_helper_cb(BlockDriverState *bs, + + static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + BlockDriverAmendStatusCB *status_cb, +- void *cb_opaque) ++ void *cb_opaque, ++ Error **errp) + { + BDRVQcow2State *s = bs->opaque; + int old_version = s->qcow_version, new_version = old_version; +@@ -4180,7 +4185,6 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + bool encrypt; + int encformat; + int refcount_bits = s->refcount_bits; +- Error *local_err = NULL; + int ret; + QemuOptDesc *desc = opts->list->desc; + Qcow2AmendHelperCBInfo helper_cb_info; +@@ -4201,11 +4205,11 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + } else if (!strcmp(compat, "1.1")) { + new_version = 3; + } else { +- error_report("Unknown compatibility level %s", compat); ++ error_setg(errp, "Unknown compatibility level %s", compat); + return -EINVAL; + } + } else if (!strcmp(desc->name, BLOCK_OPT_PREALLOC)) { +- error_report("Cannot change preallocation mode"); ++ error_setg(errp, "Cannot change preallocation mode"); + return -ENOTSUP; + } else if (!strcmp(desc->name, BLOCK_OPT_SIZE)) { + new_size = qemu_opt_get_size(opts, BLOCK_OPT_SIZE, 0); +@@ -4218,7 +4222,8 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + !!s->crypto); + + if (encrypt != !!s->crypto) { +- error_report("Changing the encryption flag is not supported"); ++ error_setg(errp, ++ "Changing the encryption flag is not supported"); + return -ENOTSUP; + } + } else if (!strcmp(desc->name, BLOCK_OPT_ENCRYPT_FORMAT)) { +@@ -4226,17 +4231,19 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + qemu_opt_get(opts, BLOCK_OPT_ENCRYPT_FORMAT)); + + if (encformat != s->crypt_method_header) { +- error_report("Changing the encryption format is not supported"); ++ error_setg(errp, ++ "Changing the encryption format is not supported"); + return -ENOTSUP; + } + } else if (g_str_has_prefix(desc->name, "encrypt.")) { +- error_report("Changing the encryption parameters is not supported"); ++ error_setg(errp, ++ "Changing the encryption parameters is not supported"); + return -ENOTSUP; + } else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) { + cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE, + cluster_size); + if (cluster_size != s->cluster_size) { +- error_report("Changing the cluster size is not supported"); ++ error_setg(errp, "Changing the cluster size is not supported"); + return -ENOTSUP; + } + } else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) { +@@ -4249,8 +4256,8 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + if (refcount_bits <= 0 || refcount_bits > 64 || + !is_power_of_2(refcount_bits)) + { +- error_report("Refcount width must be a power of two and may " +- "not exceed 64 bits"); ++ error_setg(errp, "Refcount width must be a power of two and " ++ "may not exceed 64 bits"); + return -EINVAL; + } + } else { +@@ -4275,6 +4282,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + ret = qcow2_update_header(bs); + if (ret < 0) { + s->qcow_version = old_version; ++ error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + } +@@ -4283,18 +4291,17 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + int refcount_order = ctz32(refcount_bits); + + if (new_version < 3 && refcount_bits != 16) { +- error_report("Different refcount widths than 16 bits require " +- "compatibility level 1.1 or above (use compat=1.1 or " +- "greater)"); ++ error_setg(errp, "Refcount widths other than 16 bits require " ++ "compatibility level 1.1 or above (use compat=1.1 or " ++ "greater)"); + return -EINVAL; + } + + helper_cb_info.current_operation = QCOW2_CHANGING_REFCOUNT_ORDER; + ret = qcow2_change_refcount_order(bs, refcount_order, + &qcow2_amend_helper_cb, +- &helper_cb_info, &local_err); ++ &helper_cb_info, errp); + if (ret < 0) { +- error_report_err(local_err); + return ret; + } + } +@@ -4304,6 +4311,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + backing_file ?: s->image_backing_file, + backing_format ?: s->image_backing_format); + if (ret < 0) { ++ error_setg_errno(errp, -ret, "Failed to change the backing file"); + return ret; + } + } +@@ -4311,14 +4319,16 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + if (s->use_lazy_refcounts != lazy_refcounts) { + if (lazy_refcounts) { + if (new_version < 3) { +- error_report("Lazy refcounts only supported with compatibility " +- "level 1.1 and above (use compat=1.1 or greater)"); ++ error_setg(errp, "Lazy refcounts only supported with " ++ "compatibility level 1.1 and above (use compat=1.1 " ++ "or greater)"); + return -EINVAL; + } + s->compatible_features |= QCOW2_COMPAT_LAZY_REFCOUNTS; + ret = qcow2_update_header(bs); + if (ret < 0) { + s->compatible_features &= ~QCOW2_COMPAT_LAZY_REFCOUNTS; ++ error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + s->use_lazy_refcounts = true; +@@ -4326,6 +4336,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + /* make image clean first */ + ret = qcow2_mark_clean(bs); + if (ret < 0) { ++ error_setg_errno(errp, -ret, "Failed to make the image clean"); + return ret; + } + /* now disallow lazy refcounts */ +@@ -4333,6 +4344,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + ret = qcow2_update_header(bs); + if (ret < 0) { + s->compatible_features |= QCOW2_COMPAT_LAZY_REFCOUNTS; ++ error_setg_errno(errp, -ret, "Failed to update the image header"); + return ret; + } + s->use_lazy_refcounts = false; +@@ -4341,17 +4353,15 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + + if (new_size) { + BlockBackend *blk = blk_new(BLK_PERM_RESIZE, BLK_PERM_ALL); +- ret = blk_insert_bs(blk, bs, &local_err); ++ ret = blk_insert_bs(blk, bs, errp); + if (ret < 0) { +- error_report_err(local_err); + blk_unref(blk); + return ret; + } + +- ret = blk_truncate(blk, new_size, PREALLOC_MODE_OFF, &local_err); ++ ret = blk_truncate(blk, new_size, PREALLOC_MODE_OFF, errp); + blk_unref(blk); + if (ret < 0) { +- error_report_err(local_err); + return ret; + } + } +@@ -4360,7 +4370,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts, + if (new_version < old_version) { + helper_cb_info.current_operation = QCOW2_DOWNGRADING; + ret = qcow2_downgrade(bs, new_version, &qcow2_amend_helper_cb, +- &helper_cb_info); ++ &helper_cb_info, errp); + if (ret < 0) { + return ret; + } +diff --git a/include/block/block.h b/include/block/block.h +index 06cd772..2d17b09 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -343,7 +343,8 @@ int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix); + typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset, + int64_t total_work_size, void *opaque); + int bdrv_amend_options(BlockDriverState *bs_new, QemuOpts *opts, +- BlockDriverAmendStatusCB *status_cb, void *cb_opaque); ++ BlockDriverAmendStatusCB *status_cb, void *cb_opaque, ++ Error **errp); + + /* external snapshots */ + bool bdrv_recurse_is_first_non_filter(BlockDriverState *bs, +diff --git a/include/block/block_int.h b/include/block/block_int.h +index c4dd1d4..d913ed1 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -324,7 +324,8 @@ struct BlockDriver { + + int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts, + BlockDriverAmendStatusCB *status_cb, +- void *cb_opaque); ++ void *cb_opaque, ++ Error **errp); + + void (*bdrv_debug_event)(BlockDriverState *bs, BlkdebugEvent event); + +diff --git a/qemu-img.c b/qemu-img.c +index 2f7c491..e40d6ff 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -3761,10 +3761,10 @@ static int img_amend(int argc, char **argv) + + /* In case the driver does not call amend_status_cb() */ + qemu_progress_print(0.f, 0); +- ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL); ++ ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL, &err); + qemu_progress_print(100.f, 0); + if (ret < 0) { +- error_report("Error while amending options: %s", strerror(-ret)); ++ error_report_err(err); + goto out; + } + +diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out +index 99234fb..bff023d 100644 +--- a/tests/qemu-iotests/060.out ++++ b/tests/qemu-iotests/060.out +@@ -129,7 +129,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + wrote 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + qcow2: Marking image as corrupt: L2 table offset 0x42a00 unaligned (L1 index: 0); further corruption events will be suppressed +-qemu-img: Error while amending options: Input/output error ++qemu-img: Failed to turn zero into data clusters: Input/output error + + === Testing unaligned L2 entry === + +@@ -145,7 +145,7 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + wrote 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + qcow2: Marking image as corrupt: Cluster allocation offset 0x52a00 unaligned (L2 offset: 0x40000, L2 index: 0); further corruption events will be suppressed +-qemu-img: Error while amending options: Input/output error ++qemu-img: Failed to turn zero into data clusters: Input/output error + + === Testing unaligned reftable entry === + +diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out +index e857ef9..183f7dd 100644 +--- a/tests/qemu-iotests/061.out ++++ b/tests/qemu-iotests/061.out +@@ -358,18 +358,12 @@ No errors were found on the image. + + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater) +-qemu-img: Error while amending options: Invalid argument + qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater) +-qemu-img: Error while amending options: Invalid argument + qemu-img: Unknown compatibility level 0.42 +-qemu-img: Error while amending options: Invalid argument + qemu-img: Invalid parameter 'foo' + qemu-img: Changing the cluster size is not supported +-qemu-img: Error while amending options: Operation not supported + qemu-img: Changing the encryption flag is not supported +-qemu-img: Error while amending options: Operation not supported + qemu-img: Cannot change preallocation mode +-qemu-img: Error while amending options: Operation not supported + + === Testing correct handling of unset value === + +@@ -377,7 +371,6 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Should work: + Should not work: + qemu-img: Changing the cluster size is not supported +-qemu-img: Error while amending options: Operation not supported + + === Testing zero expansion on inactive clusters === + +diff --git a/tests/qemu-iotests/080.out b/tests/qemu-iotests/080.out +index 4e0f7f7..281c7e0 100644 +--- a/tests/qemu-iotests/080.out ++++ b/tests/qemu-iotests/080.out +@@ -65,7 +65,7 @@ wrote 512/512 bytes at offset 0 + 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + qemu-img: Failed to load snapshot: Snapshot L1 table offset invalid + qemu-img: Snapshot L1 table offset invalid +-qemu-img: Error while amending options: Invalid argument ++qemu-img: Failed to turn zero into data clusters: Invalid argument + Failed to flush the refcount block cache: Invalid argument + write failed: Invalid argument + qemu-img: Snapshot L1 table offset invalid +@@ -88,7 +88,7 @@ wrote 512/512 bytes at offset 0 + 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + qemu-img: Failed to load snapshot: Snapshot L1 table too large + qemu-img: Snapshot L1 table too large +-qemu-img: Error while amending options: File too large ++qemu-img: Failed to turn zero into data clusters: File too large + Failed to flush the refcount block cache: File too large + write failed: File too large + qemu-img: Snapshot L1 table too large +diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out +index 86f0410..ae0318c 100644 +--- a/tests/qemu-iotests/112.out ++++ b/tests/qemu-iotests/112.out +@@ -99,13 +99,11 @@ refcount bits: 64 + === Amend to compat=0.10 === + + qemu-img: compat=0.10 requires refcount_bits=16 +-qemu-img: Error while amending options: Operation not supported + refcount bits: 64 + No errors were found on the image. + refcount bits: 16 + refcount bits: 16 +-qemu-img: Different refcount widths than 16 bits require compatibility level 1.1 or above (use compat=1.1 or greater) +-qemu-img: Error while amending options: Invalid argument ++qemu-img: Refcount widths other than 16 bits require compatibility level 1.1 or above (use compat=1.1 or greater) + refcount bits: 16 + + === Amend with snapshot === +@@ -113,7 +111,6 @@ refcount bits: 16 + wrote 16777216/16777216 bytes at offset 0 + 16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + qemu-img: Cannot decrease refcount entry width to 1 bits: Cluster at offset 0x50000 has a refcount of 2 +-qemu-img: Error while amending options: Invalid argument + No errors were found on the image. + refcount bits: 16 + No errors were found on the image. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Add-auto-read-only-option.patch b/SOURCES/kvm-block-Add-auto-read-only-option.patch new file mode 100644 index 0000000..e38f2f8 --- /dev/null +++ b/SOURCES/kvm-block-Add-auto-read-only-option.patch @@ -0,0 +1,200 @@ +From cff7af832cadce3d5afd2819483b1b61a115ace2 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:32 +0000 +Subject: [PATCH 02/14] block: Add auto-read-only option + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-3-kwolf@redhat.com> +Patchwork-id: 83952 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 02/12] block: Add auto-read-only option +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +If a management application builds the block graph node by node, the +protocol layer doesn't inherit its read-only option from the format +layer any more, so it must be set explicitly. + +Backing files should work on read-only storage, but at the same time, a +block job like commit should be able to reopen them read-write if they +are on read-write storage. However, without option inheritance, reopen +only changes the read-only option for the root node (typically the +format layer), but not the protocol layer, so reopening fails (the +format layer wants to get write permissions, but the protocol layer is +still read-only). + +A simple workaround for the problem in the management tool would be to +open the protocol layer always read-write and to make only the format +layer read-only for backing files. However, sometimes the file is +actually stored on read-only storage and we don't know whether the image +can be opened read-write (for example, for NBD it depends on the server +we're trying to connect to). This adds an option that makes QEMU try to +open the image read-write, but allows it to degrade to a read-only mode +without returning an error. + +The documentation for this option is consciously phrased in a way that +allows QEMU to switch to a better model eventually: Instead of trying +when the image is first opened, making the read-only flag dynamic and +changing it automatically whenever the first BLK_PERM_WRITE user is +attached or the last one is detached would be much more useful +behaviour. + +Unfortunately, this more useful behaviour is also a lot harder to +implement, and libvirt needs a solution now before it can switch to +-blockdev, so let's start with this easier approach for now. + +Instead of adding a new auto-read-only option, turning the existing +read-only into an enum (with a bool alternate for compatibility) was +considered, but it complicated the implementation to the point that it +didn't seem to be worth it. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit e35bdc123a4ace9f4d3fccaaf88907014e2438cd) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 17 +++++++++++++++++ + block/vvfat.c | 1 + + blockdev.c | 2 +- + include/block/block.h | 2 ++ + qapi/block-core.json | 7 +++++++ + 5 files changed, 28 insertions(+), 1 deletion(-) + +diff --git a/block.c b/block.c +index 6f1d53b..f357975 100644 +--- a/block.c ++++ b/block.c +@@ -905,6 +905,7 @@ static void bdrv_inherited_options(int *child_flags, QDict *child_options, + + /* Inherit the read-only option from the parent if it's not set */ + qdict_copy_default(child_options, parent_options, BDRV_OPT_READ_ONLY); ++ qdict_copy_default(child_options, parent_options, BDRV_OPT_AUTO_READ_ONLY); + + /* Our block drivers take care to send flushes and respect unmap policy, + * so we can default to enable both on lower layers regardless of the +@@ -1028,6 +1029,7 @@ static void bdrv_backing_options(int *child_flags, QDict *child_options, + + /* backing files always opened read-only */ + qdict_set_default_str(child_options, BDRV_OPT_READ_ONLY, "on"); ++ qdict_set_default_str(child_options, BDRV_OPT_AUTO_READ_ONLY, "off"); + flags &= ~BDRV_O_COPY_ON_READ; + + /* snapshot=on is handled on the top layer */ +@@ -1117,6 +1119,10 @@ static void update_flags_from_options(int *flags, QemuOpts *opts) + *flags |= BDRV_O_RDWR; + } + ++ assert(qemu_opt_find(opts, BDRV_OPT_AUTO_READ_ONLY)); ++ if (qemu_opt_get_bool_del(opts, BDRV_OPT_AUTO_READ_ONLY, false)) { ++ *flags |= BDRV_O_AUTO_RDONLY; ++ } + } + + static void update_options_from_flags(QDict *options, int flags) +@@ -1131,6 +1137,10 @@ static void update_options_from_flags(QDict *options, int flags) + if (!qdict_haskey(options, BDRV_OPT_READ_ONLY)) { + qdict_put_bool(options, BDRV_OPT_READ_ONLY, !(flags & BDRV_O_RDWR)); + } ++ if (!qdict_haskey(options, BDRV_OPT_AUTO_READ_ONLY)) { ++ qdict_put_bool(options, BDRV_OPT_AUTO_READ_ONLY, ++ flags & BDRV_O_AUTO_RDONLY); ++ } + } + + static void bdrv_assign_node_name(BlockDriverState *bs, +@@ -1304,6 +1314,11 @@ QemuOptsList bdrv_runtime_opts = { + .help = "Node is opened in read-only mode", + }, + { ++ .name = BDRV_OPT_AUTO_READ_ONLY, ++ .type = QEMU_OPT_BOOL, ++ .help = "Node can become read-only if opening read-write fails", ++ }, ++ { + .name = "detect-zeroes", + .type = QEMU_OPT_STRING, + .help = "try to optimize zero writes (off, on, unmap)", +@@ -2490,6 +2505,8 @@ BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp) + qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off"); + qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off"); + qdict_set_default_str(qdict, BDRV_OPT_READ_ONLY, "off"); ++ qdict_set_default_str(qdict, BDRV_OPT_AUTO_READ_ONLY, "off"); ++ + } + + bs = bdrv_open_inherit(NULL, reference, qdict, 0, NULL, NULL, errp); +diff --git a/block/vvfat.c b/block/vvfat.c +index c7d2ed2..3efce9e 100644 +--- a/block/vvfat.c ++++ b/block/vvfat.c +@@ -3130,6 +3130,7 @@ static void vvfat_qcow_options(int *child_flags, QDict *child_options, + int parent_flags, QDict *parent_options) + { + qdict_set_default_str(child_options, BDRV_OPT_READ_ONLY, "off"); ++ qdict_set_default_str(child_options, BDRV_OPT_AUTO_READ_ONLY, "off"); + qdict_set_default_str(child_options, BDRV_OPT_CACHE_NO_FLUSH, "on"); + } + +diff --git a/blockdev.c b/blockdev.c +index 56a3d0f..be650d0 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2760,7 +2760,7 @@ void qmp_blockdev_change_medium(bool has_device, const char *device, + + bdrv_flags = blk_get_open_flags_from_root_state(blk); + bdrv_flags &= ~(BDRV_O_TEMPORARY | BDRV_O_SNAPSHOT | BDRV_O_NO_BACKING | +- BDRV_O_PROTOCOL); ++ BDRV_O_PROTOCOL | BDRV_O_AUTO_RDONLY); + + if (!has_read_only) { + read_only = BLOCKDEV_CHANGE_READ_ONLY_MODE_RETAIN; +diff --git a/include/block/block.h b/include/block/block.h +index 8e78daf..6ee8b2a 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -114,6 +114,7 @@ typedef struct HDGeometry { + select an appropriate protocol driver, + ignoring the format layer */ + #define BDRV_O_NO_IO 0x10000 /* don't initialize for I/O */ ++#define BDRV_O_AUTO_RDONLY 0x20000 /* degrade to read-only if opening read-write fails */ + + #define BDRV_O_CACHE_MASK (BDRV_O_NOCACHE | BDRV_O_NO_FLUSH) + +@@ -124,6 +125,7 @@ typedef struct HDGeometry { + #define BDRV_OPT_CACHE_DIRECT "cache.direct" + #define BDRV_OPT_CACHE_NO_FLUSH "cache.no-flush" + #define BDRV_OPT_READ_ONLY "read-only" ++#define BDRV_OPT_AUTO_READ_ONLY "auto-read-only" + #define BDRV_OPT_DISCARD "discard" + #define BDRV_OPT_FORCE_SHARE "force-share" + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index db47fb8..5e5f4f9 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -3604,6 +3604,12 @@ + # either generally or in certain configurations. In this case, + # the default value does not work and the option must be + # specified explicitly. ++# @auto-read-only: if true and @read-only is false, QEMU may automatically ++# decide not to open the image read-write as requested, but ++# fall back to read-only instead (and switch between the modes ++# later), e.g. depending on whether the image file is writable ++# or whether a writing user is attached to the node ++# (default: false, since 3.1) + # @detect-zeroes: detect and optimize zero writes (Since 2.1) + # (default: off) + # @force-share: force share all permission on added nodes. +@@ -3619,6 +3625,7 @@ + '*discard': 'BlockdevDiscardOptions', + '*cache': 'BlockdevCacheOptions', + '*read-only': 'bool', ++ '*auto-read-only': 'bool', + '*force-share': 'bool', + '*detect-zeroes': 'BlockdevDetectZeroesOptions' }, + 'discriminator': 'driver', +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Add-block-specific-QDict-header.patch b/SOURCES/kvm-block-Add-block-specific-QDict-header.patch new file mode 100644 index 0000000..dde7fc6 --- /dev/null +++ b/SOURCES/kvm-block-Add-block-specific-QDict-header.patch @@ -0,0 +1,423 @@ +From 8c7a6486578c7d3e95ca72d798a3e2a5c3f8c348 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:15 +0200 +Subject: [PATCH 017/268] block: Add block-specific QDict header + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-9-armbru@redhat.com> +Patchwork-id: 80736 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 08/23] block: Add block-specific QDict header +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +From: Max Reitz + +There are numerous QDict functions that have been introduced for and are +used only by the block layer. Move their declarations into an own +header file to reflect that. + +While qdict_extract_subqdict() is in fact used outside of the block +layer (in util/qemu-config.c), it is still a function related very +closely to how the block layer works with nested QDicts, namely by +sometimes flattening them. Therefore, its declaration is put into this +header as well and util/qemu-config.c includes it with a comment stating +exactly which function it needs. + +Suggested-by: Markus Armbruster +Signed-off-by: Max Reitz +Message-Id: <20180509165530.29561-7-mreitz@redhat.com> +[Copyright note tweaked, superfluous includes dropped] +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 609f45ea9507fc1603eaeda7f5066b99beac6721) +[Trivial conflict in block/nbd.c resolved] + +Signed-off-by: Miroslav Rezanina +--- + block.c | 1 + + block/gluster.c | 1 + + block/iscsi.c | 1 + + block/nbd.c | 1 + + block/nfs.c | 1 + + block/parallels.c | 1 + + block/qcow.c | 1 + + block/qcow2.c | 1 + + block/qed.c | 1 + + block/quorum.c | 1 + + block/rbd.c | 1 + + block/sheepdog.c | 1 + + block/snapshot.c | 1 + + block/ssh.c | 1 + + block/vhdx.c | 1 + + block/vpc.c | 1 + + block/vvfat.c | 1 + + block/vxhs.c | 1 + + blockdev.c | 1 + + include/block/qdict.h | 32 ++++++++++++++++++++++++++++++++ + include/qapi/qmp/qdict.h | 17 ----------------- + qobject/qdict.c | 1 + + tests/check-qdict.c | 1 + + tests/check-qobject.c | 1 + + tests/test-replication.c | 1 + + util/qemu-config.c | 1 + + 26 files changed, 56 insertions(+), 17 deletions(-) + create mode 100644 include/block/qdict.h + +diff --git a/block.c b/block.c +index 676e57f..3c3e8fd 100644 +--- a/block.c ++++ b/block.c +@@ -27,6 +27,7 @@ + #include "block/block_int.h" + #include "block/blockjob.h" + #include "block/nbd.h" ++#include "block/qdict.h" + #include "qemu/error-report.h" + #include "module_block.h" + #include "qemu/module.h" +diff --git a/block/gluster.c b/block/gluster.c +index 55be566..418bb73 100644 +--- a/block/gluster.c ++++ b/block/gluster.c +@@ -11,6 +11,7 @@ + #include "qemu/osdep.h" + #include + #include "block/block_int.h" ++#include "block/qdict.h" + #include "qapi/error.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qerror.h" +diff --git a/block/iscsi.c b/block/iscsi.c +index 658462b..1705187 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -33,6 +33,7 @@ + #include "qemu/bitops.h" + #include "qemu/bitmap.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "scsi/constants.h" + #include "qemu/iov.h" + #include "qemu/option.h" +diff --git a/block/nbd.c b/block/nbd.c +index 3e1693c..f499830 100644 +--- a/block/nbd.c ++++ b/block/nbd.c +@@ -28,6 +28,7 @@ + + #include "qemu/osdep.h" + #include "block/nbd-client.h" ++#include "block/qdict.h" + #include "qapi/error.h" + #include "qemu/uri.h" + #include "block/block_int.h" +diff --git a/block/nfs.c b/block/nfs.c +index 66fddf1..5159ef0 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -29,6 +29,7 @@ + #include "qemu/error-report.h" + #include "qapi/error.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "trace.h" + #include "qemu/iov.h" + #include "qemu/option.h" +diff --git a/block/parallels.c b/block/parallels.c +index 045810d..0ee1f6a 100644 +--- a/block/parallels.c ++++ b/block/parallels.c +@@ -31,6 +31,7 @@ + #include "qemu/osdep.h" + #include "qapi/error.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/module.h" + #include "qemu/option.h" +diff --git a/block/qcow.c b/block/qcow.c +index 4b2f7db..fb821ad 100644 +--- a/block/qcow.c ++++ b/block/qcow.c +@@ -26,6 +26,7 @@ + #include "qapi/error.h" + #include "qemu/error-report.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/module.h" + #include "qemu/option.h" +diff --git a/block/qcow2.c b/block/qcow2.c +index 2f36e63..fa9f557 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -24,6 +24,7 @@ + + #include "qemu/osdep.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/module.h" + #include +diff --git a/block/qed.c b/block/qed.c +index 1db8eaf..9a8997a 100644 +--- a/block/qed.c ++++ b/block/qed.c +@@ -13,6 +13,7 @@ + */ + + #include "qemu/osdep.h" ++#include "block/qdict.h" + #include "qapi/error.h" + #include "qemu/timer.h" + #include "qemu/bswap.h" +diff --git a/block/quorum.c b/block/quorum.c +index a5051da..f1f39ba 100644 +--- a/block/quorum.c ++++ b/block/quorum.c +@@ -17,6 +17,7 @@ + #include "qemu/cutils.h" + #include "qemu/option.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "qapi/error.h" + #include "qapi/qapi-events-block.h" + #include "qapi/qmp/qdict.h" +diff --git a/block/rbd.c b/block/rbd.c +index 2842c0e..e695cf2 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -18,6 +18,7 @@ + #include "qemu/error-report.h" + #include "qemu/option.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "crypto/secret.h" + #include "qemu/cutils.h" + #include "qapi/qmp/qstring.h" +diff --git a/block/sheepdog.c b/block/sheepdog.c +index 07529f4..fd3876f 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -24,6 +24,7 @@ + #include "qemu/option.h" + #include "qemu/sockets.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/bitops.h" + #include "qemu/cutils.h" +diff --git a/block/snapshot.c b/block/snapshot.c +index 2953d96..f9903bc 100644 +--- a/block/snapshot.c ++++ b/block/snapshot.c +@@ -25,6 +25,7 @@ + #include "qemu/osdep.h" + #include "block/snapshot.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "qapi/error.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qerror.h" +diff --git a/block/ssh.c b/block/ssh.c +index 412a1bf..5931064 100644 +--- a/block/ssh.c ++++ b/block/ssh.c +@@ -28,6 +28,7 @@ + #include + + #include "block/block_int.h" ++#include "block/qdict.h" + #include "qapi/error.h" + #include "qemu/error-report.h" + #include "qemu/option.h" +diff --git a/block/vhdx.c b/block/vhdx.c +index c3a4220..26c05aa 100644 +--- a/block/vhdx.c ++++ b/block/vhdx.c +@@ -18,6 +18,7 @@ + #include "qemu/osdep.h" + #include "qapi/error.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/module.h" + #include "qemu/option.h" +diff --git a/block/vpc.c b/block/vpc.c +index 0ebfcd3..41c8c98 100644 +--- a/block/vpc.c ++++ b/block/vpc.c +@@ -26,6 +26,7 @@ + #include "qemu/osdep.h" + #include "qapi/error.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/module.h" + #include "qemu/option.h" +diff --git a/block/vvfat.c b/block/vvfat.c +index 662dca0..4595f33 100644 +--- a/block/vvfat.c ++++ b/block/vvfat.c +@@ -27,6 +27,7 @@ + #include + #include "qapi/error.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "qemu/module.h" + #include "qemu/option.h" + #include "qemu/bswap.h" +diff --git a/block/vxhs.c b/block/vxhs.c +index 96e83d9..25fea7f 100644 +--- a/block/vxhs.c ++++ b/block/vxhs.c +@@ -13,6 +13,7 @@ + #include + #include + #include "block/block_int.h" ++#include "block/qdict.h" + #include "qapi/qmp/qerror.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qstring.h" +diff --git a/blockdev.c b/blockdev.c +index 3808b1f..19c04d9 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -35,6 +35,7 @@ + #include "sysemu/blockdev.h" + #include "hw/block/block.h" + #include "block/blockjob.h" ++#include "block/qdict.h" + #include "block/throttle-groups.h" + #include "monitor/monitor.h" + #include "qemu/error-report.h" +diff --git a/include/block/qdict.h b/include/block/qdict.h +new file mode 100644 +index 0000000..71c037a +--- /dev/null ++++ b/include/block/qdict.h +@@ -0,0 +1,32 @@ ++/* ++ * Special QDict functions used by the block layer ++ * ++ * Copyright (c) 2013-2018 Red Hat, Inc. ++ * ++ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. ++ * See the COPYING.LIB file in the top-level directory. ++ */ ++ ++#ifndef BLOCK_QDICT_H ++#define BLOCK_QDICT_H ++ ++#include "qapi/qmp/qdict.h" ++ ++void qdict_copy_default(QDict *dst, QDict *src, const char *key); ++void qdict_set_default_str(QDict *dst, const char *key, const char *val); ++ ++void qdict_join(QDict *dest, QDict *src, bool overwrite); ++ ++void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); ++void qdict_array_split(QDict *src, QList **dst); ++int qdict_array_entries(QDict *src, const char *subqdict); ++QObject *qdict_crumple(const QDict *src, Error **errp); ++void qdict_flatten(QDict *qdict); ++ ++typedef struct QDictRenames { ++ const char *from; ++ const char *to; ++} QDictRenames; ++bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); ++ ++#endif +diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h +index 921a28d..7f3ec10 100644 +--- a/include/qapi/qmp/qdict.h ++++ b/include/qapi/qmp/qdict.h +@@ -67,23 +67,6 @@ int64_t qdict_get_try_int(const QDict *qdict, const char *key, + bool qdict_get_try_bool(const QDict *qdict, const char *key, bool def_value); + const char *qdict_get_try_str(const QDict *qdict, const char *key); + +-void qdict_copy_default(QDict *dst, QDict *src, const char *key); +-void qdict_set_default_str(QDict *dst, const char *key, const char *val); +- + QDict *qdict_clone_shallow(const QDict *src); +-void qdict_flatten(QDict *qdict); +- +-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); +-void qdict_array_split(QDict *src, QList **dst); +-int qdict_array_entries(QDict *src, const char *subqdict); +-QObject *qdict_crumple(const QDict *src, Error **errp); +- +-void qdict_join(QDict *dest, QDict *src, bool overwrite); +- +-typedef struct QDictRenames { +- const char *from; +- const char *to; +-} QDictRenames; +-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); + + #endif /* QDICT_H */ +diff --git a/qobject/qdict.c b/qobject/qdict.c +index 22800ee..0554c64 100644 +--- a/qobject/qdict.c ++++ b/qobject/qdict.c +@@ -11,6 +11,7 @@ + */ + + #include "qemu/osdep.h" ++#include "block/qdict.h" + #include "qapi/qmp/qnum.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qbool.h" +diff --git a/tests/check-qdict.c b/tests/check-qdict.c +index eba5d35..93e2112 100644 +--- a/tests/check-qdict.c ++++ b/tests/check-qdict.c +@@ -11,6 +11,7 @@ + */ + + #include "qemu/osdep.h" ++#include "block/qdict.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qlist.h" + #include "qapi/qmp/qnum.h" +diff --git a/tests/check-qobject.c b/tests/check-qobject.c +index 5cb08fc..16ccbde 100644 +--- a/tests/check-qobject.c ++++ b/tests/check-qobject.c +@@ -8,6 +8,7 @@ + */ + + #include "qemu/osdep.h" ++#include "block/qdict.h" + #include "qapi/qmp/qbool.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qlist.h" +diff --git a/tests/test-replication.c b/tests/test-replication.c +index 68c0d04..c8165ae 100644 +--- a/tests/test-replication.c ++++ b/tests/test-replication.c +@@ -15,6 +15,7 @@ + #include "qemu/option.h" + #include "replication.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + + #define IMG_SIZE (64 * 1024 * 1024) +diff --git a/util/qemu-config.c b/util/qemu-config.c +index 14d8402..9d2e278 100644 +--- a/util/qemu-config.c ++++ b/util/qemu-config.c +@@ -1,4 +1,5 @@ + #include "qemu/osdep.h" ++#include "block/qdict.h" /* for qdict_extract_subqdict() */ + #include "qapi/error.h" + #include "qapi/qapi-commands-misc.h" + #include "qapi/qmp/qdict.h" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Add-missing-locking-in-bdrv_co_drain_bh_cb.patch b/SOURCES/kvm-block-Add-missing-locking-in-bdrv_co_drain_bh_cb.patch new file mode 100644 index 0000000..8c8f1f2 --- /dev/null +++ b/SOURCES/kvm-block-Add-missing-locking-in-bdrv_co_drain_bh_cb.patch @@ -0,0 +1,96 @@ +From 59f0fc4b8011298bc542eb23cca385d83d1963cd Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:02 +0100 +Subject: [PATCH 36/49] block: Add missing locking in bdrv_co_drain_bh_cb() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-24-kwolf@redhat.com> +Patchwork-id: 82613 +O-Subject: [RHEL-8 qemu-kvm PATCH 33/44] block: Add missing locking in bdrv_co_drain_bh_cb() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +bdrv_do_drained_begin/end() assume that they are called with the +AioContext lock of bs held. If we call drain functions from a coroutine +with the AioContext lock held, we yield and schedule a BH to move out of +coroutine context. This means that the lock for the home context of the +coroutine is released and must be re-acquired in the bottom half. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit aa1361d54aac43094b98024b8b6c804eb6e41661) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 15 +++++++++++++++ + include/qemu/coroutine.h | 5 +++++ + util/qemu-coroutine.c | 5 +++++ + 3 files changed, 25 insertions(+) + +diff --git a/block/io.c b/block/io.c +index d404088..19db35e 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -286,6 +286,18 @@ static void bdrv_co_drain_bh_cb(void *opaque) + BlockDriverState *bs = data->bs; + + if (bs) { ++ AioContext *ctx = bdrv_get_aio_context(bs); ++ AioContext *co_ctx = qemu_coroutine_get_aio_context(co); ++ ++ /* ++ * When the coroutine yielded, the lock for its home context was ++ * released, so we need to re-acquire it here. If it explicitly ++ * acquired a different context, the lock is still held and we don't ++ * want to lock it a second time (or AIO_WAIT_WHILE() would hang). ++ */ ++ if (ctx == co_ctx) { ++ aio_context_acquire(ctx); ++ } + bdrv_dec_in_flight(bs); + if (data->begin) { + bdrv_do_drained_begin(bs, data->recursive, data->parent, +@@ -294,6 +306,9 @@ static void bdrv_co_drain_bh_cb(void *opaque) + bdrv_do_drained_end(bs, data->recursive, data->parent, + data->ignore_bds_parents); + } ++ if (ctx == co_ctx) { ++ aio_context_release(ctx); ++ } + } else { + assert(data->begin); + bdrv_drain_all_begin(); +diff --git a/include/qemu/coroutine.h b/include/qemu/coroutine.h +index 6f8a487..9801e7f 100644 +--- a/include/qemu/coroutine.h ++++ b/include/qemu/coroutine.h +@@ -90,6 +90,11 @@ void qemu_aio_coroutine_enter(AioContext *ctx, Coroutine *co); + void coroutine_fn qemu_coroutine_yield(void); + + /** ++ * Get the AioContext of the given coroutine ++ */ ++AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co); ++ ++/** + * Get the currently executing coroutine + */ + Coroutine *coroutine_fn qemu_coroutine_self(void); +diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c +index 1ba4191..2295928 100644 +--- a/util/qemu-coroutine.c ++++ b/util/qemu-coroutine.c +@@ -198,3 +198,8 @@ bool qemu_coroutine_entered(Coroutine *co) + { + return co->caller; + } ++ ++AioContext *coroutine_fn qemu_coroutine_get_aio_context(Coroutine *co) ++{ ++ return co->ctx; ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Allow-AIO_WAIT_WHILE-with-NULL-ctx.patch b/SOURCES/kvm-block-Allow-AIO_WAIT_WHILE-with-NULL-ctx.patch new file mode 100644 index 0000000..36a7614 --- /dev/null +++ b/SOURCES/kvm-block-Allow-AIO_WAIT_WHILE-with-NULL-ctx.patch @@ -0,0 +1,74 @@ +From f4414f5600168109da86e321f1651f71605cf9aa Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:45 +0100 +Subject: [PATCH 19/49] block: Allow AIO_WAIT_WHILE with NULL ctx + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-7-kwolf@redhat.com> +Patchwork-id: 82596 +O-Subject: [RHEL-8 qemu-kvm PATCH 16/44] block: Allow AIO_WAIT_WHILE with NULL ctx +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +bdrv_drain_all() wants to have a single polling loop for draining the +in-flight requests of all nodes. This means that the AIO_WAIT_WHILE() +condition relies on activity in multiple AioContexts, which is polled +from the mainloop context. We must therefore call AIO_WAIT_WHILE() from +the mainloop thread and use the AioWait notification mechanism. + +Just randomly picking the AioContext of any non-mainloop thread would +work, but instead of bothering to find such a context in the caller, we +can just as well accept NULL for ctx. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 4d22bbf4ef72583eefdf44db6bf9fc7683fbc4c2) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + include/block/aio-wait.h | 13 +++++++++---- + 1 file changed, 9 insertions(+), 4 deletions(-) + +diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h +index 783d367..c85a62f 100644 +--- a/include/block/aio-wait.h ++++ b/include/block/aio-wait.h +@@ -57,7 +57,8 @@ typedef struct { + /** + * AIO_WAIT_WHILE: + * @wait: the aio wait object +- * @ctx: the aio context ++ * @ctx: the aio context, or NULL if multiple aio contexts (for which the ++ * caller does not hold a lock) are involved in the polling condition. + * @cond: wait while this conditional expression is true + * + * Wait while a condition is true. Use this to implement synchronous +@@ -75,7 +76,7 @@ typedef struct { + bool waited_ = false; \ + AioWait *wait_ = (wait); \ + AioContext *ctx_ = (ctx); \ +- if (in_aio_context_home_thread(ctx_)) { \ ++ if (ctx_ && in_aio_context_home_thread(ctx_)) { \ + while ((cond)) { \ + aio_poll(ctx_, true); \ + waited_ = true; \ +@@ -86,9 +87,13 @@ typedef struct { + /* Increment wait_->num_waiters before evaluating cond. */ \ + atomic_inc(&wait_->num_waiters); \ + while ((cond)) { \ +- aio_context_release(ctx_); \ ++ if (ctx_) { \ ++ aio_context_release(ctx_); \ ++ } \ + aio_poll(qemu_get_aio_context(), true); \ +- aio_context_acquire(ctx_); \ ++ if (ctx_) { \ ++ aio_context_acquire(ctx_); \ ++ } \ + waited_ = true; \ + } \ + atomic_dec(&wait_->num_waiters); \ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Allow-graph-changes-in-bdrv_drain_all_begin-en.patch b/SOURCES/kvm-block-Allow-graph-changes-in-bdrv_drain_all_begin-en.patch new file mode 100644 index 0000000..86bd508 --- /dev/null +++ b/SOURCES/kvm-block-Allow-graph-changes-in-bdrv_drain_all_begin-en.patch @@ -0,0 +1,266 @@ +From 7946b28c7e68c564b0734869ff6f4e36bd5e2e2d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:48 +0100 +Subject: [PATCH 22/49] block: Allow graph changes in bdrv_drain_all_begin/end + sections + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-10-kwolf@redhat.com> +Patchwork-id: 82597 +O-Subject: [RHEL-8 qemu-kvm PATCH 19/44] block: Allow graph changes in bdrv_drain_all_begin/end sections +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +bdrv_drain_all_*() used bdrv_next() to iterate over all root nodes and +did a subtree drain for each of them. This works fine as long as the +graph is static, but sadly, reality looks different. + +If the graph changes so that root nodes are added or removed, we would +have to compensate for this. bdrv_next() returns each root node only +once even if it's the root node for multiple BlockBackends or for a +monitor-owned block driver tree, which would only complicate things. + +The much easier and more obviously correct way is to fundamentally +change the way the functions work: Iterate over all BlockDriverStates, +no matter who owns them, and drain them individually. Compensation is +only necessary when a new BDS is created inside a drain_all section. +Removal of a BDS doesn't require any action because it's gone afterwards +anyway. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 0f12264e7a41458179ad10276a7c33c72024861a) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 34 ++++++++++++++++++++++++--- + block/io.c | 60 ++++++++++++++++++++++++++++++++++++----------- + include/block/block.h | 1 + + include/block/block_int.h | 1 + + 4 files changed, 79 insertions(+), 17 deletions(-) + +diff --git a/block.c b/block.c +index 519be6e..eea9c6f 100644 +--- a/block.c ++++ b/block.c +@@ -333,6 +333,10 @@ BlockDriverState *bdrv_new(void) + + qemu_co_queue_init(&bs->flush_queue); + ++ for (i = 0; i < bdrv_drain_all_count; i++) { ++ bdrv_drained_begin(bs); ++ } ++ + QTAILQ_INSERT_TAIL(&all_bdrv_states, bs, bs_list); + + return bs; +@@ -1170,7 +1174,7 @@ static int bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv, + int open_flags, Error **errp) + { + Error *local_err = NULL; +- int ret; ++ int i, ret; + + bdrv_assign_node_name(bs, node_name, &local_err); + if (local_err) { +@@ -1218,6 +1222,12 @@ static int bdrv_open_driver(BlockDriverState *bs, BlockDriver *drv, + assert(bdrv_min_mem_align(bs) != 0); + assert(is_power_of_2(bs->bl.request_alignment)); + ++ for (i = 0; i < bs->quiesce_counter; i++) { ++ if (drv->bdrv_co_drain_begin) { ++ drv->bdrv_co_drain_begin(bs); ++ } ++ } ++ + return 0; + open_failed: + bs->drv = NULL; +@@ -2039,7 +2049,12 @@ static void bdrv_replace_child_noperm(BdrvChild *child, + child->role->detach(child); + } + if (old_bs->quiesce_counter && child->role->drained_end) { +- for (i = 0; i < old_bs->quiesce_counter; i++) { ++ int num = old_bs->quiesce_counter; ++ if (child->role->parent_is_bds) { ++ num -= bdrv_drain_all_count; ++ } ++ assert(num >= 0); ++ for (i = 0; i < num; i++) { + child->role->drained_end(child); + } + } +@@ -2051,7 +2066,12 @@ static void bdrv_replace_child_noperm(BdrvChild *child, + if (new_bs) { + QLIST_INSERT_HEAD(&new_bs->parents, child, next_parent); + if (new_bs->quiesce_counter && child->role->drained_begin) { +- for (i = 0; i < new_bs->quiesce_counter; i++) { ++ int num = new_bs->quiesce_counter; ++ if (child->role->parent_is_bds) { ++ num -= bdrv_drain_all_count; ++ } ++ assert(num >= 0); ++ for (i = 0; i < num; i++) { + child->role->drained_begin(child); + } + } +@@ -3993,6 +4013,14 @@ BlockDriverState *bdrv_next_node(BlockDriverState *bs) + return QTAILQ_NEXT(bs, node_list); + } + ++BlockDriverState *bdrv_next_all_states(BlockDriverState *bs) ++{ ++ if (!bs) { ++ return QTAILQ_FIRST(&all_bdrv_states); ++ } ++ return QTAILQ_NEXT(bs, bs_list); ++} ++ + const char *bdrv_get_node_name(const BlockDriverState *bs) + { + return bs->node_name; +diff --git a/block/io.c b/block/io.c +index 0021fefd..38ae299 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -38,6 +38,8 @@ + /* Maximum bounce buffer for copy-on-read and write zeroes, in bytes */ + #define MAX_BOUNCE_BUFFER (32768 << BDRV_SECTOR_BITS) + ++static AioWait drain_all_aio_wait; ++ + static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int bytes, BdrvRequestFlags flags); + +@@ -471,6 +473,29 @@ static void bdrv_drain_assert_idle(BlockDriverState *bs) + } + } + ++unsigned int bdrv_drain_all_count = 0; ++ ++static bool bdrv_drain_all_poll(void) ++{ ++ BlockDriverState *bs = NULL; ++ bool result = false; ++ ++ /* Execute pending BHs first (may modify the graph) and check everything ++ * else only after the BHs have executed. */ ++ while (aio_poll(qemu_get_aio_context(), false)); ++ ++ /* bdrv_drain_poll() can't make changes to the graph and we are holding the ++ * main AioContext lock, so iterating bdrv_next_all_states() is safe. */ ++ while ((bs = bdrv_next_all_states(bs))) { ++ AioContext *aio_context = bdrv_get_aio_context(bs); ++ aio_context_acquire(aio_context); ++ result |= bdrv_drain_poll(bs, false, NULL, true); ++ aio_context_release(aio_context); ++ } ++ ++ return result; ++} ++ + /* + * Wait for pending requests to complete across all BlockDriverStates + * +@@ -485,45 +510,51 @@ static void bdrv_drain_assert_idle(BlockDriverState *bs) + */ + void bdrv_drain_all_begin(void) + { +- BlockDriverState *bs; +- BdrvNextIterator it; ++ BlockDriverState *bs = NULL; + + if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(NULL, true, false, NULL, false, true); ++ bdrv_co_yield_to_drain(NULL, true, false, NULL, true, true); + return; + } + +- /* BDRV_POLL_WHILE() for a node can only be called from its own I/O thread +- * or the main loop AioContext. We potentially use BDRV_POLL_WHILE() on +- * nodes in several different AioContexts, so make sure we're in the main +- * context. */ ++ /* AIO_WAIT_WHILE() with a NULL context can only be called from the main ++ * loop AioContext, so make sure we're in the main context. */ + assert(qemu_get_current_aio_context() == qemu_get_aio_context()); ++ assert(bdrv_drain_all_count < INT_MAX); ++ bdrv_drain_all_count++; + +- for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { ++ /* Quiesce all nodes, without polling in-flight requests yet. The graph ++ * cannot change during this loop. */ ++ while ((bs = bdrv_next_all_states(bs))) { + AioContext *aio_context = bdrv_get_aio_context(bs); + + aio_context_acquire(aio_context); +- bdrv_do_drained_begin(bs, true, NULL, false, true); ++ bdrv_do_drained_begin(bs, false, NULL, true, false); + aio_context_release(aio_context); + } + +- for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { ++ /* Now poll the in-flight requests */ ++ AIO_WAIT_WHILE(&drain_all_aio_wait, NULL, bdrv_drain_all_poll()); ++ ++ while ((bs = bdrv_next_all_states(bs))) { + bdrv_drain_assert_idle(bs); + } + } + + void bdrv_drain_all_end(void) + { +- BlockDriverState *bs; +- BdrvNextIterator it; ++ BlockDriverState *bs = NULL; + +- for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { ++ while ((bs = bdrv_next_all_states(bs))) { + AioContext *aio_context = bdrv_get_aio_context(bs); + + aio_context_acquire(aio_context); +- bdrv_do_drained_end(bs, true, NULL, false); ++ bdrv_do_drained_end(bs, false, NULL, true); + aio_context_release(aio_context); + } ++ ++ assert(bdrv_drain_all_count > 0); ++ bdrv_drain_all_count--; + } + + void bdrv_drain_all(void) +@@ -658,6 +689,7 @@ void bdrv_inc_in_flight(BlockDriverState *bs) + void bdrv_wakeup(BlockDriverState *bs) + { + aio_wait_kick(bdrv_get_aio_wait(bs)); ++ aio_wait_kick(&drain_all_aio_wait); + } + + void bdrv_dec_in_flight(BlockDriverState *bs) +diff --git a/include/block/block.h b/include/block/block.h +index 6e91803..f9079ac 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -449,6 +449,7 @@ BlockDriverState *bdrv_lookup_bs(const char *device, + Error **errp); + bool bdrv_chain_contains(BlockDriverState *top, BlockDriverState *base); + BlockDriverState *bdrv_next_node(BlockDriverState *bs); ++BlockDriverState *bdrv_next_all_states(BlockDriverState *bs); + + typedef struct BdrvNextIterator { + enum { +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 0ad8a76..9757d5e 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -845,6 +845,7 @@ int coroutine_fn bdrv_co_pwritev(BdrvChild *child, + int64_t offset, unsigned int bytes, QEMUIOVector *qiov, + BdrvRequestFlags flags); + ++extern unsigned int bdrv_drain_all_count; + void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_parent); + void bdrv_unapply_subtree_drain(BdrvChild *child, BlockDriverState *old_parent); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Avoid-unnecessary-aio_poll-in-AIO_WAIT_WHILE.patch b/SOURCES/kvm-block-Avoid-unnecessary-aio_poll-in-AIO_WAIT_WHILE.patch new file mode 100644 index 0000000..03f5be8 --- /dev/null +++ b/SOURCES/kvm-block-Avoid-unnecessary-aio_poll-in-AIO_WAIT_WHILE.patch @@ -0,0 +1,112 @@ +From 4f12612c2c25fb3093d1afa45030e424477344c7 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:40 +0100 +Subject: [PATCH 09/49] block: Avoid unnecessary aio_poll() in AIO_WAIT_WHILE() + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-7-kwolf@redhat.com> +Patchwork-id: 82586 +O-Subject: [RHEL-8 qemu-kvm PATCH 06/44] block: Avoid unnecessary aio_poll() in AIO_WAIT_WHILE() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Commit 91af091f923 added an additional aio_poll() to BDRV_POLL_WHILE() +in order to make sure that all pending BHs are executed on drain. This +was the wrong place to make the fix, as it is useless overhead for all +other users of the macro and unnecessarily complicates the mechanism. + +This patch effectively reverts said commit (the context has changed a +bit and the code has moved to AIO_WAIT_WHILE()) and instead polls in the +loop condition for drain. + +The effect is probably hard to measure in any real-world use case +because actual I/O will dominate, but if I run only the initialisation +part of 'qemu-img convert' where it calls bdrv_block_status() for the +whole image to find out how much data there is copy, this phase actually +needs only roughly half the time after this patch. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 1cc8e54ada97f7ac479554e15ca9e426c895b158) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 11 ++++++++++- + include/block/aio-wait.h | 22 ++++++++-------------- + 2 files changed, 18 insertions(+), 15 deletions(-) + +diff --git a/block/io.c b/block/io.c +index e5fc42c..4d332c3 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -181,13 +181,22 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin) + BDRV_POLL_WHILE(bs, !data.done); + } + ++/* Returns true if BDRV_POLL_WHILE() should go into a blocking aio_poll() */ ++static bool bdrv_drain_poll(BlockDriverState *bs) ++{ ++ /* Execute pending BHs first and check everything else only after the BHs ++ * have executed. */ ++ while (aio_poll(bs->aio_context, false)); ++ return atomic_read(&bs->in_flight); ++} ++ + static bool bdrv_drain_recurse(BlockDriverState *bs) + { + BdrvChild *child, *tmp; + bool waited; + + /* Wait for drained requests to finish */ +- waited = BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0); ++ waited = BDRV_POLL_WHILE(bs, bdrv_drain_poll(bs)); + + QLIST_FOREACH_SAFE(child, &bs->children, next, tmp) { + BlockDriverState *bs = child->bs; +diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h +index 8c90a2e..783d367 100644 +--- a/include/block/aio-wait.h ++++ b/include/block/aio-wait.h +@@ -73,29 +73,23 @@ typedef struct { + */ + #define AIO_WAIT_WHILE(wait, ctx, cond) ({ \ + bool waited_ = false; \ +- bool busy_ = true; \ + AioWait *wait_ = (wait); \ + AioContext *ctx_ = (ctx); \ + if (in_aio_context_home_thread(ctx_)) { \ +- while ((cond) || busy_) { \ +- busy_ = aio_poll(ctx_, (cond)); \ +- waited_ |= !!(cond) | busy_; \ ++ while ((cond)) { \ ++ aio_poll(ctx_, true); \ ++ waited_ = true; \ + } \ + } else { \ + assert(qemu_get_current_aio_context() == \ + qemu_get_aio_context()); \ + /* Increment wait_->num_waiters before evaluating cond. */ \ + atomic_inc(&wait_->num_waiters); \ +- while (busy_) { \ +- if ((cond)) { \ +- waited_ = busy_ = true; \ +- aio_context_release(ctx_); \ +- aio_poll(qemu_get_aio_context(), true); \ +- aio_context_acquire(ctx_); \ +- } else { \ +- busy_ = aio_poll(ctx_, false); \ +- waited_ |= busy_; \ +- } \ ++ while ((cond)) { \ ++ aio_context_release(ctx_); \ ++ aio_poll(qemu_get_aio_context(), true); \ ++ aio_context_acquire(ctx_); \ ++ waited_ = true; \ + } \ + atomic_dec(&wait_->num_waiters); \ + } \ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-BLK_PERM_WRITE-includes-._UNCHANGED.patch b/SOURCES/kvm-block-BLK_PERM_WRITE-includes-._UNCHANGED.patch new file mode 100644 index 0000000..833e8ca --- /dev/null +++ b/SOURCES/kvm-block-BLK_PERM_WRITE-includes-._UNCHANGED.patch @@ -0,0 +1,50 @@ +From 9e527bd9df162b9af555106675efdb909164051e Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:05 +0200 +Subject: [PATCH 038/268] block: BLK_PERM_WRITE includes ..._UNCHANGED + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-4-mreitz@redhat.com> +Patchwork-id: 80763 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 03/10] block: BLK_PERM_WRITE includes ..._UNCHANGED +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Currently we never actually check whether the WRITE_UNCHANGED +permission has been taken for unchanging writes. But the one check that +is commented out checks both WRITE and WRITE_UNCHANGED; and considering +that WRITE_UNCHANGED is already documented as being weaker than WRITE, +we should probably explicitly document WRITE to include WRITE_UNCHANGED. + +Signed-off-by: Max Reitz +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Alberto Garcia +Message-id: 20180421132929.21610-3-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 24b7c538fea86b598e2a335f4805a0ab50a30e98) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + include/block/block.h | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/include/block/block.h b/include/block/block.h +index cdec363..397b5e8 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -205,6 +205,9 @@ enum { + * This permission (which is weaker than BLK_PERM_WRITE) is both enough and + * required for writes to the block node when the caller promises that + * the visible disk content doesn't change. ++ * ++ * As the BLK_PERM_WRITE permission is strictly stronger, either is ++ * sufficient to perform an unchanging write. + */ + BLK_PERM_WRITE_UNCHANGED = 0x04, + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Cancel-job-in-bdrv_close_all-callers.patch b/SOURCES/kvm-block-Cancel-job-in-bdrv_close_all-callers.patch new file mode 100644 index 0000000..5651e59 --- /dev/null +++ b/SOURCES/kvm-block-Cancel-job-in-bdrv_close_all-callers.patch @@ -0,0 +1,85 @@ +From dcfa2a8c812791ab69e0ac9656fc3613d750e00d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:24 +0200 +Subject: [PATCH 116/268] block: Cancel job in bdrv_close_all() callers + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-42-kwolf@redhat.com> +Patchwork-id: 81098 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 41/73] block: Cancel job in bdrv_close_all() callers +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Now that we cancel all jobs and not only block jobs on shutdown, doing +that in bdrv_close_all() isn't really appropriate any more. Move the +job_cancel_sync_all() call to the callers, and only assert that there +are no job running in bdrv_close_all(). + +Signed-off-by: Kevin Wolf +(cherry picked from commit b3b5299d58bce4366c647af40374e6b063f371eb) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block.c | 4 +--- + qemu-nbd.c | 8 +++++++- + vl.c | 1 + + 3 files changed, 9 insertions(+), 4 deletions(-) + +diff --git a/block.c b/block.c +index a83787f..afe30ca 100644 +--- a/block.c ++++ b/block.c +@@ -3374,9 +3374,7 @@ static void bdrv_close(BlockDriverState *bs) + + void bdrv_close_all(void) + { +- /* TODO We do want to cancel all jobs instead of just block jobs on +- * shutdown, but bdrv_close_all() isn't the right place any more. */ +- job_cancel_sync_all(); ++ assert(job_next(NULL) == NULL); + nbd_export_close_all(); + + /* Drop references from requests still in flight, such as canceled block +diff --git a/qemu-nbd.c b/qemu-nbd.c +index 0af0560..51b9d38 100644 +--- a/qemu-nbd.c ++++ b/qemu-nbd.c +@@ -482,6 +482,12 @@ static const char *socket_activation_validate_opts(const char *device, + return NULL; + } + ++static void qemu_nbd_shutdown(void) ++{ ++ job_cancel_sync_all(); ++ bdrv_close_all(); ++} ++ + int main(int argc, char **argv) + { + BlockBackend *blk; +@@ -928,7 +934,7 @@ int main(int argc, char **argv) + exit(EXIT_FAILURE); + } + bdrv_init(); +- atexit(bdrv_close_all); ++ atexit(qemu_nbd_shutdown); + + srcpath = argv[optind]; + if (imageOpts) { +diff --git a/vl.c b/vl.c +index ce7d04d..f253876 100644 +--- a/vl.c ++++ b/vl.c +@@ -4766,6 +4766,7 @@ int main(int argc, char **argv, char **envp) + /* No more vcpu or device emulation activity beyond this point */ + vm_shutdown(); + ++ job_cancel_sync_all(); + bdrv_close_all(); + + res_free(); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Clean-up-a-misuse-of-qobject_to-in-.bdrv_co_cr.patch b/SOURCES/kvm-block-Clean-up-a-misuse-of-qobject_to-in-.bdrv_co_cr.patch new file mode 100644 index 0000000..a0fc296 --- /dev/null +++ b/SOURCES/kvm-block-Clean-up-a-misuse-of-qobject_to-in-.bdrv_co_cr.patch @@ -0,0 +1,241 @@ +From 9cc36895f5a773ac65abedd6b56748c8e15d2eaf Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:19 +0200 +Subject: [PATCH 021/268] block: Clean up a misuse of qobject_to() in + .bdrv_co_create_opts() + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-13-armbru@redhat.com> +Patchwork-id: 80733 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 12/23] block: Clean up a misuse of qobject_to() in .bdrv_co_create_opts() +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +The following pattern occurs in the .bdrv_co_create_opts() methods of +parallels, qcow, qcow2, qed, vhdx and vpc: + + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; + goto done; + } + + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); + [...] + ret = 0; +done: + qobject_unref(qdict); + [...] + return ret; + +If qobject_to() fails, we return failure without setting errp. That's +wrong. As far as I can tell, it cannot fail here. Clean it up +anyway, by removing the useless conversion. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 92adf9dbcd3cf9cedddae995b04a99f5c42ae08c) +Signed-off-by: Miroslav Rezanina +--- + block/parallels.c | 9 ++++----- + block/qcow.c | 9 ++++----- + block/qcow2.c | 9 ++++----- + block/qed.c | 9 ++++----- + block/vhdx.c | 9 ++++----- + block/vpc.c | 9 ++++----- + 6 files changed, 24 insertions(+), 30 deletions(-) + +diff --git a/block/parallels.c b/block/parallels.c +index aa58955..1c96c39 100644 +--- a/block/parallels.c ++++ b/block/parallels.c +@@ -614,7 +614,7 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + BlockdevCreateOptions *create_options = NULL; + Error *local_err = NULL; + BlockDriverState *bs = NULL; +- QDict *qdict = NULL; ++ QDict *qdict; + QObject *qobj; + Visitor *v; + int ret; +@@ -652,14 +652,13 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- qobject_unref(qdict); +- qdict = qobject_to(QDict, qobj); +- if (qdict == NULL) { ++ if (!qobj) { + ret = -EINVAL; + goto done; + } + +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_keyval(qobj); ++ qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/qcow.c b/block/qcow.c +index 14b7296..43a595a 100644 +--- a/block/qcow.c ++++ b/block/qcow.c +@@ -944,7 +944,7 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + { + BlockdevCreateOptions *create_options = NULL; + BlockDriverState *bs = NULL; +- QDict *qdict = NULL; ++ QDict *qdict; + QObject *qobj; + Visitor *v; + const char *val; +@@ -996,14 +996,13 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- qobject_unref(qdict); +- qdict = qobject_to(QDict, qobj); +- if (qdict == NULL) { ++ if (!qobj) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_keyval(qobj); ++ qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/qcow2.c b/block/qcow2.c +index fa06b41..ede52a8 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3067,7 +3067,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + Error **errp) + { + BlockdevCreateOptions *create_options = NULL; +- QDict *qdict = NULL; ++ QDict *qdict; + QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; +@@ -3140,14 +3140,13 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + + /* Now get the QAPI type BlockdevCreateOptions */ + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- qobject_unref(qdict); +- qdict = qobject_to(QDict, qobj); +- if (qdict == NULL) { ++ if (!qobj) { + ret = -EINVAL; + goto finish; + } + +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_keyval(qobj); ++ qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/qed.c b/block/qed.c +index d8810f5..3818888 100644 +--- a/block/qed.c ++++ b/block/qed.c +@@ -722,7 +722,7 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + Error **errp) + { + BlockdevCreateOptions *create_options = NULL; +- QDict *qdict = NULL; ++ QDict *qdict; + QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; +@@ -764,14 +764,13 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- qobject_unref(qdict); +- qdict = qobject_to(QDict, qobj); +- if (qdict == NULL) { ++ if (!qobj) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_keyval(qobj); ++ qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/vhdx.c b/block/vhdx.c +index 32939c4..728d8b3 100644 +--- a/block/vhdx.c ++++ b/block/vhdx.c +@@ -1963,7 +1963,7 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + Error **errp) + { + BlockdevCreateOptions *create_options = NULL; +- QDict *qdict = NULL; ++ QDict *qdict; + QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; +@@ -2004,14 +2004,13 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- qobject_unref(qdict); +- qdict = qobject_to(QDict, qobj); +- if (qdict == NULL) { ++ if (!qobj) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_keyval(qobj); ++ qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/vpc.c b/block/vpc.c +index 16178e5..a9bb041 100644 +--- a/block/vpc.c ++++ b/block/vpc.c +@@ -1081,7 +1081,7 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + QemuOpts *opts, Error **errp) + { + BlockdevCreateOptions *create_options = NULL; +- QDict *qdict = NULL; ++ QDict *qdict; + QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; +@@ -1120,14 +1120,13 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- qobject_unref(qdict); +- qdict = qobject_to(QDict, qobj); +- if (qdict == NULL) { ++ if (!qobj) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_keyval(qobj); ++ qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Convert-.bdrv_truncate-callback-to-coroutine_f.patch b/SOURCES/kvm-block-Convert-.bdrv_truncate-callback-to-coroutine_f.patch new file mode 100644 index 0000000..28c295c --- /dev/null +++ b/SOURCES/kvm-block-Convert-.bdrv_truncate-callback-to-coroutine_f.patch @@ -0,0 +1,866 @@ +From 655e1605b0ec1e798482e36ccfcf43cfb983c26e Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 14:42:54 +0200 +Subject: [PATCH 209/268] block: Convert .bdrv_truncate callback to + coroutine_fn + +RH-Author: Kevin Wolf +Message-id: <20180712144258.17303-3-kwolf@redhat.com> +Patchwork-id: 81327 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/6] block: Convert .bdrv_truncate callback to coroutine_fn +Bugzilla: 1595173 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: John Snow + +bdrv_truncate() is an operation that can block (even for a quite long +time, depending on the PreallocMode) in I/O paths that shouldn't block. +Convert it to a coroutine_fn so that we have the infrastructure for +drivers to make their .bdrv_co_truncate implementation asynchronous. + +This change could potentially introduce new race conditions because +bdrv_truncate() isn't necessarily executed atomically any more. Whether +this is a problem needs to be evaluated for each block driver that +supports truncate: + +* file-posix/win32, gluster, iscsi, nfs, rbd, ssh, sheepdog: The + protocol drivers are trivially safe because they don't actually yield + yet, so there is no change in behaviour. + +* copy-on-read, crypto, raw-format: Essentially just filter drivers that + pass the request to a child node, no problem. + +* qcow2: The implementation modifies metadata, so it needs to hold + s->lock to be safe with concurrent I/O requests. In order to avoid + double locking, this requires pulling the locking out into + preallocate_co() and using qcow2_write_caches() instead of + bdrv_flush(). + +* qed: Does a single header update, this is fine without locking. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 061ca8a368165fae300748c17971824a089f521f) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block.c | 63 +++++++++++++++++++++++++++++++++++----- + block/copy-on-read.c | 8 ++--- + block/crypto.c | 9 +++--- + block/file-posix.c | 12 ++++---- + block/file-win32.c | 6 ++-- + block/gluster.c | 14 +++++---- + block/iscsi.c | 8 ++--- + block/nfs.c | 7 +++-- + block/qcow2.c | 74 ++++++++++++++++++++++++++++------------------- + block/qed.c | 8 +++-- + block/raw-format.c | 8 ++--- + block/rbd.c | 8 +++-- + block/sheepdog.c | 12 ++++---- + block/ssh.c | 6 ++-- + include/block/block.h | 4 +++ + include/block/block_int.h | 4 +-- + 16 files changed, 162 insertions(+), 89 deletions(-) + +diff --git a/block.c b/block.c +index 0516284..0af08ca 100644 +--- a/block.c ++++ b/block.c +@@ -3738,8 +3738,8 @@ exit: + /** + * Truncate file to 'offset' bytes (needed only for file protocols) + */ +-int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, +- Error **errp) ++int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BlockDriverState *bs = child->bs; + BlockDriver *drv = bs->drv; +@@ -3757,23 +3757,28 @@ int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, + return -EINVAL; + } + +- if (!drv->bdrv_truncate) { ++ bdrv_inc_in_flight(bs); ++ ++ if (!drv->bdrv_co_truncate) { + if (bs->file && drv->is_filter) { +- return bdrv_truncate(bs->file, offset, prealloc, errp); ++ ret = bdrv_co_truncate(bs->file, offset, prealloc, errp); ++ goto out; + } + error_setg(errp, "Image format driver does not support resize"); +- return -ENOTSUP; ++ ret = -ENOTSUP; ++ goto out; + } + if (bs->read_only) { + error_setg(errp, "Image is read-only"); +- return -EACCES; ++ ret = -EACCES; ++ goto out; + } + + assert(!(bs->open_flags & BDRV_O_INACTIVE)); + +- ret = drv->bdrv_truncate(bs, offset, prealloc, errp); ++ ret = drv->bdrv_co_truncate(bs, offset, prealloc, errp); + if (ret < 0) { +- return ret; ++ goto out; + } + ret = refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); + if (ret < 0) { +@@ -3784,9 +3789,51 @@ int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, + bdrv_dirty_bitmap_truncate(bs, offset); + bdrv_parent_cb_resize(bs); + atomic_inc(&bs->write_gen); ++ ++out: ++ bdrv_dec_in_flight(bs); + return ret; + } + ++typedef struct TruncateCo { ++ BdrvChild *child; ++ int64_t offset; ++ PreallocMode prealloc; ++ Error **errp; ++ int ret; ++} TruncateCo; ++ ++static void coroutine_fn bdrv_truncate_co_entry(void *opaque) ++{ ++ TruncateCo *tco = opaque; ++ tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->prealloc, ++ tco->errp); ++} ++ ++int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, ++ Error **errp) ++{ ++ Coroutine *co; ++ TruncateCo tco = { ++ .child = child, ++ .offset = offset, ++ .prealloc = prealloc, ++ .errp = errp, ++ .ret = NOT_DONE, ++ }; ++ ++ if (qemu_in_coroutine()) { ++ /* Fast-path if already in coroutine context */ ++ bdrv_truncate_co_entry(&tco); ++ } else { ++ co = qemu_coroutine_create(bdrv_truncate_co_entry, &tco); ++ qemu_coroutine_enter(co); ++ BDRV_POLL_WHILE(child->bs, tco.ret == NOT_DONE); ++ } ++ ++ return tco.ret; ++} ++ + /** + * Length of a allocated file in bytes. Sparse files are counted by actual + * allocated space. Return < 0 if error or unknown. +diff --git a/block/copy-on-read.c b/block/copy-on-read.c +index 6a97208..1dcdaee 100644 +--- a/block/copy-on-read.c ++++ b/block/copy-on-read.c +@@ -80,10 +80,10 @@ static int64_t cor_getlength(BlockDriverState *bs) + } + + +-static int cor_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn cor_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { +- return bdrv_truncate(bs->file, offset, prealloc, errp); ++ return bdrv_co_truncate(bs->file, offset, prealloc, errp); + } + + +@@ -147,7 +147,7 @@ BlockDriver bdrv_copy_on_read = { + .bdrv_child_perm = cor_child_perm, + + .bdrv_getlength = cor_getlength, +- .bdrv_truncate = cor_truncate, ++ .bdrv_co_truncate = cor_co_truncate, + + .bdrv_co_preadv = cor_co_preadv, + .bdrv_co_pwritev = cor_co_pwritev, +diff --git a/block/crypto.c b/block/crypto.c +index f5151f4..02f04f3 100644 +--- a/block/crypto.c ++++ b/block/crypto.c +@@ -357,8 +357,9 @@ static int block_crypto_co_create_generic(BlockDriverState *bs, + return ret; + } + +-static int block_crypto_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn ++block_crypto_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BlockCrypto *crypto = bs->opaque; + uint64_t payload_offset = +@@ -371,7 +372,7 @@ static int block_crypto_truncate(BlockDriverState *bs, int64_t offset, + + offset += payload_offset; + +- return bdrv_truncate(bs->file, offset, prealloc, errp); ++ return bdrv_co_truncate(bs->file, offset, prealloc, errp); + } + + static void block_crypto_close(BlockDriverState *bs) +@@ -700,7 +701,7 @@ BlockDriver bdrv_crypto_luks = { + .bdrv_child_perm = bdrv_format_default_perms, + .bdrv_co_create = block_crypto_co_create_luks, + .bdrv_co_create_opts = block_crypto_co_create_opts_luks, +- .bdrv_truncate = block_crypto_truncate, ++ .bdrv_co_truncate = block_crypto_co_truncate, + .create_opts = &block_crypto_create_opts_luks, + + .bdrv_reopen_prepare = block_crypto_reopen_prepare, +diff --git a/block/file-posix.c b/block/file-posix.c +index cbf7c11..f8488ec 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -1832,8 +1832,8 @@ out: + return result; + } + +-static int raw_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BDRVRawState *s = bs->opaque; + struct stat st; +@@ -2474,7 +2474,7 @@ BlockDriver bdrv_file = { + .bdrv_io_plug = raw_aio_plug, + .bdrv_io_unplug = raw_aio_unplug, + +- .bdrv_truncate = raw_truncate, ++ .bdrv_co_truncate = raw_co_truncate, + .bdrv_getlength = raw_getlength, + .bdrv_get_info = raw_get_info, + .bdrv_get_allocated_file_size +@@ -2953,7 +2953,7 @@ static BlockDriver bdrv_host_device = { + .bdrv_io_plug = raw_aio_plug, + .bdrv_io_unplug = raw_aio_unplug, + +- .bdrv_truncate = raw_truncate, ++ .bdrv_co_truncate = raw_co_truncate, + .bdrv_getlength = raw_getlength, + .bdrv_get_info = raw_get_info, + .bdrv_get_allocated_file_size +@@ -3074,7 +3074,7 @@ static BlockDriver bdrv_host_cdrom = { + .bdrv_io_plug = raw_aio_plug, + .bdrv_io_unplug = raw_aio_unplug, + +- .bdrv_truncate = raw_truncate, ++ .bdrv_co_truncate = raw_co_truncate, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, + .bdrv_get_allocated_file_size +@@ -3204,7 +3204,7 @@ static BlockDriver bdrv_host_cdrom = { + .bdrv_io_plug = raw_aio_plug, + .bdrv_io_unplug = raw_aio_unplug, + +- .bdrv_truncate = raw_truncate, ++ .bdrv_co_truncate = raw_co_truncate, + .bdrv_getlength = raw_getlength, + .has_variable_length = true, + .bdrv_get_allocated_file_size +diff --git a/block/file-win32.c b/block/file-win32.c +index 2e2f746..6573338 100644 +--- a/block/file-win32.c ++++ b/block/file-win32.c +@@ -463,8 +463,8 @@ static void raw_close(BlockDriverState *bs) + } + } + +-static int raw_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BDRVRawState *s = bs->opaque; + LONG low, high; +@@ -636,7 +636,7 @@ BlockDriver bdrv_file = { + .bdrv_aio_writev = raw_aio_writev, + .bdrv_aio_flush = raw_aio_flush, + +- .bdrv_truncate = raw_truncate, ++ .bdrv_co_truncate = raw_co_truncate, + .bdrv_getlength = raw_getlength, + .bdrv_get_allocated_file_size + = raw_get_allocated_file_size, +diff --git a/block/gluster.c b/block/gluster.c +index 418bb73..cecfe09 100644 +--- a/block/gluster.c ++++ b/block/gluster.c +@@ -1177,8 +1177,10 @@ static coroutine_fn int qemu_gluster_co_rw(BlockDriverState *bs, + return acb.ret; + } + +-static int qemu_gluster_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static coroutine_fn int qemu_gluster_co_truncate(BlockDriverState *bs, ++ int64_t offset, ++ PreallocMode prealloc, ++ Error **errp) + { + BDRVGlusterState *s = bs->opaque; + return qemu_gluster_do_truncate(s->fd, offset, prealloc, errp); +@@ -1497,7 +1499,7 @@ static BlockDriver bdrv_gluster = { + .bdrv_co_create_opts = qemu_gluster_co_create_opts, + .bdrv_getlength = qemu_gluster_getlength, + .bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size, +- .bdrv_truncate = qemu_gluster_truncate, ++ .bdrv_co_truncate = qemu_gluster_co_truncate, + .bdrv_co_readv = qemu_gluster_co_readv, + .bdrv_co_writev = qemu_gluster_co_writev, + .bdrv_co_flush_to_disk = qemu_gluster_co_flush_to_disk, +@@ -1526,7 +1528,7 @@ static BlockDriver bdrv_gluster_tcp = { + .bdrv_co_create_opts = qemu_gluster_co_create_opts, + .bdrv_getlength = qemu_gluster_getlength, + .bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size, +- .bdrv_truncate = qemu_gluster_truncate, ++ .bdrv_co_truncate = qemu_gluster_co_truncate, + .bdrv_co_readv = qemu_gluster_co_readv, + .bdrv_co_writev = qemu_gluster_co_writev, + .bdrv_co_flush_to_disk = qemu_gluster_co_flush_to_disk, +@@ -1555,7 +1557,7 @@ static BlockDriver bdrv_gluster_unix = { + .bdrv_co_create_opts = qemu_gluster_co_create_opts, + .bdrv_getlength = qemu_gluster_getlength, + .bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size, +- .bdrv_truncate = qemu_gluster_truncate, ++ .bdrv_co_truncate = qemu_gluster_co_truncate, + .bdrv_co_readv = qemu_gluster_co_readv, + .bdrv_co_writev = qemu_gluster_co_writev, + .bdrv_co_flush_to_disk = qemu_gluster_co_flush_to_disk, +@@ -1590,7 +1592,7 @@ static BlockDriver bdrv_gluster_rdma = { + .bdrv_co_create_opts = qemu_gluster_co_create_opts, + .bdrv_getlength = qemu_gluster_getlength, + .bdrv_get_allocated_file_size = qemu_gluster_allocated_file_size, +- .bdrv_truncate = qemu_gluster_truncate, ++ .bdrv_co_truncate = qemu_gluster_co_truncate, + .bdrv_co_readv = qemu_gluster_co_readv, + .bdrv_co_writev = qemu_gluster_co_writev, + .bdrv_co_flush_to_disk = qemu_gluster_co_flush_to_disk, +diff --git a/block/iscsi.c b/block/iscsi.c +index 751884d..5047e83 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -2085,8 +2085,8 @@ static void iscsi_reopen_commit(BDRVReopenState *reopen_state) + } + } + +-static int iscsi_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn iscsi_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + IscsiLun *iscsilun = bs->opaque; + Error *local_err = NULL; +@@ -2431,7 +2431,7 @@ static BlockDriver bdrv_iscsi = { + + .bdrv_getlength = iscsi_getlength, + .bdrv_get_info = iscsi_get_info, +- .bdrv_truncate = iscsi_truncate, ++ .bdrv_co_truncate = iscsi_co_truncate, + .bdrv_refresh_limits = iscsi_refresh_limits, + + .bdrv_co_block_status = iscsi_co_block_status, +@@ -2468,7 +2468,7 @@ static BlockDriver bdrv_iser = { + + .bdrv_getlength = iscsi_getlength, + .bdrv_get_info = iscsi_get_info, +- .bdrv_truncate = iscsi_truncate, ++ .bdrv_co_truncate = iscsi_co_truncate, + .bdrv_refresh_limits = iscsi_refresh_limits, + + .bdrv_co_block_status = iscsi_co_block_status, +diff --git a/block/nfs.c b/block/nfs.c +index 743ca04..eab1a2c 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -743,8 +743,9 @@ static int64_t nfs_get_allocated_file_size(BlockDriverState *bs) + return (task.ret < 0 ? task.ret : st.st_blocks * 512); + } + +-static int nfs_file_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn ++nfs_file_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + NFSClient *client = bs->opaque; + int ret; +@@ -873,7 +874,7 @@ static BlockDriver bdrv_nfs = { + + .bdrv_has_zero_init = nfs_has_zero_init, + .bdrv_get_allocated_file_size = nfs_get_allocated_file_size, +- .bdrv_truncate = nfs_file_truncate, ++ .bdrv_co_truncate = nfs_file_co_truncate, + + .bdrv_file_open = nfs_file_open, + .bdrv_close = nfs_file_close, +diff --git a/block/qcow2.c b/block/qcow2.c +index dbd448c..c5c6ae9 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -2539,15 +2539,12 @@ static void coroutine_fn preallocate_co(void *opaque) + BlockDriverState *bs = params->bs; + uint64_t offset = params->offset; + uint64_t new_length = params->new_length; +- BDRVQcow2State *s = bs->opaque; + uint64_t bytes; + uint64_t host_offset = 0; + unsigned int cur_bytes; + int ret; + QCowL2Meta *meta; + +- qemu_co_mutex_lock(&s->lock); +- + assert(offset <= new_length); + bytes = new_length - offset; + +@@ -2600,7 +2597,6 @@ static void coroutine_fn preallocate_co(void *opaque) + ret = 0; + + done: +- qemu_co_mutex_unlock(&s->lock); + params->ret = ret; + } + +@@ -3037,7 +3033,11 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) + + /* And if we're supposed to preallocate metadata, do that now */ + if (qcow2_opts->preallocation != PREALLOC_MODE_OFF) { ++ BDRVQcow2State *s = blk_bs(blk)->opaque; ++ qemu_co_mutex_lock(&s->lock); + ret = preallocate(blk_bs(blk), 0, qcow2_opts->size); ++ qemu_co_mutex_unlock(&s->lock); ++ + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not preallocate metadata"); + goto out; +@@ -3434,8 +3434,8 @@ fail: + return ret; + } + +-static int qcow2_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BDRVQcow2State *s = bs->opaque; + uint64_t old_length; +@@ -3455,17 +3455,21 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + return -EINVAL; + } + ++ qemu_co_mutex_lock(&s->lock); ++ + /* cannot proceed if image has snapshots */ + if (s->nb_snapshots) { + error_setg(errp, "Can't resize an image which has snapshots"); +- return -ENOTSUP; ++ ret = -ENOTSUP; ++ goto fail; + } + + /* cannot proceed if image has bitmaps */ + if (s->nb_bitmaps) { + /* TODO: resize bitmaps in the image */ + error_setg(errp, "Can't resize an image which has bitmaps"); +- return -ENOTSUP; ++ ret = -ENOTSUP; ++ goto fail; + } + + old_length = bs->total_sectors * 512; +@@ -3476,7 +3480,8 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + if (prealloc != PREALLOC_MODE_OFF) { + error_setg(errp, + "Preallocation can't be used for shrinking an image"); +- return -EINVAL; ++ ret = -EINVAL; ++ goto fail; + } + + ret = qcow2_cluster_discard(bs, ROUND_UP(offset, s->cluster_size), +@@ -3485,40 +3490,42 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + QCOW2_DISCARD_ALWAYS, true); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to discard cropped clusters"); +- return ret; ++ goto fail; + } + + ret = qcow2_shrink_l1_table(bs, new_l1_size); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Failed to reduce the number of L2 tables"); +- return ret; ++ goto fail; + } + + ret = qcow2_shrink_reftable(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Failed to discard unused refblocks"); +- return ret; ++ goto fail; + } + + old_file_size = bdrv_getlength(bs->file->bs); + if (old_file_size < 0) { + error_setg_errno(errp, -old_file_size, + "Failed to inquire current file length"); +- return old_file_size; ++ ret = old_file_size; ++ goto fail; + } + last_cluster = qcow2_get_last_cluster(bs, old_file_size); + if (last_cluster < 0) { + error_setg_errno(errp, -last_cluster, + "Failed to find the last cluster"); +- return last_cluster; ++ ret = last_cluster; ++ goto fail; + } + if ((last_cluster + 1) * s->cluster_size < old_file_size) { + Error *local_err = NULL; + +- bdrv_truncate(bs->file, (last_cluster + 1) * s->cluster_size, +- PREALLOC_MODE_OFF, &local_err); ++ bdrv_co_truncate(bs->file, (last_cluster + 1) * s->cluster_size, ++ PREALLOC_MODE_OFF, &local_err); + if (local_err) { + warn_reportf_err(local_err, + "Failed to truncate the tail of the image: "); +@@ -3528,7 +3535,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + ret = qcow2_grow_l1_table(bs, new_l1_size, true); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to grow the L1 table"); +- return ret; ++ goto fail; + } + } + +@@ -3540,7 +3547,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + ret = preallocate(bs, old_length, offset); + if (ret < 0) { + error_setg_errno(errp, -ret, "Preallocation failed"); +- return ret; ++ goto fail; + } + break; + +@@ -3556,7 +3563,8 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + if (old_file_size < 0) { + error_setg_errno(errp, -old_file_size, + "Failed to inquire current file length"); +- return old_file_size; ++ ret = old_file_size; ++ goto fail; + } + old_file_size = ROUND_UP(old_file_size, s->cluster_size); + +@@ -3586,7 +3594,8 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + if (allocation_start < 0) { + error_setg_errno(errp, -allocation_start, + "Failed to resize refcount structures"); +- return allocation_start; ++ ret = allocation_start; ++ goto fail; + } + + clusters_allocated = qcow2_alloc_clusters_at(bs, allocation_start, +@@ -3594,7 +3603,8 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + if (clusters_allocated < 0) { + error_setg_errno(errp, -clusters_allocated, + "Failed to allocate data clusters"); +- return clusters_allocated; ++ ret = clusters_allocated; ++ goto fail; + } + + assert(clusters_allocated == nb_new_data_clusters); +@@ -3602,13 +3612,13 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + /* Allocate the data area */ + new_file_size = allocation_start + + nb_new_data_clusters * s->cluster_size; +- ret = bdrv_truncate(bs->file, new_file_size, prealloc, errp); ++ ret = bdrv_co_truncate(bs->file, new_file_size, prealloc, errp); + if (ret < 0) { + error_prepend(errp, "Failed to resize underlying file: "); + qcow2_free_clusters(bs, allocation_start, + nb_new_data_clusters * s->cluster_size, + QCOW2_DISCARD_OTHER); +- return ret; ++ goto fail; + } + + /* Create the necessary L2 entries */ +@@ -3631,7 +3641,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + qcow2_free_clusters(bs, host_offset, + nb_new_data_clusters * s->cluster_size, + QCOW2_DISCARD_OTHER); +- return ret; ++ goto fail; + } + + guest_offset += nb_clusters * s->cluster_size; +@@ -3647,11 +3657,11 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + + if (prealloc != PREALLOC_MODE_OFF) { + /* Flush metadata before actually changing the image size */ +- ret = bdrv_flush(bs); ++ ret = qcow2_write_caches(bs); + if (ret < 0) { + error_setg_errno(errp, -ret, + "Failed to flush the preallocated area to disk"); +- return ret; ++ goto fail; + } + } + +@@ -3661,11 +3671,14 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + &offset, sizeof(uint64_t)); + if (ret < 0) { + error_setg_errno(errp, -ret, "Failed to update the image size"); +- return ret; ++ goto fail; + } + + s->l1_vm_state_index = new_l1_size; +- return 0; ++ ret = 0; ++fail: ++ qemu_co_mutex_unlock(&s->lock); ++ return ret; + } + + /* XXX: put compressed sectors first, then all the cluster aligned +@@ -3689,7 +3702,8 @@ qcow2_co_pwritev_compressed(BlockDriverState *bs, uint64_t offset, + if (cluster_offset < 0) { + return cluster_offset; + } +- return bdrv_truncate(bs->file, cluster_offset, PREALLOC_MODE_OFF, NULL); ++ return bdrv_co_truncate(bs->file, cluster_offset, PREALLOC_MODE_OFF, ++ NULL); + } + + if (offset_into_cluster(s, offset)) { +@@ -4694,7 +4708,7 @@ BlockDriver bdrv_qcow2 = { + .bdrv_co_pdiscard = qcow2_co_pdiscard, + .bdrv_co_copy_range_from = qcow2_co_copy_range_from, + .bdrv_co_copy_range_to = qcow2_co_copy_range_to, +- .bdrv_truncate = qcow2_truncate, ++ .bdrv_co_truncate = qcow2_co_truncate, + .bdrv_co_pwritev_compressed = qcow2_co_pwritev_compressed, + .bdrv_make_empty = qcow2_make_empty, + +diff --git a/block/qed.c b/block/qed.c +index 4c5d7e8..34cccea 100644 +--- a/block/qed.c ++++ b/block/qed.c +@@ -1466,8 +1466,10 @@ static int coroutine_fn bdrv_qed_co_pwrite_zeroes(BlockDriverState *bs, + QED_AIOCB_WRITE | QED_AIOCB_ZERO); + } + +-static int bdrv_qed_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn bdrv_qed_co_truncate(BlockDriverState *bs, ++ int64_t offset, ++ PreallocMode prealloc, ++ Error **errp) + { + BDRVQEDState *s = bs->opaque; + uint64_t old_image_size; +@@ -1677,7 +1679,7 @@ static BlockDriver bdrv_qed = { + .bdrv_co_readv = bdrv_qed_co_readv, + .bdrv_co_writev = bdrv_qed_co_writev, + .bdrv_co_pwrite_zeroes = bdrv_qed_co_pwrite_zeroes, +- .bdrv_truncate = bdrv_qed_truncate, ++ .bdrv_co_truncate = bdrv_qed_co_truncate, + .bdrv_getlength = bdrv_qed_getlength, + .bdrv_get_info = bdrv_qed_get_info, + .bdrv_refresh_limits = bdrv_qed_refresh_limits, +diff --git a/block/raw-format.c b/block/raw-format.c +index f2e468d..b78da56 100644 +--- a/block/raw-format.c ++++ b/block/raw-format.c +@@ -366,8 +366,8 @@ static void raw_refresh_limits(BlockDriverState *bs, Error **errp) + } + } + +-static int raw_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BDRVRawState *s = bs->opaque; + +@@ -383,7 +383,7 @@ static int raw_truncate(BlockDriverState *bs, int64_t offset, + + s->size = offset; + offset += s->offset; +- return bdrv_truncate(bs->file, offset, prealloc, errp); ++ return bdrv_co_truncate(bs->file, offset, prealloc, errp); + } + + static void raw_eject(BlockDriverState *bs, bool eject_flag) +@@ -545,7 +545,7 @@ BlockDriver bdrv_raw = { + .bdrv_co_block_status = &raw_co_block_status, + .bdrv_co_copy_range_from = &raw_co_copy_range_from, + .bdrv_co_copy_range_to = &raw_co_copy_range_to, +- .bdrv_truncate = &raw_truncate, ++ .bdrv_co_truncate = &raw_co_truncate, + .bdrv_getlength = &raw_getlength, + .has_variable_length = true, + .bdrv_measure = &raw_measure, +diff --git a/block/rbd.c b/block/rbd.c +index 3242bcd..b93046b 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -987,8 +987,10 @@ static int64_t qemu_rbd_getlength(BlockDriverState *bs) + return info.size; + } + +-static int qemu_rbd_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn qemu_rbd_co_truncate(BlockDriverState *bs, ++ int64_t offset, ++ PreallocMode prealloc, ++ Error **errp) + { + BDRVRBDState *s = bs->opaque; + int r; +@@ -1180,7 +1182,7 @@ static BlockDriver bdrv_rbd = { + .bdrv_get_info = qemu_rbd_getinfo, + .create_opts = &qemu_rbd_create_opts, + .bdrv_getlength = qemu_rbd_getlength, +- .bdrv_truncate = qemu_rbd_truncate, ++ .bdrv_co_truncate = qemu_rbd_co_truncate, + .protocol_name = "rbd", + + .bdrv_aio_readv = qemu_rbd_aio_readv, +diff --git a/block/sheepdog.c b/block/sheepdog.c +index 933880c..14165d3 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -2294,8 +2294,8 @@ static int64_t sd_getlength(BlockDriverState *bs) + return s->inode.vdi_size; + } + +-static int sd_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn sd_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BDRVSheepdogState *s = bs->opaque; + int ret, fd; +@@ -2609,7 +2609,7 @@ static coroutine_fn int sd_co_writev(BlockDriverState *bs, int64_t sector_num, + BDRVSheepdogState *s = bs->opaque; + + if (offset > s->inode.vdi_size) { +- ret = sd_truncate(bs, offset, PREALLOC_MODE_OFF, NULL); ++ ret = sd_co_truncate(bs, offset, PREALLOC_MODE_OFF, NULL); + if (ret < 0) { + return ret; + } +@@ -3229,7 +3229,7 @@ static BlockDriver bdrv_sheepdog = { + .bdrv_has_zero_init = bdrv_has_zero_init_1, + .bdrv_getlength = sd_getlength, + .bdrv_get_allocated_file_size = sd_get_allocated_file_size, +- .bdrv_truncate = sd_truncate, ++ .bdrv_co_truncate = sd_co_truncate, + + .bdrv_co_readv = sd_co_readv, + .bdrv_co_writev = sd_co_writev, +@@ -3266,7 +3266,7 @@ static BlockDriver bdrv_sheepdog_tcp = { + .bdrv_has_zero_init = bdrv_has_zero_init_1, + .bdrv_getlength = sd_getlength, + .bdrv_get_allocated_file_size = sd_get_allocated_file_size, +- .bdrv_truncate = sd_truncate, ++ .bdrv_co_truncate = sd_co_truncate, + + .bdrv_co_readv = sd_co_readv, + .bdrv_co_writev = sd_co_writev, +@@ -3303,7 +3303,7 @@ static BlockDriver bdrv_sheepdog_unix = { + .bdrv_has_zero_init = bdrv_has_zero_init_1, + .bdrv_getlength = sd_getlength, + .bdrv_get_allocated_file_size = sd_get_allocated_file_size, +- .bdrv_truncate = sd_truncate, ++ .bdrv_co_truncate = sd_co_truncate, + + .bdrv_co_readv = sd_co_readv, + .bdrv_co_writev = sd_co_writev, +diff --git a/block/ssh.c b/block/ssh.c +index aab6996..6a55d82 100644 +--- a/block/ssh.c ++++ b/block/ssh.c +@@ -1241,8 +1241,8 @@ static int64_t ssh_getlength(BlockDriverState *bs) + return length; + } + +-static int ssh_truncate(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp) ++static int coroutine_fn ssh_co_truncate(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { + BDRVSSHState *s = bs->opaque; + +@@ -1277,7 +1277,7 @@ static BlockDriver bdrv_ssh = { + .bdrv_co_readv = ssh_co_readv, + .bdrv_co_writev = ssh_co_writev, + .bdrv_getlength = ssh_getlength, +- .bdrv_truncate = ssh_truncate, ++ .bdrv_co_truncate = ssh_co_truncate, + .bdrv_co_flush_to_disk = ssh_co_flush, + .create_opts = &ssh_create_opts, + }; +diff --git a/include/block/block.h b/include/block/block.h +index e677080..c3cfb40 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -300,8 +300,12 @@ int coroutine_fn bdrv_co_pwrite_zeroes(BdrvChild *child, int64_t offset, + BlockDriverState *bdrv_find_backing_image(BlockDriverState *bs, + const char *backing_file); + void bdrv_refresh_filename(BlockDriverState *bs); ++ ++int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, ++ PreallocMode prealloc, Error **errp); + int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, + Error **errp); ++ + int64_t bdrv_nb_sectors(BlockDriverState *bs); + int64_t bdrv_getlength(BlockDriverState *bs); + int64_t bdrv_get_allocated_file_size(BlockDriverState *bs); +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 3da86a7..46b2f87 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -291,8 +291,8 @@ struct BlockDriver { + * bdrv_parse_filename. + */ + const char *protocol_name; +- int (*bdrv_truncate)(BlockDriverState *bs, int64_t offset, +- PreallocMode prealloc, Error **errp); ++ int coroutine_fn (*bdrv_co_truncate)(BlockDriverState *bs, int64_t offset, ++ PreallocMode prealloc, Error **errp); + + int64_t (*bdrv_getlength)(BlockDriverState *bs); + bool has_variable_length; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Defer-.bdrv_drain_begin-callback-to-polling-ph.patch b/SOURCES/kvm-block-Defer-.bdrv_drain_begin-callback-to-polling-ph.patch new file mode 100644 index 0000000..396d8b8 --- /dev/null +++ b/SOURCES/kvm-block-Defer-.bdrv_drain_begin-callback-to-polling-ph.patch @@ -0,0 +1,83 @@ +From d62dd4f2f0b7acca3177935fbad1f1253ac722ef Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:43 +0100 +Subject: [PATCH 17/49] block: Defer .bdrv_drain_begin callback to polling + phase + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-5-kwolf@redhat.com> +Patchwork-id: 82594 +O-Subject: [RHEL-8 qemu-kvm PATCH 14/44] block: Defer .bdrv_drain_begin callback to polling phase +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +We cannot allow aio_poll() in bdrv_drain_invoke(begin=true) until we're +done with propagating the drain through the graph and are doing the +single final BDRV_POLL_WHILE(). + +Just schedule the coroutine with the callback and increase bs->in_flight +to make sure that the polling phase will wait for it. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 0109e7e6f83ae5166b81bbd9a4319d60be49985a) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 28 +++++++++++++++++++++++----- + 1 file changed, 23 insertions(+), 5 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 9c41ff9..23fe069 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -181,22 +181,40 @@ static void coroutine_fn bdrv_drain_invoke_entry(void *opaque) + + /* Set data->done before reading bs->wakeup. */ + atomic_mb_set(&data->done, true); +- bdrv_wakeup(bs); ++ bdrv_dec_in_flight(bs); ++ ++ if (data->begin) { ++ g_free(data); ++ } + } + + /* Recursively call BlockDriver.bdrv_co_drain_begin/end callbacks */ + static void bdrv_drain_invoke(BlockDriverState *bs, bool begin) + { +- BdrvCoDrainData data = { .bs = bs, .done = false, .begin = begin}; ++ BdrvCoDrainData *data; + + if (!bs->drv || (begin && !bs->drv->bdrv_co_drain_begin) || + (!begin && !bs->drv->bdrv_co_drain_end)) { + return; + } + +- data.co = qemu_coroutine_create(bdrv_drain_invoke_entry, &data); +- bdrv_coroutine_enter(bs, data.co); +- BDRV_POLL_WHILE(bs, !data.done); ++ data = g_new(BdrvCoDrainData, 1); ++ *data = (BdrvCoDrainData) { ++ .bs = bs, ++ .done = false, ++ .begin = begin ++ }; ++ ++ /* Make sure the driver callback completes during the polling phase for ++ * drain_begin. */ ++ bdrv_inc_in_flight(bs); ++ data->co = qemu_coroutine_create(bdrv_drain_invoke_entry, data); ++ aio_co_schedule(bdrv_get_aio_context(bs), data->co); ++ ++ if (!begin) { ++ BDRV_POLL_WHILE(bs, !data->done); ++ g_free(data); ++ } + } + + /* Returns true if BDRV_POLL_WHILE() should go into a blocking aio_poll() */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Don-t-inactivate-children-before-parents.patch b/SOURCES/kvm-block-Don-t-inactivate-children-before-parents.patch new file mode 100644 index 0000000..312328c --- /dev/null +++ b/SOURCES/kvm-block-Don-t-inactivate-children-before-parents.patch @@ -0,0 +1,182 @@ +From ff19fad07e21bac2534edd867120645fa2a0d7d3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Fri, 14 Dec 2018 09:49:45 +0000 +Subject: [PATCH 1/2] block: Don't inactivate children before parents + +RH-Author: Kevin Wolf +Message-id: <20181214094946.26226-2-kwolf@redhat.com> +Patchwork-id: 83511 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 1/2] block: Don't inactivate children before parents +Bugzilla: 1659395 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +bdrv_child_cb_inactivate() asserts that parents are already inactive +when children get inactivated. This precondition is necessary because +parents could still issue requests in their inactivation code. + +When block nodes are created individually with -blockdev, all of them +are monitor owned and will be returned by bdrv_next() in an undefined +order (in practice, in the order of their creation, which is usually +children before parents), which obviously fails the assertion: + +qemu: block.c:899: bdrv_child_cb_inactivate: Assertion `bs->open_flags & BDRV_O_INACTIVE' failed. + +This patch fixes the ordering by skipping nodes with still active +parents in bdrv_inactivate_recurse() because we know that they will be +covered by recursion when the last active parent becomes inactive. + +With the correct parents-before-children ordering, we also got rid of +the reason why commit aad0b7a0bfb introduced two passes, so we can go +back to a single-pass recursion. This is necessary so we can rely on the +BDRV_O_INACTIVE flag to skip nodes with active parents (the flag used +to be set only in pass 2, so we would always skip non-root nodes in +pass 1 because all parents would still be considered active; setting the +flag in pass 1 would mean, that we never skip anything in pass 2 because +all parents are already considered inactive). + +Because of the change to single pass, this patch is best reviewed with +whitespace changes ignored. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 9e37271f50ec2e95f299dc297ac08f9be0096b48) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 84 +++++++++++++++++++++++++++++++++++++++++------------------------ + 1 file changed, 53 insertions(+), 31 deletions(-) + +diff --git a/block.c b/block.c +index 18ae591..73f55a1 100644 +--- a/block.c ++++ b/block.c +@@ -4442,45 +4442,68 @@ void bdrv_invalidate_cache_all(Error **errp) + } + } + +-static int bdrv_inactivate_recurse(BlockDriverState *bs, +- bool setting_flag) ++static bool bdrv_has_bds_parent(BlockDriverState *bs, bool only_active) ++{ ++ BdrvChild *parent; ++ ++ QLIST_FOREACH(parent, &bs->parents, next_parent) { ++ if (parent->role->parent_is_bds) { ++ BlockDriverState *parent_bs = parent->opaque; ++ if (!only_active || !(parent_bs->open_flags & BDRV_O_INACTIVE)) { ++ return true; ++ } ++ } ++ } ++ ++ return false; ++} ++ ++static int bdrv_inactivate_recurse(BlockDriverState *bs) + { + BdrvChild *child, *parent; ++ uint64_t perm, shared_perm; + int ret; + + if (!bs->drv) { + return -ENOMEDIUM; + } + +- if (!setting_flag && bs->drv->bdrv_inactivate) { ++ /* Make sure that we don't inactivate a child before its parent. ++ * It will be covered by recursion from the yet active parent. */ ++ if (bdrv_has_bds_parent(bs, true)) { ++ return 0; ++ } ++ ++ assert(!(bs->open_flags & BDRV_O_INACTIVE)); ++ ++ /* Inactivate this node */ ++ if (bs->drv->bdrv_inactivate) { + ret = bs->drv->bdrv_inactivate(bs); + if (ret < 0) { + return ret; + } + } + +- if (setting_flag && !(bs->open_flags & BDRV_O_INACTIVE)) { +- uint64_t perm, shared_perm; +- +- QLIST_FOREACH(parent, &bs->parents, next_parent) { +- if (parent->role->inactivate) { +- ret = parent->role->inactivate(parent); +- if (ret < 0) { +- return ret; +- } ++ QLIST_FOREACH(parent, &bs->parents, next_parent) { ++ if (parent->role->inactivate) { ++ ret = parent->role->inactivate(parent); ++ if (ret < 0) { ++ return ret; + } + } ++ } + +- bs->open_flags |= BDRV_O_INACTIVE; ++ bs->open_flags |= BDRV_O_INACTIVE; ++ ++ /* Update permissions, they may differ for inactive nodes */ ++ bdrv_get_cumulative_perm(bs, &perm, &shared_perm); ++ bdrv_check_perm(bs, NULL, perm, shared_perm, NULL, &error_abort); ++ bdrv_set_perm(bs, perm, shared_perm); + +- /* Update permissions, they may differ for inactive nodes */ +- bdrv_get_cumulative_perm(bs, &perm, &shared_perm); +- bdrv_check_perm(bs, NULL, perm, shared_perm, NULL, &error_abort); +- bdrv_set_perm(bs, perm, shared_perm); +- } + ++ /* Recursively inactivate children */ + QLIST_FOREACH(child, &bs->children, next) { +- ret = bdrv_inactivate_recurse(child->bs, setting_flag); ++ ret = bdrv_inactivate_recurse(child->bs); + if (ret < 0) { + return ret; + } +@@ -4494,7 +4517,6 @@ int bdrv_inactivate_all(void) + BlockDriverState *bs = NULL; + BdrvNextIterator it; + int ret = 0; +- int pass; + GSList *aio_ctxs = NULL, *ctx; + + for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { +@@ -4506,17 +4528,17 @@ int bdrv_inactivate_all(void) + } + } + +- /* We do two passes of inactivation. The first pass calls to drivers' +- * .bdrv_inactivate callbacks recursively so all cache is flushed to disk; +- * the second pass sets the BDRV_O_INACTIVE flag so that no further write +- * is allowed. */ +- for (pass = 0; pass < 2; pass++) { +- for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { +- ret = bdrv_inactivate_recurse(bs, pass); +- if (ret < 0) { +- bdrv_next_cleanup(&it); +- goto out; +- } ++ for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { ++ /* Nodes with BDS parents are covered by recursion from the last ++ * parent that gets inactivated. Don't inactivate them a second ++ * time if that has already happened. */ ++ if (bdrv_has_bds_parent(bs, false)) { ++ continue; ++ } ++ ret = bdrv_inactivate_recurse(bs); ++ if (ret < 0) { ++ bdrv_next_cleanup(&it); ++ goto out; + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Don-t-manually-poll-in-bdrv_drain_all.patch b/SOURCES/kvm-block-Don-t-manually-poll-in-bdrv_drain_all.patch new file mode 100644 index 0000000..7d793d4 --- /dev/null +++ b/SOURCES/kvm-block-Don-t-manually-poll-in-bdrv_drain_all.patch @@ -0,0 +1,113 @@ +From 6fe050124f9433521227b5da8f560c7a3c248513 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:38 +0100 +Subject: [PATCH 07/49] block: Don't manually poll in bdrv_drain_all() + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-5-kwolf@redhat.com> +Patchwork-id: 82584 +O-Subject: [RHEL-8 qemu-kvm PATCH 04/44] block: Don't manually poll in bdrv_drain_all() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +All involved nodes are already idle, we called bdrv_do_drain_begin() on +them. + +The comment in the code suggested that this was not correct because the +completion of a request on one node could spawn a new request on a +different node (which might have been drained before, so we wouldn't +drain the new request). In reality, new requests to different nodes +aren't spawned out of nothing, but only in the context of a parent +request, and they aren't submitted to random nodes, but only to child +nodes. As long as we still poll for the completion of the parent request +(which we do), draining each root node separately is good enough. + +Remove the additional polling code from bdrv_drain_all_begin() and +replace it with an assertion that all nodes are already idle after we +drained them separately. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit c13ad59f012cbbccb866a10477458e69bc868dbb) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 41 ++++++++++++----------------------------- + 1 file changed, 12 insertions(+), 29 deletions(-) + +diff --git a/block/io.c b/block/io.c +index aa41f1e..e5fc42c 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -376,6 +376,16 @@ void bdrv_drain(BlockDriverState *bs) + bdrv_drained_end(bs); + } + ++static void bdrv_drain_assert_idle(BlockDriverState *bs) ++{ ++ BdrvChild *child, *next; ++ ++ assert(atomic_read(&bs->in_flight) == 0); ++ QLIST_FOREACH_SAFE(child, &bs->children, next, next) { ++ bdrv_drain_assert_idle(child->bs); ++ } ++} ++ + /* + * Wait for pending requests to complete across all BlockDriverStates + * +@@ -390,11 +400,8 @@ void bdrv_drain(BlockDriverState *bs) + */ + void bdrv_drain_all_begin(void) + { +- /* Always run first iteration so any pending completion BHs run */ +- bool waited = true; + BlockDriverState *bs; + BdrvNextIterator it; +- GSList *aio_ctxs = NULL, *ctx; + + /* BDRV_POLL_WHILE() for a node can only be called from its own I/O thread + * or the main loop AioContext. We potentially use BDRV_POLL_WHILE() on +@@ -408,35 +415,11 @@ void bdrv_drain_all_begin(void) + aio_context_acquire(aio_context); + bdrv_do_drained_begin(bs, true, NULL); + aio_context_release(aio_context); +- +- if (!g_slist_find(aio_ctxs, aio_context)) { +- aio_ctxs = g_slist_prepend(aio_ctxs, aio_context); +- } + } + +- /* Note that completion of an asynchronous I/O operation can trigger any +- * number of other I/O operations on other devices---for example a +- * coroutine can submit an I/O request to another device in response to +- * request completion. Therefore we must keep looping until there was no +- * more activity rather than simply draining each device independently. +- */ +- while (waited) { +- waited = false; +- +- for (ctx = aio_ctxs; ctx != NULL; ctx = ctx->next) { +- AioContext *aio_context = ctx->data; +- +- aio_context_acquire(aio_context); +- for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { +- if (aio_context == bdrv_get_aio_context(bs)) { +- waited |= bdrv_drain_recurse(bs); +- } +- } +- aio_context_release(aio_context); +- } ++ for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { ++ bdrv_drain_assert_idle(bs); + } +- +- g_slist_free(aio_ctxs); + } + + void bdrv_drain_all_end(void) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Don-t-poll-in-parent-drain-callbacks.patch b/SOURCES/kvm-block-Don-t-poll-in-parent-drain-callbacks.patch new file mode 100644 index 0000000..b43741a --- /dev/null +++ b/SOURCES/kvm-block-Don-t-poll-in-parent-drain-callbacks.patch @@ -0,0 +1,112 @@ +From c56d4b513e1a0b868b788ea495857f41cde1acf6 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:41 +0100 +Subject: [PATCH 15/49] block: Don't poll in parent drain callbacks + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-3-kwolf@redhat.com> +Patchwork-id: 82592 +O-Subject: [RHEL-8 qemu-kvm PATCH 12/44] block: Don't poll in parent drain callbacks +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +bdrv_do_drained_begin() is only safe if we have a single +BDRV_POLL_WHILE() after quiescing all affected nodes. We cannot allow +that parent callbacks introduce a nested polling loop that could cause +graph changes while we're traversing the graph. + +Split off bdrv_do_drained_begin_quiesce(), which only quiesces a single +node without waiting for its requests to complete. These requests will +be waited for in the BDRV_POLL_WHILE() call down the call chain. + +Signed-off-by: Kevin Wolf +(cherry picked from commit dcf94a23b1add0f856db51e9ff5ba0774e096076) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 2 +- + block/io.c | 24 ++++++++++++++++-------- + include/block/block.h | 9 +++++++++ + 3 files changed, 26 insertions(+), 9 deletions(-) + +diff --git a/block.c b/block.c +index 9afaf26..5a50de6 100644 +--- a/block.c ++++ b/block.c +@@ -818,7 +818,7 @@ static char *bdrv_child_get_parent_desc(BdrvChild *c) + static void bdrv_child_cb_drained_begin(BdrvChild *child) + { + BlockDriverState *bs = child->opaque; +- bdrv_drained_begin(bs); ++ bdrv_do_drained_begin_quiesce(bs, NULL); + } + + static bool bdrv_child_cb_drained_poll(BdrvChild *child) +diff --git a/block/io.c b/block/io.c +index 442ded1..9c41ff9 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -285,15 +285,10 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + assert(data.done); + } + +-void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, +- BdrvChild *parent, bool poll) ++void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, ++ BdrvChild *parent) + { +- BdrvChild *child, *next; +- +- if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(bs, true, recursive, parent, poll); +- return; +- } ++ assert(!qemu_in_coroutine()); + + /* Stop things in parent-to-child order */ + if (atomic_fetch_inc(&bs->quiesce_counter) == 0) { +@@ -302,6 +297,19 @@ void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + + bdrv_parent_drained_begin(bs, parent); + bdrv_drain_invoke(bs, true); ++} ++ ++static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, ++ BdrvChild *parent, bool poll) ++{ ++ BdrvChild *child, *next; ++ ++ if (qemu_in_coroutine()) { ++ bdrv_co_yield_to_drain(bs, true, recursive, parent, poll); ++ return; ++ } ++ ++ bdrv_do_drained_begin_quiesce(bs, parent); + + if (recursive) { + bs->recursive_quiesce_counter++; +diff --git a/include/block/block.h b/include/block/block.h +index 2bbea7c..43f29b5 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -619,6 +619,15 @@ bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, + void bdrv_drained_begin(BlockDriverState *bs); + + /** ++ * bdrv_do_drained_begin_quiesce: ++ * ++ * Quiesces a BDS like bdrv_drained_begin(), but does not wait for already ++ * running requests to complete. ++ */ ++void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, ++ BdrvChild *parent); ++ ++/** + * Like bdrv_drained_begin, but recursively begins a quiesced section for + * exclusive access to all child nodes as well. + */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Don-t-silently-truncate-node-names.patch b/SOURCES/kvm-block-Don-t-silently-truncate-node-names.patch new file mode 100644 index 0000000..6f7fed0 --- /dev/null +++ b/SOURCES/kvm-block-Don-t-silently-truncate-node-names.patch @@ -0,0 +1,148 @@ +From 1920bc35d149d2b28658e5354468df407c9799f1 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 5 Jul 2018 16:47:51 +0200 +Subject: [PATCH 190/268] block: Don't silently truncate node names + +RH-Author: Kevin Wolf +Message-id: <20180705164751.15271-2-kwolf@redhat.com> +Patchwork-id: 81234 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/1] block: Don't silently truncate node names +Bugzilla: 1549654 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow + +If the user passes a too long node name string, we silently truncate it +to fit into BlockDriverState.node_name, i.e. to 31 characters. Apart +from surprising the user when the node has a different name than +requested, this also bypasses the check for duplicate names, so that the +same name can be assigned to multiple nodes. + +Fix this by just making too long node names an error. + +Reported-by: Peter Krempa +Signed-off-by: Kevin Wolf +(cherry picked from commit 824808dd77821ceba05357cb1ee4069a6a95bebd) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block.c | 6 ++++++ + tests/qemu-iotests/051 | 15 +++++++++++++++ + tests/qemu-iotests/051.out | 23 +++++++++++++++++++++++ + tests/qemu-iotests/051.pc.out | 23 +++++++++++++++++++++++ + 4 files changed, 67 insertions(+) + +diff --git a/block.c b/block.c +index afe30ca..0516284 100644 +--- a/block.c ++++ b/block.c +@@ -1140,6 +1140,12 @@ static void bdrv_assign_node_name(BlockDriverState *bs, + goto out; + } + ++ /* Make sure that the node name isn't truncated */ ++ if (strlen(node_name) >= sizeof(bs->node_name)) { ++ error_setg(errp, "Node name too long"); ++ goto out; ++ } ++ + /* copy node name into the bs and insert it into the graph list */ + pstrcpy(bs->node_name, sizeof(bs->node_name), node_name); + QTAILQ_INSERT_TAIL(&graph_bdrv_states, bs, node_list); +diff --git a/tests/qemu-iotests/051 b/tests/qemu-iotests/051 +index 69d34eb..c5cc0ee 100755 +--- a/tests/qemu-iotests/051 ++++ b/tests/qemu-iotests/051 +@@ -100,6 +100,21 @@ run_qemu -drive file="$TEST_IMG",driver=raw,format=qcow2 + run_qemu -drive file="$TEST_IMG",driver=qcow2,format=qcow2 + + echo ++echo === Node names === ++echo ++ ++# Maximum length: 31 characters ++run_qemu -drive file="$TEST_IMG",node-name=x123456789012345678901234567890 ++run_qemu -drive file="$TEST_IMG",node-name=x1234567890123456789012345678901 ++ ++# First character must be alphabetic ++# Following characters alphanumeric or -._ ++run_qemu -drive file="$TEST_IMG",node-name=All-Types.of_all0wed_chars ++run_qemu -drive file="$TEST_IMG",node-name=123foo ++run_qemu -drive file="$TEST_IMG",node-name=_foo ++run_qemu -drive file="$TEST_IMG",node-name=foo#12 ++ ++echo + echo === Device without drive === + echo + +diff --git a/tests/qemu-iotests/051.out b/tests/qemu-iotests/051.out +index dd9846d..b727350 100644 +--- a/tests/qemu-iotests/051.out ++++ b/tests/qemu-iotests/051.out +@@ -47,6 +47,29 @@ Testing: -drive file=TEST_DIR/t.qcow2,driver=qcow2,format=qcow2 + QEMU_PROG: -drive file=TEST_DIR/t.qcow2,driver=qcow2,format=qcow2: Cannot specify both 'driver' and 'format' + + ++=== Node names === ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=x123456789012345678901234567890 ++QEMU X.Y.Z monitor - type 'help' for more information ++(qemu) quit ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=x1234567890123456789012345678901 ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=x1234567890123456789012345678901: Node name too long ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=All-Types.of_all0wed_chars ++QEMU X.Y.Z monitor - type 'help' for more information ++(qemu) quit ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=123foo ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=123foo: Invalid node name ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=_foo ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=_foo: Invalid node name ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=foo#12 ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=foo#12: Invalid node name ++ ++ + === Device without drive === + + Testing: -device VIRTIO_SCSI -device scsi-hd +diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out +index b01f9a9..e9257fe 100644 +--- a/tests/qemu-iotests/051.pc.out ++++ b/tests/qemu-iotests/051.pc.out +@@ -47,6 +47,29 @@ Testing: -drive file=TEST_DIR/t.qcow2,driver=qcow2,format=qcow2 + QEMU_PROG: -drive file=TEST_DIR/t.qcow2,driver=qcow2,format=qcow2: Cannot specify both 'driver' and 'format' + + ++=== Node names === ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=x123456789012345678901234567890 ++QEMU X.Y.Z monitor - type 'help' for more information ++(qemu) quit ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=x1234567890123456789012345678901 ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=x1234567890123456789012345678901: Node name too long ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=All-Types.of_all0wed_chars ++QEMU X.Y.Z monitor - type 'help' for more information ++(qemu) quit ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=123foo ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=123foo: Invalid node name ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=_foo ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=_foo: Invalid node name ++ ++Testing: -drive file=TEST_DIR/t.qcow2,node-name=foo#12 ++QEMU_PROG: -drive file=TEST_DIR/t.qcow2,node-name=foo#12: Invalid node name ++ ++ + === Device without drive === + + Testing: -device VIRTIO_SCSI -device scsi-hd +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Drain-recursively-with-a-single-BDRV_POLL_WHIL.patch b/SOURCES/kvm-block-Drain-recursively-with-a-single-BDRV_POLL_WHIL.patch new file mode 100644 index 0000000..a9ab5ce --- /dev/null +++ b/SOURCES/kvm-block-Drain-recursively-with-a-single-BDRV_POLL_WHIL.patch @@ -0,0 +1,242 @@ +From 013989ed30ff32eedbb1a6767f0c38c85150f951 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:39 +0100 +Subject: [PATCH 13/49] block: Drain recursively with a single + BDRV_POLL_WHILE() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-1-kwolf@redhat.com> +Patchwork-id: 82590 +O-Subject: [RHEL-8 qemu-kvm PATCH 10/44] block: Drain recursively with a single BDRV_POLL_WHILE() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Anything can happen inside BDRV_POLL_WHILE(), including graph +changes that may interfere with its callers (e.g. child list iteration +in recursive callers of bdrv_do_drained_begin). + +Switch to a single BDRV_POLL_WHILE() call for the whole subtree at the +end of bdrv_do_drained_begin() to avoid such effects. The recursion +happens now inside the loop condition. As the graph can only change +between bdrv_drain_poll() calls, but not inside of it, doing the +recursion here is safe. + +Signed-off-by: Kevin Wolf +(cherry picked from commit fe4f0614ef9e361dae12012d3c400657444836cf) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 2 +- + block/io.c | 63 ++++++++++++++++++++++++++++++++++++--------------- + include/block/block.h | 9 +++++--- + 3 files changed, 52 insertions(+), 22 deletions(-) + +diff --git a/block.c b/block.c +index 0d9698a..9afaf26 100644 +--- a/block.c ++++ b/block.c +@@ -824,7 +824,7 @@ static void bdrv_child_cb_drained_begin(BdrvChild *child) + static bool bdrv_child_cb_drained_poll(BdrvChild *child) + { + BlockDriverState *bs = child->opaque; +- return bdrv_drain_poll(bs, NULL); ++ return bdrv_drain_poll(bs, false, NULL); + } + + static void bdrv_child_cb_drained_end(BdrvChild *child) +diff --git a/block/io.c b/block/io.c +index a0e3461..442ded1 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -164,6 +164,7 @@ typedef struct { + bool done; + bool begin; + bool recursive; ++ bool poll; + BdrvChild *parent; + } BdrvCoDrainData; + +@@ -199,27 +200,42 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin) + } + + /* Returns true if BDRV_POLL_WHILE() should go into a blocking aio_poll() */ +-bool bdrv_drain_poll(BlockDriverState *bs, BdrvChild *ignore_parent) ++bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, ++ BdrvChild *ignore_parent) + { ++ BdrvChild *child, *next; ++ + if (bdrv_parent_drained_poll(bs, ignore_parent)) { + return true; + } + +- return atomic_read(&bs->in_flight); ++ if (atomic_read(&bs->in_flight)) { ++ return true; ++ } ++ ++ if (recursive) { ++ QLIST_FOREACH_SAFE(child, &bs->children, next, next) { ++ if (bdrv_drain_poll(child->bs, recursive, child)) { ++ return true; ++ } ++ } ++ } ++ ++ return false; + } + +-static bool bdrv_drain_poll_top_level(BlockDriverState *bs, ++static bool bdrv_drain_poll_top_level(BlockDriverState *bs, bool recursive, + BdrvChild *ignore_parent) + { + /* Execute pending BHs first and check everything else only after the BHs + * have executed. */ + while (aio_poll(bs->aio_context, false)); + +- return bdrv_drain_poll(bs, ignore_parent); ++ return bdrv_drain_poll(bs, recursive, ignore_parent); + } + + static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, +- BdrvChild *parent); ++ BdrvChild *parent, bool poll); + static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, + BdrvChild *parent); + +@@ -231,7 +247,7 @@ static void bdrv_co_drain_bh_cb(void *opaque) + + bdrv_dec_in_flight(bs); + if (data->begin) { +- bdrv_do_drained_begin(bs, data->recursive, data->parent); ++ bdrv_do_drained_begin(bs, data->recursive, data->parent, data->poll); + } else { + bdrv_do_drained_end(bs, data->recursive, data->parent); + } +@@ -242,7 +258,7 @@ static void bdrv_co_drain_bh_cb(void *opaque) + + static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + bool begin, bool recursive, +- BdrvChild *parent) ++ BdrvChild *parent, bool poll) + { + BdrvCoDrainData data; + +@@ -257,6 +273,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + .begin = begin, + .recursive = recursive, + .parent = parent, ++ .poll = poll, + }; + bdrv_inc_in_flight(bs); + aio_bh_schedule_oneshot(bdrv_get_aio_context(bs), +@@ -269,12 +286,12 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + } + + void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, +- BdrvChild *parent) ++ BdrvChild *parent, bool poll) + { + BdrvChild *child, *next; + + if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(bs, true, recursive, parent); ++ bdrv_co_yield_to_drain(bs, true, recursive, parent, poll); + return; + } + +@@ -286,25 +303,35 @@ void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + bdrv_parent_drained_begin(bs, parent); + bdrv_drain_invoke(bs, true); + +- /* Wait for drained requests to finish */ +- BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, parent)); +- + if (recursive) { + bs->recursive_quiesce_counter++; + QLIST_FOREACH_SAFE(child, &bs->children, next, next) { +- bdrv_do_drained_begin(child->bs, true, child); ++ bdrv_do_drained_begin(child->bs, true, child, false); + } + } ++ ++ /* ++ * Wait for drained requests to finish. ++ * ++ * Calling BDRV_POLL_WHILE() only once for the top-level node is okay: The ++ * call is needed so things in this AioContext can make progress even ++ * though we don't return to the main AioContext loop - this automatically ++ * includes other nodes in the same AioContext and therefore all child ++ * nodes. ++ */ ++ if (poll) { ++ BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, recursive, parent)); ++ } + } + + void bdrv_drained_begin(BlockDriverState *bs) + { +- bdrv_do_drained_begin(bs, false, NULL); ++ bdrv_do_drained_begin(bs, false, NULL, true); + } + + void bdrv_subtree_drained_begin(BlockDriverState *bs) + { +- bdrv_do_drained_begin(bs, true, NULL); ++ bdrv_do_drained_begin(bs, true, NULL, true); + } + + void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, +@@ -314,7 +341,7 @@ void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, + int old_quiesce_counter; + + if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(bs, false, recursive, parent); ++ bdrv_co_yield_to_drain(bs, false, recursive, parent, false); + return; + } + assert(bs->quiesce_counter > 0); +@@ -350,7 +377,7 @@ void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_parent) + int i; + + for (i = 0; i < new_parent->recursive_quiesce_counter; i++) { +- bdrv_do_drained_begin(child->bs, true, child); ++ bdrv_do_drained_begin(child->bs, true, child, true); + } + } + +@@ -420,7 +447,7 @@ void bdrv_drain_all_begin(void) + AioContext *aio_context = bdrv_get_aio_context(bs); + + aio_context_acquire(aio_context); +- bdrv_do_drained_begin(bs, true, NULL); ++ bdrv_do_drained_begin(bs, true, NULL, true); + aio_context_release(aio_context); + } + +diff --git a/include/block/block.h b/include/block/block.h +index 8c91d4c..2bbea7c 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -598,10 +598,13 @@ void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore); + /** + * bdrv_drain_poll: + * +- * Poll for pending requests in @bs and its parents (except for +- * @ignore_parent). This is part of bdrv_drained_begin. ++ * Poll for pending requests in @bs, its parents (except for @ignore_parent), ++ * and if @recursive is true its children as well. ++ * ++ * This is part of bdrv_drained_begin. + */ +-bool bdrv_drain_poll(BlockDriverState *bs, BdrvChild *ignore_parent); ++bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, ++ BdrvChild *ignore_parent); + + /** + * bdrv_drained_begin: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Factor-out-qobject_input_visitor_new_flat_conf.patch b/SOURCES/kvm-block-Factor-out-qobject_input_visitor_new_flat_conf.patch new file mode 100644 index 0000000..ee7f8fa --- /dev/null +++ b/SOURCES/kvm-block-Factor-out-qobject_input_visitor_new_flat_conf.patch @@ -0,0 +1,465 @@ +From ec12a6f04d0ecf97a8e200d360f3375def1433d5 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:20 +0200 +Subject: [PATCH 022/268] block: Factor out + qobject_input_visitor_new_flat_confused() + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-14-armbru@redhat.com> +Patchwork-id: 80728 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 13/23] block: Factor out qobject_input_visitor_new_flat_confused() +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit af91062ee1408f7f5bb58389d355d29a5040c648) +Signed-off-by: Miroslav Rezanina +--- + block/nbd.c | 7 ++----- + block/nfs.c | 7 ++----- + block/parallels.c | 7 ++----- + block/qcow.c | 7 ++----- + block/qcow2.c | 7 ++----- + block/qed.c | 7 ++----- + block/rbd.c | 7 ++----- + block/sheepdog.c | 14 ++++---------- + block/ssh.c | 7 ++----- + block/vhdx.c | 7 ++----- + block/vpc.c | 7 ++----- + include/block/qdict.h | 3 ++- + qobject/block-qdict.c | 28 +++++++++++++++++++++++++++- + 13 files changed, 53 insertions(+), 62 deletions(-) + +diff --git a/block/nbd.c b/block/nbd.c +index b0be357..10912c3 100644 +--- a/block/nbd.c ++++ b/block/nbd.c +@@ -263,7 +263,6 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, + { + SocketAddress *saddr = NULL; + QDict *addr = NULL; +- QObject *crumpled_addr = NULL; + Visitor *iv = NULL; + Error *local_err = NULL; + +@@ -273,12 +272,11 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, + goto done; + } + +- crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp); +- if (!crumpled_addr) { ++ iv = qobject_input_visitor_new_flat_confused(addr, errp); ++ if (!iv) { + goto done; + } + +- iv = qobject_input_visitor_new_keyval(crumpled_addr); + visit_type_SocketAddress(iv, NULL, &saddr, &local_err); + if (local_err) { + error_propagate(errp, local_err); +@@ -287,7 +285,6 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, + + done: + qobject_unref(addr); +- qobject_unref(crumpled_addr); + visit_free(iv); + return saddr; + } +diff --git a/block/nfs.c b/block/nfs.c +index 4090d28..1e12958 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -556,19 +556,16 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, + Error **errp) + { + BlockdevOptionsNfs *opts = NULL; +- QObject *crumpled = NULL; + Visitor *v; + Error *local_err = NULL; + +- crumpled = qdict_crumple_for_keyval_qiv(options, errp); +- if (crumpled == NULL) { ++ v = qobject_input_visitor_new_flat_confused(options, errp); ++ if (!v) { + return NULL; + } + +- v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevOptionsNfs(v, NULL, &opts, &local_err); + visit_free(v); +- qobject_unref(crumpled); + + if (local_err) { + return NULL; +diff --git a/block/parallels.c b/block/parallels.c +index 1c96c39..b6ebe36 100644 +--- a/block/parallels.c ++++ b/block/parallels.c +@@ -615,7 +615,6 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + Error *local_err = NULL; + BlockDriverState *bs = NULL; + QDict *qdict; +- QObject *qobj; + Visitor *v; + int ret; + +@@ -651,14 +650,12 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "parallels"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (!qobj) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto done; + } + +- v = qobject_input_visitor_new_keyval(qobj); +- qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/qcow.c b/block/qcow.c +index 43a595a..4b3bc31 100644 +--- a/block/qcow.c ++++ b/block/qcow.c +@@ -945,7 +945,6 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + BlockdevCreateOptions *create_options = NULL; + BlockDriverState *bs = NULL; + QDict *qdict; +- QObject *qobj; + Visitor *v; + const char *val; + Error *local_err = NULL; +@@ -995,14 +994,12 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "qcow"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (!qobj) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(qobj); +- qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/qcow2.c b/block/qcow2.c +index ede52a8..35842c5 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3068,7 +3068,6 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + { + BlockdevCreateOptions *create_options = NULL; + QDict *qdict; +- QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; + Error *local_err = NULL; +@@ -3139,14 +3138,12 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + qdict_put_str(qdict, "file", bs->node_name); + + /* Now get the QAPI type BlockdevCreateOptions */ +- qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (!qobj) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto finish; + } + +- v = qobject_input_visitor_new_keyval(qobj); +- qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/qed.c b/block/qed.c +index 3818888..4c5d7e8 100644 +--- a/block/qed.c ++++ b/block/qed.c +@@ -723,7 +723,6 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + { + BlockdevCreateOptions *create_options = NULL; + QDict *qdict; +- QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; + Error *local_err = NULL; +@@ -763,14 +762,12 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "qed"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (!qobj) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(qobj); +- qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/rbd.c b/block/rbd.c +index 0b5455f..c834d72 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -623,7 +623,6 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + BDRVRBDState *s = bs->opaque; + BlockdevOptionsRbd *opts = NULL; + Visitor *v; +- QObject *crumpled = NULL; + const QDictEntry *e; + Error *local_err = NULL; + char *keypairs, *secretid; +@@ -640,16 +639,14 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + } + + /* Convert the remaining options into a QAPI object */ +- crumpled = qdict_crumple_for_keyval_qiv(options, errp); +- if (crumpled == NULL) { ++ v = qobject_input_visitor_new_flat_confused(options, errp); ++ if (!v) { + r = -EINVAL; + goto out; + } + +- v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevOptionsRbd(v, NULL, &opts, &local_err); + visit_free(v); +- qobject_unref(crumpled); + + if (local_err) { + error_propagate(errp, local_err); +diff --git a/block/sheepdog.c b/block/sheepdog.c +index dd582d5..d1c9bf5 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -539,19 +539,17 @@ static void sd_aio_setup(SheepdogAIOCB *acb, BDRVSheepdogState *s, + static SocketAddress *sd_server_config(QDict *options, Error **errp) + { + QDict *server = NULL; +- QObject *crumpled_server = NULL; + Visitor *iv = NULL; + SocketAddress *saddr = NULL; + Error *local_err = NULL; + + qdict_extract_subqdict(options, &server, "server."); + +- crumpled_server = qdict_crumple_for_keyval_qiv(server, errp); +- if (!crumpled_server) { ++ iv = qobject_input_visitor_new_flat_confused(server, errp); ++ if (!iv) { + goto done; + } + +- iv = qobject_input_visitor_new_keyval(crumpled_server); + visit_type_SocketAddress(iv, NULL, &saddr, &local_err); + if (local_err) { + error_propagate(errp, local_err); +@@ -560,7 +558,6 @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) + + done: + visit_free(iv); +- qobject_unref(crumpled_server); + qobject_unref(server); + return saddr; + } +@@ -2174,7 +2171,6 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + { + BlockdevCreateOptions *create_options = NULL; + QDict *qdict, *location_qdict; +- QObject *crumpled; + Visitor *v; + const char *redundancy; + Error *local_err = NULL; +@@ -2210,16 +2206,14 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + } + + /* Get the QAPI object */ +- crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (crumpled == NULL) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); +- qobject_unref(crumpled); + + if (local_err) { + error_propagate(errp, local_err); +diff --git a/block/ssh.c b/block/ssh.c +index 2fc7cd9..aab6996 100644 +--- a/block/ssh.c ++++ b/block/ssh.c +@@ -606,7 +606,6 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) + BlockdevOptionsSsh *result = NULL; + QemuOpts *opts = NULL; + Error *local_err = NULL; +- QObject *crumpled; + const QDictEntry *e; + Visitor *v; + +@@ -623,15 +622,13 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) + } + + /* Create the QAPI object */ +- crumpled = qdict_crumple_for_keyval_qiv(options, errp); +- if (crumpled == NULL) { ++ v = qobject_input_visitor_new_flat_confused(options, errp); ++ if (!v) { + goto fail; + } + +- v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); + visit_free(v); +- qobject_unref(crumpled); + + if (local_err) { + error_propagate(errp, local_err); +diff --git a/block/vhdx.c b/block/vhdx.c +index 728d8b3..6731298 100644 +--- a/block/vhdx.c ++++ b/block/vhdx.c +@@ -1964,7 +1964,6 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + { + BlockdevCreateOptions *create_options = NULL; + QDict *qdict; +- QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; + Error *local_err = NULL; +@@ -2003,14 +2002,12 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "vhdx"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (!qobj) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(qobj); +- qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/block/vpc.c b/block/vpc.c +index a9bb041..bf294ab 100644 +--- a/block/vpc.c ++++ b/block/vpc.c +@@ -1082,7 +1082,6 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + { + BlockdevCreateOptions *create_options = NULL; + QDict *qdict; +- QObject *qobj; + Visitor *v; + BlockDriverState *bs = NULL; + Error *local_err = NULL; +@@ -1119,14 +1118,12 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "vpc"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple_for_keyval_qiv(qdict, errp); +- if (!qobj) { ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { + ret = -EINVAL; + goto fail; + } + +- v = qobject_input_visitor_new_keyval(qobj); +- qobject_unref(qobj); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +diff --git a/include/block/qdict.h b/include/block/qdict.h +index 47d9638..d8cb502 100644 +--- a/include/block/qdict.h ++++ b/include/block/qdict.h +@@ -21,7 +21,6 @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); + void qdict_array_split(QDict *src, QList **dst); + int qdict_array_entries(QDict *src, const char *subqdict); + QObject *qdict_crumple(const QDict *src, Error **errp); +-QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp); + void qdict_flatten(QDict *qdict); + + typedef struct QDictRenames { +@@ -30,4 +29,6 @@ typedef struct QDictRenames { + } QDictRenames; + bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp); + ++Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict, ++ Error **errp); + #endif +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index aba372c..41f39ab 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -13,6 +13,7 @@ + #include "qapi/qmp/qlist.h" + #include "qapi/qmp/qnum.h" + #include "qapi/qmp/qstring.h" ++#include "qapi/qobject-input-visitor.h" + #include "qemu/cutils.h" + #include "qapi/error.h" + +@@ -529,7 +530,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + * used for anything else, and it should go away once the block + * subsystem has been cleaned up. + */ +-QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) ++static QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) + { + QDict *tmp = NULL; + char *buf; +@@ -695,3 +696,28 @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) + } + return true; + } ++ ++/* ++ * Create a QObject input visitor for flat @qdict with possibly ++ * confused scalar types. ++ * ++ * The block subsystem uses this function to visit its flat QDict with ++ * possibly confused scalar types. It should not be used for anything ++ * else, and it should go away once the block subsystem has been ++ * cleaned up. ++ */ ++Visitor *qobject_input_visitor_new_flat_confused(QDict *qdict, ++ Error **errp) ++{ ++ QObject *crumpled; ++ Visitor *v; ++ ++ crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); ++ if (!crumpled) { ++ return NULL; ++ } ++ ++ v = qobject_input_visitor_new_keyval(crumpled); ++ qobject_unref(crumpled); ++ return v; ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Fix-blockdev-blockdev-add-for-empty-objects-an.patch b/SOURCES/kvm-block-Fix-blockdev-blockdev-add-for-empty-objects-an.patch new file mode 100644 index 0000000..7a33b60 --- /dev/null +++ b/SOURCES/kvm-block-Fix-blockdev-blockdev-add-for-empty-objects-an.patch @@ -0,0 +1,278 @@ +From 44fe5f3579f8b5dfa08ce9f750306df44f026013 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:28 +0200 +Subject: [PATCH 030/268] block: Fix -blockdev / blockdev-add for empty objects + and arrays + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-22-armbru@redhat.com> +Patchwork-id: 80726 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 21/23] block: Fix -blockdev / blockdev-add for empty objects and arrays +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +-blockdev and blockdev-add silently ignore empty objects and arrays in +their argument. That's because qmp_blockdev_add() converts the +argument to a flat QDict, and qdict_flatten() eats empty QDict and +QList members. For instance, we ignore an empty BlockdevOptions +member @cache. No real harm, as absent means the same as empty there. + +Thus, the flaw puts an artificial restriction on the QAPI schema: we +can't have potentially empty objects and arrays within +BlockdevOptions, except when they're optional and "empty" has the same +meaning as "absent". + +Our QAPI schema satisfies this restriction (I checked), but it's a +trap for the unwary, and a temptation to employ awkward workarounds +for the wary. Let's get rid of it. + +Change qdict_flatten() and qdict_crumple() to treat empty dictionaries +and lists exactly like scalars. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 2860b2b2cb883969c8f6464bd9f8bc88742c5c73) +Signed-off-by: Miroslav Rezanina +--- + qobject/block-qdict.c | 54 +++++++++++++++++++++++++++++------------------ + tests/check-block-qdict.c | 38 ++++++++++++++++++++++++++------- + 2 files changed, 63 insertions(+), 29 deletions(-) + +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index e51a3d2..df83308 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -56,6 +56,8 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) + { + QObject *value; + const QListEntry *entry; ++ QDict *dict_val; ++ QList *list_val; + char *new_key; + int i; + +@@ -69,16 +71,18 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) + + for (i = 0; entry; entry = qlist_next(entry), i++) { + value = qlist_entry_obj(entry); ++ dict_val = qobject_to(QDict, value); ++ list_val = qobject_to(QList, value); + new_key = g_strdup_printf("%s.%i", prefix, i); + + /* + * Flatten non-empty QDict and QList recursively into @target, + * copy other objects to @target + */ +- if (qobject_type(value) == QTYPE_QDICT) { +- qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); +- } else if (qobject_type(value) == QTYPE_QLIST) { +- qdict_flatten_qlist(qobject_to(QList, value), target, new_key); ++ if (dict_val && qdict_size(dict_val)) { ++ qdict_flatten_qdict(dict_val, target, new_key); ++ } else if (list_val && !qlist_empty(list_val)) { ++ qdict_flatten_qlist(list_val, target, new_key); + } else { + qdict_put_obj(target, new_key, qobject_ref(value)); + } +@@ -91,6 +95,8 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + { + QObject *value; + const QDictEntry *entry, *next; ++ QDict *dict_val; ++ QList *list_val; + char *new_key; + + entry = qdict_first(qdict); +@@ -98,6 +104,8 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + while (entry != NULL) { + next = qdict_next(qdict, entry); + value = qdict_entry_value(entry); ++ dict_val = qobject_to(QDict, value); ++ list_val = qobject_to(QList, value); + new_key = NULL; + + if (prefix) { +@@ -108,12 +116,12 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + * Flatten non-empty QDict and QList recursively into @target, + * copy other objects to @target + */ +- if (qobject_type(value) == QTYPE_QDICT) { +- qdict_flatten_qdict(qobject_to(QDict, value), target, ++ if (dict_val && qdict_size(dict_val)) { ++ qdict_flatten_qdict(dict_val, target, + new_key ? new_key : entry->key); + qdict_del(qdict, entry->key); +- } else if (qobject_type(value) == QTYPE_QLIST) { +- qdict_flatten_qlist(qobject_to(QList, value), target, ++ } else if (list_val && !qlist_empty(list_val)) { ++ qdict_flatten_qlist(list_val, target, + new_key ? new_key : entry->key); + qdict_del(qdict, entry->key); + } else if (target != qdict) { +@@ -127,10 +135,11 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + } + + /** +- * qdict_flatten(): For each nested QDict with key x, all fields with key y +- * are moved to this QDict and their key is renamed to "x.y". For each nested +- * QList with key x, the field at index y is moved to this QDict with the key +- * "x.y" (i.e., the reverse of what qdict_array_split() does). ++ * qdict_flatten(): For each nested non-empty QDict with key x, all ++ * fields with key y are moved to this QDict and their key is renamed ++ * to "x.y". For each nested non-empty QList with key x, the field at ++ * index y is moved to this QDict with the key "x.y" (i.e., the ++ * reverse of what qdict_array_split() does). + * This operation is applied recursively for nested QDicts and QLists. + */ + void qdict_flatten(QDict *qdict) +@@ -361,8 +370,8 @@ static int qdict_is_list(QDict *maybe_list, Error **errp) + * @src: the original flat dictionary (only scalar values) to crumple + * + * Takes a flat dictionary whose keys use '.' separator to indicate +- * nesting, and values are scalars, and crumples it into a nested +- * structure. ++ * nesting, and values are scalars, empty dictionaries or empty lists, ++ * and crumples it into a nested structure. + * + * To include a literal '.' in a key name, it must be escaped as '..' + * +@@ -399,6 +408,8 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + { + const QDictEntry *ent; + QDict *two_level, *multi_level = NULL, *child_dict; ++ QDict *dict_val; ++ QList *list_val; + QObject *dst = NULL, *child; + size_t i; + char *prefix = NULL; +@@ -409,10 +420,11 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + + /* Step 1: split our totally flat dict into a two level dict */ + for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { +- if (qobject_type(ent->value) == QTYPE_QDICT || +- qobject_type(ent->value) == QTYPE_QLIST) { +- error_setg(errp, "Value %s is not a scalar", +- ent->key); ++ dict_val = qobject_to(QDict, ent->value); ++ list_val = qobject_to(QList, ent->value); ++ if ((dict_val && qdict_size(dict_val)) ++ || (list_val && !qlist_empty(list_val))) { ++ error_setg(errp, "Value %s is not flat", ent->key); + goto error; + } + +@@ -451,9 +463,9 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + multi_level = qdict_new(); + for (ent = qdict_first(two_level); ent != NULL; + ent = qdict_next(two_level, ent)) { +- QDict *dict = qobject_to(QDict, ent->value); +- if (dict) { +- child = qdict_crumple(dict, errp); ++ dict_val = qobject_to(QDict, ent->value); ++ if (dict_val && qdict_size(dict_val)) { ++ child = qdict_crumple(dict_val, errp); + if (!child) { + goto error; + } +diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c +index 2da16f0..1d20fcc 100644 +--- a/tests/check-block-qdict.c ++++ b/tests/check-block-qdict.c +@@ -79,10 +79,10 @@ static void qdict_flatten_test(void) + * "e.1.2.b": 1, + * "f.c": 2, + * "f.d": 3, +- * "g": 4 ++ * "g": 4, ++ * "y.0": {}, ++ * "z.a": [] + * } +- * +- * Note that "y" and "z" get eaten. + */ + + qdict_put_int(e_1_2, "a", 0); +@@ -117,8 +117,10 @@ static void qdict_flatten_test(void) + g_assert(qdict_get_int(root, "f.c") == 2); + g_assert(qdict_get_int(root, "f.d") == 3); + g_assert(qdict_get_int(root, "g") == 4); ++ g_assert(!qdict_size(qdict_get_qdict(root, "y.0"))); ++ g_assert(qlist_empty(qdict_get_qlist(root, "z.a"))); + +- g_assert(qdict_size(root) == 8); ++ g_assert(qdict_size(root) == 10); + + qobject_unref(root); + } +@@ -387,7 +389,8 @@ static void qdict_join_test(void) + static void qdict_crumple_test_recursive(void) + { + QDict *src, *dst, *rule, *vnc, *acl, *listen; +- QList *rules; ++ QDict *empty, *empty_dict, *empty_list_0; ++ QList *rules, *empty_list, *empty_dict_a; + + src = qdict_new(); + qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); +@@ -399,10 +402,12 @@ static void qdict_crumple_test_recursive(void) + qdict_put_str(src, "vnc.acl.default", "deny"); + qdict_put_str(src, "vnc.acl..name", "acl0"); + qdict_put_str(src, "vnc.acl.rule..name", "acl0"); ++ qdict_put(src, "empty.dict.a", qlist_new()); ++ qdict_put(src, "empty.list.0", qdict_new()); + + dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); + g_assert(dst); +- g_assert_cmpint(qdict_size(dst), ==, 1); ++ g_assert_cmpint(qdict_size(dst), ==, 2); + + vnc = qdict_get_qdict(dst, "vnc"); + g_assert(vnc); +@@ -440,6 +445,21 @@ static void qdict_crumple_test_recursive(void) + g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); + g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); + ++ empty = qdict_get_qdict(dst, "empty"); ++ g_assert(empty); ++ g_assert_cmpint(qdict_size(empty), ==, 2); ++ empty_dict = qdict_get_qdict(empty, "dict"); ++ g_assert(empty_dict); ++ g_assert_cmpint(qdict_size(empty_dict), ==, 1); ++ empty_dict_a = qdict_get_qlist(empty_dict, "a"); ++ g_assert(empty_dict_a && qlist_empty(empty_dict_a)); ++ empty_list = qdict_get_qlist(empty, "list"); ++ g_assert(empty_list); ++ g_assert_cmpint(qlist_size(empty_list), ==, 1); ++ empty_list_0 = qobject_to(QDict, qlist_pop(empty_list)); ++ g_assert(empty_list_0); ++ g_assert_cmpint(qdict_size(empty_list_0), ==, 0); ++ + qobject_unref(src); + qobject_unref(dst); + } +@@ -587,7 +607,7 @@ static void qdict_rename_keys_test(void) + + static void qdict_crumple_test_bad_inputs(void) + { +- QDict *src; ++ QDict *src, *nested; + Error *error = NULL; + + src = qdict_new(); +@@ -614,7 +634,9 @@ static void qdict_crumple_test_bad_inputs(void) + + src = qdict_new(); + /* The input should be flat, ie no dicts or lists */ +- qdict_put(src, "rule.a", qdict_new()); ++ nested = qdict_new(); ++ qdict_put(nested, "x", qdict_new()); ++ qdict_put(src, "rule.a", nested); + qdict_put_str(src, "rule.b", "allow"); + + g_assert(qdict_crumple(src, &error) == NULL); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Fix-blockdev-for-certain-non-string-scalars.patch b/SOURCES/kvm-block-Fix-blockdev-for-certain-non-string-scalars.patch new file mode 100644 index 0000000..935fa89 --- /dev/null +++ b/SOURCES/kvm-block-Fix-blockdev-for-certain-non-string-scalars.patch @@ -0,0 +1,302 @@ +From 3a92d2770a0909c1ab425b4a5d24014e7fea2297 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:17 +0200 +Subject: [PATCH 019/268] block: Fix -blockdev for certain non-string scalars + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-11-armbru@redhat.com> +Patchwork-id: 80743 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 10/23] block: Fix -blockdev for certain non-string scalars +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Configuration flows through the block subsystem in a rather peculiar +way. Configuration made with -drive enters it as QemuOpts. +Configuration made with -blockdev / blockdev-add enters it as QAPI +type BlockdevOptions. The block subsystem uses QDict, QemuOpts and +QAPI types internally. The precise flow is next to impossible to +explain (I tried for this commit message, but gave up after wasting +several hours). What I can explain is a flaw in the BlockDriver +interface that leads to this bug: + + $ qemu-system-x86_64 -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234 + qemu-system-x86_64: -blockdev node-name=n1,driver=nfs,server.type=inet,server.host=localhost,path=/foo/bar,user=1234: Internal error: parameter user invalid + +QMP blockdev-add is broken the same way. + +Here's what happens. The block layer passes configuration represented +as flat QDict (with dotted keys) to BlockDriver methods +.bdrv_file_open(). The QDict's members are typed according to the +QAPI schema. + +nfs_file_open() converts it to QAPI type BlockdevOptionsNfs, with +qdict_crumple() and a qobject input visitor. + +This visitor comes in two flavors. The plain flavor requires scalars +to be typed according to the QAPI schema. That's the case here. The +keyval flavor requires string scalars. That's not the case here. +nfs_file_open() uses the latter, and promptly falls apart for members +@user, @group, @tcp-syn-count, @readahead-size, @page-cache-size, +@debug. + +Switching to the plain flavor would fix -blockdev, but break -drive, +because there the scalars arrive in nfs_file_open() as strings. + +The proper fix would be to replace the QDict by QAPI type +BlockdevOptions in the BlockDriver interface. Sadly, that's beyond my +reach right now. + +Next best would be to fix the block layer to always pass correctly +typed QDicts to the BlockDriver methods. Also beyond my reach. + +What I can do is throw another hack onto the pile: have +nfs_file_open() convert all members to string, so use of the keyval +flavor actually works, by replacing qdict_crumple() by new function +qdict_crumple_for_keyval_qiv(). + +The pattern "pass result of qdict_crumple() to +qobject_input_visitor_new_keyval()" occurs several times more: + +* qemu_rbd_open() + + Same issue as nfs_file_open(), but since BlockdevOptionsRbd has only + string members, its only a latent bug. Fix it anyway. + +* parallels_co_create_opts(), qcow_co_create_opts(), + qcow2_co_create_opts(), bdrv_qed_co_create_opts(), + sd_co_create_opts(), vhdx_co_create_opts(), vpc_co_create_opts() + + These work, because they create the QDict with + qemu_opts_to_qdict_filtered(), which creates only string scalars. + The function sports a TODO comment asking for better typing; that's + going to be fun. Use qdict_crumple_for_keyval_qiv() to be safe. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit e5af0da1dcbfb1a4694150f9954554fb6df88819) +Signed-off-by: Miroslav Rezanina +--- + block/nfs.c | 2 +- + block/parallels.c | 2 +- + block/qcow.c | 2 +- + block/qcow2.c | 2 +- + block/qed.c | 2 +- + block/rbd.c | 2 +- + block/sheepdog.c | 2 +- + block/vhdx.c | 2 +- + block/vpc.c | 2 +- + include/block/qdict.h | 1 + + qobject/block-qdict.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++ + 11 files changed, 67 insertions(+), 9 deletions(-) + +diff --git a/block/nfs.c b/block/nfs.c +index 5159ef0..4090d28 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -560,7 +560,7 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, + Visitor *v; + Error *local_err = NULL; + +- crumpled = qdict_crumple(options, errp); ++ crumpled = qdict_crumple_for_keyval_qiv(options, errp); + if (crumpled == NULL) { + return NULL; + } +diff --git a/block/parallels.c b/block/parallels.c +index 0ee1f6a..aa58955 100644 +--- a/block/parallels.c ++++ b/block/parallels.c +@@ -651,7 +651,7 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "parallels"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple(qdict, errp); ++ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { +diff --git a/block/qcow.c b/block/qcow.c +index fb821ad..14b7296 100644 +--- a/block/qcow.c ++++ b/block/qcow.c +@@ -995,7 +995,7 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "qcow"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple(qdict, errp); ++ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { +diff --git a/block/qcow2.c b/block/qcow2.c +index fa9f557..fa06b41 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3139,7 +3139,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + qdict_put_str(qdict, "file", bs->node_name); + + /* Now get the QAPI type BlockdevCreateOptions */ +- qobj = qdict_crumple(qdict, errp); ++ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { +diff --git a/block/qed.c b/block/qed.c +index 9a8997a..d8810f5 100644 +--- a/block/qed.c ++++ b/block/qed.c +@@ -763,7 +763,7 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "qed"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple(qdict, errp); ++ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { +diff --git a/block/rbd.c b/block/rbd.c +index e695cf2..0b5455f 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -640,7 +640,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + } + + /* Convert the remaining options into a QAPI object */ +- crumpled = qdict_crumple(options, errp); ++ crumpled = qdict_crumple_for_keyval_qiv(options, errp); + if (crumpled == NULL) { + r = -EINVAL; + goto out; +diff --git a/block/sheepdog.c b/block/sheepdog.c +index fd3876f..821a3c4 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -2218,7 +2218,7 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + } + + /* Get the QAPI object */ +- crumpled = qdict_crumple(qdict, errp); ++ crumpled = qdict_crumple_for_keyval_qiv(qdict, errp); + if (crumpled == NULL) { + ret = -EINVAL; + goto fail; +diff --git a/block/vhdx.c b/block/vhdx.c +index 26c05aa..32939c4 100644 +--- a/block/vhdx.c ++++ b/block/vhdx.c +@@ -2003,7 +2003,7 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "vhdx"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple(qdict, errp); ++ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { +diff --git a/block/vpc.c b/block/vpc.c +index 41c8c98..16178e5 100644 +--- a/block/vpc.c ++++ b/block/vpc.c +@@ -1119,7 +1119,7 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + qdict_put_str(qdict, "driver", "vpc"); + qdict_put_str(qdict, "file", bs->node_name); + +- qobj = qdict_crumple(qdict, errp); ++ qobj = qdict_crumple_for_keyval_qiv(qdict, errp); + qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { +diff --git a/include/block/qdict.h b/include/block/qdict.h +index 71c037a..47d9638 100644 +--- a/include/block/qdict.h ++++ b/include/block/qdict.h +@@ -21,6 +21,7 @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start); + void qdict_array_split(QDict *src, QList **dst); + int qdict_array_entries(QDict *src, const char *subqdict); + QObject *qdict_crumple(const QDict *src, Error **errp); ++QObject *qdict_crumple_for_keyval_qiv(QDict *qdict, Error **errp); + void qdict_flatten(QDict *qdict); + + typedef struct QDictRenames { +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index fb92010..aba372c 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -9,7 +9,10 @@ + + #include "qemu/osdep.h" + #include "block/qdict.h" ++#include "qapi/qmp/qbool.h" + #include "qapi/qmp/qlist.h" ++#include "qapi/qmp/qnum.h" ++#include "qapi/qmp/qstring.h" + #include "qemu/cutils.h" + #include "qapi/error.h" + +@@ -514,6 +517,60 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + } + + /** ++ * qdict_crumple_for_keyval_qiv: ++ * @src: the flat dictionary (only scalar values) to crumple ++ * @errp: location to store error ++ * ++ * Like qdict_crumple(), but additionally transforms scalar values so ++ * the result can be passed to qobject_input_visitor_new_keyval(). ++ * ++ * The block subsystem uses this function to prepare its flat QDict ++ * with possibly confused scalar types for a visit. It should not be ++ * used for anything else, and it should go away once the block ++ * subsystem has been cleaned up. ++ */ ++QObject *qdict_crumple_for_keyval_qiv(QDict *src, Error **errp) ++{ ++ QDict *tmp = NULL; ++ char *buf; ++ const char *s; ++ const QDictEntry *ent; ++ QObject *dst; ++ ++ for (ent = qdict_first(src); ent; ent = qdict_next(src, ent)) { ++ buf = NULL; ++ switch (qobject_type(ent->value)) { ++ case QTYPE_QNULL: ++ case QTYPE_QSTRING: ++ continue; ++ case QTYPE_QNUM: ++ s = buf = qnum_to_string(qobject_to(QNum, ent->value)); ++ break; ++ case QTYPE_QDICT: ++ case QTYPE_QLIST: ++ /* @src isn't flat; qdict_crumple() will fail */ ++ continue; ++ case QTYPE_QBOOL: ++ s = qbool_get_bool(qobject_to(QBool, ent->value)) ++ ? "on" : "off"; ++ break; ++ default: ++ abort(); ++ } ++ ++ if (!tmp) { ++ tmp = qdict_clone_shallow(src); ++ } ++ qdict_put(tmp, ent->key, qstring_from_str(s)); ++ g_free(buf); ++ } ++ ++ dst = qdict_crumple(tmp ?: src, errp); ++ qobject_unref(tmp); ++ return dst; ++} ++ ++/** + * qdict_array_entries(): Returns the number of direct array entries if the + * sub-QDict of src specified by the prefix in subqdict (or src itself for + * prefix == "") is valid as an array, i.e. the length of the created list if +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Fix-copy-on-read-crash-with-partial-final-clus.patch b/SOURCES/kvm-block-Fix-copy-on-read-crash-with-partial-final-clus.patch new file mode 100644 index 0000000..f24e64e --- /dev/null +++ b/SOURCES/kvm-block-Fix-copy-on-read-crash-with-partial-final-clus.patch @@ -0,0 +1,94 @@ +From 3ad6a0d2e3805867cd5af8ccea841acb3608de99 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 15:00:08 +0200 +Subject: [PATCH 214/268] block: Fix copy-on-read crash with partial final + cluster + +RH-Author: Kevin Wolf +Message-id: <20180712150008.23662-2-kwolf@redhat.com> +Patchwork-id: 81331 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/1] block: Fix copy-on-read crash with partial final cluster +Bugzilla: 1590640 +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng +RH-Acked-by: John Snow +RH-Acked-by: Richard Jones + +If the virtual disk size isn't aligned to full clusters, +bdrv_co_do_copy_on_readv() may get pnum == 0 before having the full +cluster completed, which will let it run into an assertion failure: + +qemu-io: block/io.c:1203: bdrv_co_do_copy_on_readv: Assertion `skip_bytes < pnum' failed. + +Check for EOF, assert that we read at least as much as the read request +originally wanted to have (which is true at EOF because otherwise +bdrv_check_byte_request() would already have returned an error) and +return success early even though we couldn't copy the full cluster. + +Signed-off-by: Kevin Wolf +(cherry picked from commit b0ddcbbb36a66a605eb232b905cb49b1cc72e74e) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 6 ++++++ + tests/qemu-iotests/197 | 9 +++++++++ + tests/qemu-iotests/197.out | 8 ++++++++ + 3 files changed, 23 insertions(+) + +diff --git a/block/io.c b/block/io.c +index ad8afc0..ac36d1c 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -1095,6 +1095,12 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvChild *child, + pnum = MIN(cluster_bytes, max_transfer); + } + ++ /* Stop at EOF if the image ends in the middle of the cluster */ ++ if (ret == 0 && pnum == 0) { ++ assert(progress >= bytes); ++ break; ++ } ++ + assert(skip_bytes < pnum); + + if (ret <= 0) { +diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 +index 3ae4975..0369aa5 100755 +--- a/tests/qemu-iotests/197 ++++ b/tests/qemu-iotests/197 +@@ -109,6 +109,15 @@ $QEMU_IO -f qcow2 -c map "$TEST_WRAP" + _check_test_img + $QEMU_IMG compare -f $IMGFMT -F qcow2 "$TEST_IMG" "$TEST_WRAP" + ++echo ++echo '=== Partial final cluster ===' ++echo ++ ++_make_test_img 1024 ++$QEMU_IO -f $IMGFMT -C -c 'read 0 1024' "$TEST_IMG" | _filter_qemu_io ++$QEMU_IO -f $IMGFMT -c map "$TEST_IMG" ++_check_test_img ++ + # success, all done + echo '*** done' + status=0 +diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out +index 52b4137..8febda5 100644 +--- a/tests/qemu-iotests/197.out ++++ b/tests/qemu-iotests/197.out +@@ -23,4 +23,12 @@ can't open device TEST_DIR/t.wrap.qcow2: Can't use copy-on-read on read-only dev + 1023.938 MiB (0x3fff0000) bytes not allocated at offset 3 GiB (0xc0010000) + No errors were found on the image. + Images are identical. ++ ++=== Partial final cluster === ++ ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1024 ++read 1024/1024 bytes at offset 0 ++1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++1 KiB (0x400) bytes allocated at offset 0 bytes (0x0) ++No errors were found on the image. + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Fix-drive-for-certain-non-string-scalars.patch b/SOURCES/kvm-block-Fix-drive-for-certain-non-string-scalars.patch new file mode 100644 index 0000000..c4f8970 --- /dev/null +++ b/SOURCES/kvm-block-Fix-drive-for-certain-non-string-scalars.patch @@ -0,0 +1,123 @@ +From 36c5bfa7a0dbc9812400390e446ae8198df189d1 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:18 +0200 +Subject: [PATCH 020/268] block: Fix -drive for certain non-string scalars + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-12-armbru@redhat.com> +Patchwork-id: 80737 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 11/23] block: Fix -drive for certain non-string scalars +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +The previous commit fixed -blockdev breakage due to misuse of the +qobject input visitor's keyval flavor in bdrv_file_open(). The commit +message explain why using the plain flavor would be just as wrong; it +would break -drive. Turns out we break it in three places: +nbd_open(), sd_open() and ssh_file_open(). They are even marked +FIXME. Example breakage: + + $ qemu-system-x86 -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off + qemu-system-x86: -drive node-name=n1,driver=nbd,server.type=inet,server.host=localhost,server.port=1234,server.numeric=off: Invalid parameter type for 'numeric', expected: boolean + +Fix it the same way: replace qdict_crumple() by +qdict_crumple_for_keyval_qiv(), and switch from plain to the keyval +flavor. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 374c52467a38c2e811f6c0db4edc9ea7d5f34341) +Signed-off-by: Miroslav Rezanina +--- + block/nbd.c | 12 ++---------- + block/sheepdog.c | 12 ++---------- + block/ssh.c | 12 ++---------- + 3 files changed, 6 insertions(+), 30 deletions(-) + +diff --git a/block/nbd.c b/block/nbd.c +index f499830..b0be357 100644 +--- a/block/nbd.c ++++ b/block/nbd.c +@@ -273,20 +273,12 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, + goto done; + } + +- crumpled_addr = qdict_crumple(addr, errp); ++ crumpled_addr = qdict_crumple_for_keyval_qiv(addr, errp); + if (!crumpled_addr) { + goto done; + } + +- /* +- * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive +- * server.type=inet. .to doesn't matter, it's ignored anyway. +- * That's because when @options come from -blockdev or +- * blockdev_add, members are typed according to the QAPI schema, +- * but when they come from -drive, they're all QString. The +- * visitor expects the former. +- */ +- iv = qobject_input_visitor_new(crumpled_addr); ++ iv = qobject_input_visitor_new_keyval(crumpled_addr); + visit_type_SocketAddress(iv, NULL, &saddr, &local_err); + if (local_err) { + error_propagate(errp, local_err); +diff --git a/block/sheepdog.c b/block/sheepdog.c +index 821a3c4..dd582d5 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -546,20 +546,12 @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) + + qdict_extract_subqdict(options, &server, "server."); + +- crumpled_server = qdict_crumple(server, errp); ++ crumpled_server = qdict_crumple_for_keyval_qiv(server, errp); + if (!crumpled_server) { + goto done; + } + +- /* +- * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive +- * server.type=inet. .to doesn't matter, it's ignored anyway. +- * That's because when @options come from -blockdev or +- * blockdev_add, members are typed according to the QAPI schema, +- * but when they come from -drive, they're all QString. The +- * visitor expects the former. +- */ +- iv = qobject_input_visitor_new(crumpled_server); ++ iv = qobject_input_visitor_new_keyval(crumpled_server); + visit_type_SocketAddress(iv, NULL, &saddr, &local_err); + if (local_err) { + error_propagate(errp, local_err); +diff --git a/block/ssh.c b/block/ssh.c +index 5931064..2fc7cd9 100644 +--- a/block/ssh.c ++++ b/block/ssh.c +@@ -623,20 +623,12 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) + } + + /* Create the QAPI object */ +- crumpled = qdict_crumple(options, errp); ++ crumpled = qdict_crumple_for_keyval_qiv(options, errp); + if (crumpled == NULL) { + goto fail; + } + +- /* +- * FIXME .numeric, .to, .ipv4 or .ipv6 don't work with -drive. +- * .to doesn't matter, it's ignored anyway. +- * That's because when @options come from -blockdev or +- * blockdev_add, members are typed according to the QAPI schema, +- * but when they come from -drive, they're all QString. The +- * visitor expects the former. +- */ +- v = qobject_input_visitor_new(crumpled); ++ v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); + visit_free(v); + qobject_unref(crumpled); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Fix-parameter-checking-in-bdrv_co_copy_range_i.patch b/SOURCES/kvm-block-Fix-parameter-checking-in-bdrv_co_copy_range_i.patch new file mode 100644 index 0000000..8006595 --- /dev/null +++ b/SOURCES/kvm-block-Fix-parameter-checking-in-bdrv_co_copy_range_i.patch @@ -0,0 +1,101 @@ +From 521c003737f893ff4a6c6e95b0d1555d28a86252 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 24 Jul 2018 12:50:06 +0200 +Subject: [PATCH 232/268] block: Fix parameter checking in + bdrv_co_copy_range_internal + +RH-Author: John Snow +Message-id: <20180718225511.14878-15-jsnow@redhat.com> +Patchwork-id: 81411 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 14/35] block: Fix parameter checking in bdrv_co_copy_range_internal +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Fam Zheng + +src may be NULL if BDRV_REQ_ZERO_WRITE flag is set, in this case only +check dst and dst->bs. This bug was introduced when moving in the +request tracking code from bdrv_co_copy_range, in 37aec7d75eb. + +This especially fixes the possible segfault when initializing src_bs +with a NULL src. + +Signed-off-by: Fam Zheng +Message-id: 20180703023758.14422-2-famz@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit d4d3e5a0d53a57282955e8a3ed7acc1ca90552d9) +Signed-off-by: John Snow +--- + block/io.c | 28 +++++++++++++++------------- + 1 file changed, 15 insertions(+), 13 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 136a5d0..7981239 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2848,17 +2848,11 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + bool recurse_src) + { + BdrvTrackedRequest src_req, dst_req; +- BlockDriverState *src_bs = src->bs; +- BlockDriverState *dst_bs = dst->bs; + int ret; + +- if (!src || !dst || !src->bs || !dst->bs) { ++ if (!dst || !dst->bs) { + return -ENOMEDIUM; + } +- ret = bdrv_check_byte_request(src->bs, src_offset, bytes); +- if (ret) { +- return ret; +- } + + ret = bdrv_check_byte_request(dst->bs, dst_offset, bytes); + if (ret) { +@@ -2868,17 +2862,25 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + return bdrv_co_pwrite_zeroes(dst, dst_offset, bytes, flags); + } + ++ if (!src || !src->bs) { ++ return -ENOMEDIUM; ++ } ++ ret = bdrv_check_byte_request(src->bs, src_offset, bytes); ++ if (ret) { ++ return ret; ++ } ++ + if (!src->bs->drv->bdrv_co_copy_range_from + || !dst->bs->drv->bdrv_co_copy_range_to + || src->bs->encrypted || dst->bs->encrypted) { + return -ENOTSUP; + } + +- bdrv_inc_in_flight(src_bs); +- bdrv_inc_in_flight(dst_bs); +- tracked_request_begin(&src_req, src_bs, src_offset, ++ bdrv_inc_in_flight(src->bs); ++ bdrv_inc_in_flight(dst->bs); ++ tracked_request_begin(&src_req, src->bs, src_offset, + bytes, BDRV_TRACKED_READ); +- tracked_request_begin(&dst_req, dst_bs, dst_offset, ++ tracked_request_begin(&dst_req, dst->bs, dst_offset, + bytes, BDRV_TRACKED_WRITE); + + wait_serialising_requests(&src_req); +@@ -2896,8 +2898,8 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + } + tracked_request_end(&src_req); + tracked_request_end(&dst_req); +- bdrv_dec_in_flight(src_bs); +- bdrv_dec_in_flight(dst_bs); ++ bdrv_dec_in_flight(src->bs); ++ bdrv_dec_in_flight(dst->bs); + return ret; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Fix-update-of-BDRV_O_AUTO_RDONLY-in-update_fla.patch b/SOURCES/kvm-block-Fix-update-of-BDRV_O_AUTO_RDONLY-in-update_fla.patch new file mode 100644 index 0000000..5569c99 --- /dev/null +++ b/SOURCES/kvm-block-Fix-update-of-BDRV_O_AUTO_RDONLY-in-update_fla.patch @@ -0,0 +1,56 @@ +From be690258db5ef9b4b80dd1d24d8a275ffc36f84c Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:42 +0000 +Subject: [PATCH 12/14] block: Fix update of BDRV_O_AUTO_RDONLY in + update_flags_from_options() + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-13-kwolf@redhat.com> +Patchwork-id: 83960 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 12/12] block: Fix update of BDRV_O_AUTO_RDONLY in update_flags_from_options() +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +From: Alberto Garcia + +Commit e35bdc123a4ace9f4d3fcca added the auto-read-only option and the +code to update its corresponding flag in update_flags_from_options(), +but forgot to clear the flag if auto-read-only is false. + +Signed-off-by: Alberto Garcia +Reported-by: Max Reitz +Signed-off-by: Kevin Wolf +(cherry picked from commit 2a3d4331fa2d40708188b8000f98ff1f7dcd33bc) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 4 +--- + 1 file changed, 1 insertion(+), 3 deletions(-) + +diff --git a/block.c b/block.c +index 268debe..d0f0dc6 100644 +--- a/block.c ++++ b/block.c +@@ -1112,7 +1112,7 @@ static int bdrv_open_flags(BlockDriverState *bs, int flags) + + static void update_flags_from_options(int *flags, QemuOpts *opts) + { +- *flags &= ~BDRV_O_CACHE_MASK; ++ *flags &= ~(BDRV_O_CACHE_MASK | BDRV_O_RDWR | BDRV_O_AUTO_RDONLY); + + assert(qemu_opt_find(opts, BDRV_OPT_CACHE_NO_FLUSH)); + if (qemu_opt_get_bool(opts, BDRV_OPT_CACHE_NO_FLUSH, false)) { +@@ -1124,8 +1124,6 @@ static void update_flags_from_options(int *flags, QemuOpts *opts) + *flags |= BDRV_O_NOCACHE; + } + +- *flags &= ~BDRV_O_RDWR; +- + assert(qemu_opt_find(opts, BDRV_OPT_READ_ONLY)); + if (!qemu_opt_get_bool(opts, BDRV_OPT_READ_ONLY, false)) { + *flags |= BDRV_O_RDWR; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Honour-BDRV_REQ_NO_SERIALISING-in-copy-range.patch b/SOURCES/kvm-block-Honour-BDRV_REQ_NO_SERIALISING-in-copy-range.patch new file mode 100644 index 0000000..7110e2c --- /dev/null +++ b/SOURCES/kvm-block-Honour-BDRV_REQ_NO_SERIALISING-in-copy-range.patch @@ -0,0 +1,72 @@ +From 088ddadf21938cfa527bb47ce5452ad502ff2c8d Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:51 +0200 +Subject: [PATCH 233/268] block: Honour BDRV_REQ_NO_SERIALISING in copy range + +RH-Author: John Snow +Message-id: <20180718225511.14878-16-jsnow@redhat.com> +Patchwork-id: 81402 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 15/35] block: Honour BDRV_REQ_NO_SERIALISING in copy range +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Fam Zheng + +This semantics is needed by drive-backup so implement it before using +this API there. + +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Fam Zheng +Message-id: 20180703023758.14422-3-famz@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit dee12de89380483656072f775f5ef99f4426f966) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 6 ++++-- + include/block/block.h | 5 +++-- + 2 files changed, 7 insertions(+), 4 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 7981239..b6754f3 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2883,8 +2883,10 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + tracked_request_begin(&dst_req, dst->bs, dst_offset, + bytes, BDRV_TRACKED_WRITE); + +- wait_serialising_requests(&src_req); +- wait_serialising_requests(&dst_req); ++ if (!(flags & BDRV_REQ_NO_SERIALISING)) { ++ wait_serialising_requests(&src_req); ++ wait_serialising_requests(&dst_req); ++ } + if (recurse_src) { + ret = src->bs->drv->bdrv_co_copy_range_from(src->bs, + src, src_offset, +diff --git a/include/block/block.h b/include/block/block.h +index c3cfb40..e1d5e47 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -636,13 +636,14 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host); + * @dst: Destination child to copy data to + * @dst_offset: offset in @dst image to write data + * @bytes: number of bytes to copy +- * @flags: request flags. Must be one of: +- * 0 - actually read data from src; ++ * @flags: request flags. Supported flags: + * BDRV_REQ_ZERO_WRITE - treat the @src range as zero data and do zero + * write on @dst as if bdrv_co_pwrite_zeroes is + * called. Used to simplify caller code, or + * during BlockDriver.bdrv_co_copy_range_from() + * recursion. ++ * BDRV_REQ_NO_SERIALISING - do not serialize with other overlapping ++ * requests currently in flight. + * + * Returns: 0 if succeeded; negative error code if failed. + **/ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Introduce-API-for-copy-offloading.patch b/SOURCES/kvm-block-Introduce-API-for-copy-offloading.patch new file mode 100644 index 0000000..6f58550 --- /dev/null +++ b/SOURCES/kvm-block-Introduce-API-for-copy-offloading.patch @@ -0,0 +1,236 @@ +From d193d49f5180d6a6b959808368247cdd506bf989 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:41 +0200 +Subject: [PATCH 167/268] block: Introduce API for copy offloading + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-2-famz@redhat.com> +Patchwork-id: 81153 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 01/13] block: Introduce API for copy offloading +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Introduce the bdrv_co_copy_range() API for copy offloading. Block +drivers implementing this API support efficient copy operations that +avoid reading each block from the source device and writing it to the +destination devices. Examples of copy offload primitives are SCSI +EXTENDED COPY and Linux copy_file_range(2). + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Message-id: 20180601092648.24614-2-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit fcc6767836efe1b160289905dce7228d594c123c) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++ + include/block/block.h | 32 ++++++++++++++++ + include/block/block_int.h | 38 +++++++++++++++++++ + 3 files changed, 167 insertions(+) + +diff --git a/block/io.c b/block/io.c +index fada4ef..5c043a4 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2832,3 +2832,100 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host) + bdrv_unregister_buf(child->bs, host); + } + } ++ ++static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags flags, ++ bool recurse_src) ++{ ++ int ret; ++ ++ if (!src || !dst || !src->bs || !dst->bs) { ++ return -ENOMEDIUM; ++ } ++ ret = bdrv_check_byte_request(src->bs, src_offset, bytes); ++ if (ret) { ++ return ret; ++ } ++ ++ ret = bdrv_check_byte_request(dst->bs, dst_offset, bytes); ++ if (ret) { ++ return ret; ++ } ++ if (flags & BDRV_REQ_ZERO_WRITE) { ++ return bdrv_co_pwrite_zeroes(dst, dst_offset, bytes, flags); ++ } ++ ++ if (!src->bs->drv->bdrv_co_copy_range_from ++ || !dst->bs->drv->bdrv_co_copy_range_to ++ || src->bs->encrypted || dst->bs->encrypted) { ++ return -ENOTSUP; ++ } ++ if (recurse_src) { ++ return src->bs->drv->bdrv_co_copy_range_from(src->bs, ++ src, src_offset, ++ dst, dst_offset, ++ bytes, flags); ++ } else { ++ return dst->bs->drv->bdrv_co_copy_range_to(dst->bs, ++ src, src_offset, ++ dst, dst_offset, ++ bytes, flags); ++ } ++} ++ ++/* Copy range from @src to @dst. ++ * ++ * See the comment of bdrv_co_copy_range for the parameter and return value ++ * semantics. */ ++int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset, ++ bytes, flags, true); ++} ++ ++/* Copy range from @src to @dst. ++ * ++ * See the comment of bdrv_co_copy_range for the parameter and return value ++ * semantics. */ ++int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset, ++ bytes, flags, false); ++} ++ ++int coroutine_fn bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ BdrvTrackedRequest src_req, dst_req; ++ BlockDriverState *src_bs = src->bs; ++ BlockDriverState *dst_bs = dst->bs; ++ int ret; ++ ++ bdrv_inc_in_flight(src_bs); ++ bdrv_inc_in_flight(dst_bs); ++ tracked_request_begin(&src_req, src_bs, src_offset, ++ bytes, BDRV_TRACKED_READ); ++ tracked_request_begin(&dst_req, dst_bs, dst_offset, ++ bytes, BDRV_TRACKED_WRITE); ++ ++ wait_serialising_requests(&src_req); ++ wait_serialising_requests(&dst_req); ++ ret = bdrv_co_copy_range_from(src, src_offset, ++ dst, dst_offset, ++ bytes, flags); ++ ++ tracked_request_end(&src_req); ++ tracked_request_end(&dst_req); ++ bdrv_dec_in_flight(src_bs); ++ bdrv_dec_in_flight(dst_bs); ++ return ret; ++} +diff --git a/include/block/block.h b/include/block/block.h +index 2d17b09..e677080 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -613,4 +613,36 @@ bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name, + */ + void bdrv_register_buf(BlockDriverState *bs, void *host, size_t size); + void bdrv_unregister_buf(BlockDriverState *bs, void *host); ++ ++/** ++ * ++ * bdrv_co_copy_range: ++ * ++ * Do offloaded copy between two children. If the operation is not implemented ++ * by the driver, or if the backend storage doesn't support it, a negative ++ * error code will be returned. ++ * ++ * Note: block layer doesn't emulate or fallback to a bounce buffer approach ++ * because usually the caller shouldn't attempt offloaded copy any more (e.g. ++ * calling copy_file_range(2)) after the first error, thus it should fall back ++ * to a read+write path in the caller level. ++ * ++ * @src: Source child to copy data from ++ * @src_offset: offset in @src image to read data ++ * @dst: Destination child to copy data to ++ * @dst_offset: offset in @dst image to write data ++ * @bytes: number of bytes to copy ++ * @flags: request flags. Must be one of: ++ * 0 - actually read data from src; ++ * BDRV_REQ_ZERO_WRITE - treat the @src range as zero data and do zero ++ * write on @dst as if bdrv_co_pwrite_zeroes is ++ * called. Used to simplify caller code, or ++ * during BlockDriver.bdrv_co_copy_range_from() ++ * recursion. ++ * ++ * Returns: 0 if succeeded; negative error code if failed. ++ **/ ++int coroutine_fn bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags); + #endif +diff --git a/include/block/block_int.h b/include/block/block_int.h +index ad2b852..3da86a7 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -206,6 +206,37 @@ struct BlockDriver { + int coroutine_fn (*bdrv_co_pdiscard)(BlockDriverState *bs, + int64_t offset, int bytes); + ++ /* Map [offset, offset + nbytes) range onto a child of @bs to copy from, ++ * and invoke bdrv_co_copy_range_from(child, ...), or invoke ++ * bdrv_co_copy_range_to() if @bs is the leaf child to copy data from. ++ * ++ * See the comment of bdrv_co_copy_range for the parameter and return value ++ * semantics. ++ */ ++ int coroutine_fn (*bdrv_co_copy_range_from)(BlockDriverState *bs, ++ BdrvChild *src, ++ uint64_t offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags flags); ++ ++ /* Map [offset, offset + nbytes) range onto a child of bs to copy data to, ++ * and invoke bdrv_co_copy_range_to(child, src, ...), or perform the copy ++ * operation if @bs is the leaf and @src has the same BlockDriver. Return ++ * -ENOTSUP if @bs is the leaf but @src has a different BlockDriver. ++ * ++ * See the comment of bdrv_co_copy_range for the parameter and return value ++ * semantics. ++ */ ++ int coroutine_fn (*bdrv_co_copy_range_to)(BlockDriverState *bs, ++ BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags flags); ++ + /* + * Building block for bdrv_block_status[_above] and + * bdrv_is_allocated[_above]. The driver should answer only +@@ -1091,4 +1122,11 @@ void bdrv_dec_in_flight(BlockDriverState *bs); + + void blockdev_close_all_bdrv_states(void); + ++int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags); ++int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags); ++ + #endif /* BLOCK_INT_H */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Make-auto-read-only-on-default-for-drive.patch b/SOURCES/kvm-block-Make-auto-read-only-on-default-for-drive.patch new file mode 100644 index 0000000..be0a3ec --- /dev/null +++ b/SOURCES/kvm-block-Make-auto-read-only-on-default-for-drive.patch @@ -0,0 +1,46 @@ +From 199b75933947cbfdedb3b1afd74ec1d24b9379b0 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:40 +0000 +Subject: [PATCH 10/14] block: Make auto-read-only=on default for -drive + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-11-kwolf@redhat.com> +Patchwork-id: 83958 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 10/12] block: Make auto-read-only=on default for -drive +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +While we want machine interfaces like -blockdev and QMP blockdev-add to +add as little auto-detection as possible so that management tools are +explicit about their needs, -drive is a convenience option for human +users. Enabling auto-read-only=on by default there enables users to use +read-only images for read-only guest devices without having to specify +read-only=on explicitly. If they try to attach the image to a read-write +device, they will still get an error message. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 9384a444f6ebcd66bba0ae3c8434120d03c8dfea) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/blockdev.c b/blockdev.c +index be650d0..f3b60f6 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -590,6 +590,7 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts, + qdict_set_default_str(bs_opts, BDRV_OPT_CACHE_NO_FLUSH, "off"); + qdict_set_default_str(bs_opts, BDRV_OPT_READ_ONLY, + read_only ? "on" : "off"); ++ qdict_set_default_str(bs_opts, BDRV_OPT_AUTO_READ_ONLY, "on"); + assert((bdrv_flags & BDRV_O_CACHE_MASK) == 0); + + if (runstate_check(RUN_STATE_INMIGRATE)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Make-bdrv_is_writable-public.patch b/SOURCES/kvm-block-Make-bdrv_is_writable-public.patch new file mode 100644 index 0000000..f98ad43 --- /dev/null +++ b/SOURCES/kvm-block-Make-bdrv_is_writable-public.patch @@ -0,0 +1,97 @@ +From 2bcabbe52038d80315596e3606bf3afb80e1ce80 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 18:00:53 +0200 +Subject: [PATCH 056/268] block: Make bdrv_is_writable() public + +RH-Author: Max Reitz +Message-id: <20180618180055.22739-2-mreitz@redhat.com> +Patchwork-id: 80792 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/3] block: Make bdrv_is_writable() public +Bugzilla: 1588039 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +This is a useful function for the whole block layer, so make it public. +At the same time, users outside of block.c probably do not need to make +use of the reopen functionality, so rename the current function to +bdrv_is_writable_after_reopen() create a new bdrv_is_writable() function +that just passes NULL to it for the reopen queue. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Max Reitz +Message-id: 20180606193702.7113-2-mreitz@redhat.com +Reviewed-by: John Snow +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit cc022140972f8b6ac3973c12ccf9dd6b1d2fd200) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block.c | 17 ++++++++++++++--- + include/block/block.h | 1 + + 2 files changed, 15 insertions(+), 3 deletions(-) + +diff --git a/block.c b/block.c +index 3c3e8fd..982d54e 100644 +--- a/block.c ++++ b/block.c +@@ -1621,13 +1621,24 @@ static int bdrv_reopen_get_flags(BlockReopenQueue *q, BlockDriverState *bs) + + /* Returns whether the image file can be written to after the reopen queue @q + * has been successfully applied, or right now if @q is NULL. */ +-static bool bdrv_is_writable(BlockDriverState *bs, BlockReopenQueue *q) ++static bool bdrv_is_writable_after_reopen(BlockDriverState *bs, ++ BlockReopenQueue *q) + { + int flags = bdrv_reopen_get_flags(q, bs); + + return (flags & (BDRV_O_RDWR | BDRV_O_INACTIVE)) == BDRV_O_RDWR; + } + ++/* ++ * Return whether the BDS can be written to. This is not necessarily ++ * the same as !bdrv_is_read_only(bs), as inactivated images may not ++ * be written to but do not count as read-only images. ++ */ ++bool bdrv_is_writable(BlockDriverState *bs) ++{ ++ return bdrv_is_writable_after_reopen(bs, NULL); ++} ++ + static void bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs, + BdrvChild *c, const BdrvChildRole *role, + BlockReopenQueue *reopen_queue, +@@ -1665,7 +1676,7 @@ static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q, + + /* Write permissions never work with read-only images */ + if ((cumulative_perms & (BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED)) && +- !bdrv_is_writable(bs, q)) ++ !bdrv_is_writable_after_reopen(bs, q)) + { + error_setg(errp, "Block node is read-only"); + return -EPERM; +@@ -1957,7 +1968,7 @@ void bdrv_format_default_perms(BlockDriverState *bs, BdrvChild *c, + &perm, &shared); + + /* Format drivers may touch metadata even if the guest doesn't write */ +- if (bdrv_is_writable(bs, reopen_queue)) { ++ if (bdrv_is_writable_after_reopen(bs, reopen_queue)) { + perm |= BLK_PERM_WRITE | BLK_PERM_RESIZE; + } + +diff --git a/include/block/block.h b/include/block/block.h +index 3894edd..06cd772 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -407,6 +407,7 @@ bool bdrv_is_read_only(BlockDriverState *bs); + int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only, + bool ignore_allow_rdw, Error **errp); + int bdrv_set_read_only(BlockDriverState *bs, bool read_only, Error **errp); ++bool bdrv_is_writable(BlockDriverState *bs); + bool bdrv_is_sg(BlockDriverState *bs); + bool bdrv_is_inserted(BlockDriverState *bs); + void bdrv_lock_medium(BlockDriverState *bs, bool locked); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Make-more-block-drivers-compile-time-configura.patch b/SOURCES/kvm-block-Make-more-block-drivers-compile-time-configura.patch new file mode 100644 index 0000000..1904264 --- /dev/null +++ b/SOURCES/kvm-block-Make-more-block-drivers-compile-time-configura.patch @@ -0,0 +1,229 @@ +From 274453614da9cb63ec8be5d0525c8b709fc51333 Mon Sep 17 00:00:00 2001 +From: Jeff Cody +Date: Tue, 13 Nov 2018 15:26:09 +0000 +Subject: [PATCH 1/2] block: Make more block drivers compile-time configurable +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Markus Armbruster +Message-id: <20181113162610.30902-2-armbru@redhat.com> +Patchwork-id: 83001 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v3 1/2] block: Make more block drivers compile-time configurable +Bugzilla: 1598842 1598842 +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Danilo de Paula + +From: Jeff Cody +This adds configure options to control the following block drivers: + +* Bochs +* Cloop +* Dmg +* Qcow (V1) +* Vdi +* Vvfat +* qed +* parallels +* sheepdog + +Each of these defaults to being enabled. + +Signed-off-by: Jeff Cody +Signed-off-by: Markus Armbruster +Message-id: 20181107063644.2254-1-armbru@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 2f74013655e562cb97271e2ed75144ea15f0d670) + +Straightforward conflict due to lack of commit bfcc224e3cf "block: Add +blklogwrites". + +Signed-off-by: Markus Armbruster +Acked-by: Philippe Mathieu-Daudé +--- + block/Makefile.objs | 22 +++++++++---- + configure | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 107 insertions(+), 6 deletions(-) + +diff --git a/block/Makefile.objs b/block/Makefile.objs +index be2cda1..037c76b 100644 +--- a/block/Makefile.objs ++++ b/block/Makefile.objs +@@ -1,10 +1,18 @@ +-block-obj-y += raw-format.o qcow.o vdi.o vmdk.o cloop.o bochs.o vpc.o vvfat.o dmg.o ++block-obj-y += raw-format.o vmdk.o vpc.o ++block-obj-$(CONFIG_QCOW1) += qcow.o ++block-obj-$(CONFIG_VDI) += vdi.o ++block-obj-$(CONFIG_CLOOP) += cloop.o ++block-obj-$(CONFIG_BOCHS) += bochs.o ++block-obj-$(CONFIG_VVFAT) += vvfat.o ++block-obj-$(CONFIG_DMG) += dmg.o ++ + block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o +-block-obj-y += qed.o qed-l2-cache.o qed-table.o qed-cluster.o +-block-obj-y += qed-check.o ++block-obj-$(CONFIG_QED) += qed.o qed-l2-cache.o qed-table.o qed-cluster.o ++block-obj-$(CONFIG_QED) += qed-check.o + block-obj-y += vhdx.o vhdx-endian.o vhdx-log.o + block-obj-y += quorum.o +-block-obj-y += parallels.o blkdebug.o blkverify.o blkreplay.o ++block-obj-y += blkdebug.o blkverify.o blkreplay.o ++block-obj-$(CONFIG_PARALLELS) += parallels.o + block-obj-y += block-backend.o snapshot.o qapi.o + block-obj-$(CONFIG_WIN32) += file-win32.o win32-aio.o + block-obj-$(CONFIG_POSIX) += file-posix.o +@@ -13,7 +21,8 @@ block-obj-y += null.o mirror.o commit.o io.o create.o + block-obj-y += throttle-groups.o + block-obj-$(CONFIG_LINUX) += nvme.o + +-block-obj-y += nbd.o nbd-client.o sheepdog.o ++block-obj-y += nbd.o nbd-client.o ++block-obj-$(CONFIG_SHEEPDOG) += sheepdog.o + block-obj-$(CONFIG_LIBISCSI) += iscsi.o + block-obj-$(if $(CONFIG_LIBISCSI),y,n) += iscsi-opts.o + block-obj-$(CONFIG_LIBNFS) += nfs.o +@@ -44,7 +53,8 @@ gluster.o-libs := $(GLUSTERFS_LIBS) + vxhs.o-libs := $(VXHS_LIBS) + ssh.o-cflags := $(LIBSSH2_CFLAGS) + ssh.o-libs := $(LIBSSH2_LIBS) +-block-obj-$(if $(CONFIG_BZIP2),m,n) += dmg-bz2.o ++block-obj-dmg-bz2-$(CONFIG_BZIP2) += dmg-bz2.o ++block-obj-$(if $(CONFIG_DMG),m,n) += $(block-obj-dmg-bz2-y) + dmg-bz2.o-libs := $(BZIP2_LIBS) + qcow.o-libs := -lz + linux-aio.o-libs := -laio +diff --git a/configure b/configure +index 0cb2b79..9446f49 100755 +--- a/configure ++++ b/configure +@@ -450,6 +450,15 @@ tcmalloc="no" + jemalloc="no" + replication="yes" + vxhs="" ++bochs="yes" ++cloop="yes" ++dmg="yes" ++qcow1="yes" ++vdi="yes" ++vvfat="yes" ++qed="yes" ++parallels="yes" ++sheepdog="yes" + libxml2="" + libudev="no" + +@@ -1354,6 +1363,42 @@ for opt do + ;; + --enable-vxhs) vxhs="yes" + ;; ++ --disable-bochs) bochs="no" ++ ;; ++ --enable-bochs) bochs="yes" ++ ;; ++ --disable-cloop) cloop="no" ++ ;; ++ --enable-cloop) cloop="yes" ++ ;; ++ --disable-dmg) dmg="no" ++ ;; ++ --enable-dmg) dmg="yes" ++ ;; ++ --disable-qcow1) qcow1="no" ++ ;; ++ --enable-qcow1) qcow1="yes" ++ ;; ++ --disable-vdi) vdi="no" ++ ;; ++ --enable-vdi) vdi="yes" ++ ;; ++ --disable-vvfat) vvfat="no" ++ ;; ++ --enable-vvfat) vvfat="yes" ++ ;; ++ --disable-qed) qed="no" ++ ;; ++ --enable-qed) qed="yes" ++ ;; ++ --disable-parallels) parallels="no" ++ ;; ++ --enable-parallels) parallels="yes" ++ ;; ++ --disable-sheepdog) sheepdog="no" ++ ;; ++ --enable-sheepdog) sheepdog="yes" ++ ;; + --disable-vhost-user) vhost_user="no" + ;; + --enable-vhost-user) +@@ -1630,6 +1675,15 @@ disabled with --disable-FEATURE, default is enabled if available: + qom-cast-debug cast debugging support + tools build qemu-io, qemu-nbd and qemu-image tools + vxhs Veritas HyperScale vDisk backend support ++ bochs bochs image format support ++ cloop cloop image format support ++ dmg dmg image format support ++ qcow1 qcow v1 image format support ++ vdi vdi image format support ++ vvfat vvfat image format support ++ qed qed image format support ++ parallels parallels image format support ++ sheepdog sheepdog block driver support + crypto-afalg Linux AF_ALG crypto backend driver + vhost-user vhost-user support + capstone capstone disassembler support +@@ -5870,6 +5924,15 @@ echo "jemalloc support $jemalloc" + echo "avx2 optimization $avx2_opt" + echo "replication support $replication" + echo "VxHS block device $vxhs" ++echo "bochs support $bochs" ++echo "cloop support $cloop" ++echo "dmg support $dmg" ++echo "qcow v1 support $qcow1" ++echo "vdi support $vdi" ++echo "vvfat support $vvfat" ++echo "qed support $qed" ++echo "parallels support $parallels" ++echo "sheepdog support $sheepdog" + echo "capstone $capstone" + echo "libudev $libudev" + +@@ -6619,6 +6682,34 @@ if test "$vxhs" = "yes" ; then + echo "VXHS_LIBS= -lssl" >> $config_host_mak + fi + ++if test "$bochs" = "yes" ; then ++ echo "CONFIG_BOCHS=y" >> $config_host_mak ++fi ++if test "$cloop" = "yes" ; then ++ echo "CONFIG_CLOOP=y" >> $config_host_mak ++fi ++if test "$dmg" = "yes" ; then ++ echo "CONFIG_DMG=y" >> $config_host_mak ++fi ++if test "$qcow1" = "yes" ; then ++ echo "CONFIG_QCOW1=y" >> $config_host_mak ++fi ++if test "$vdi" = "yes" ; then ++ echo "CONFIG_VDI=y" >> $config_host_mak ++fi ++if test "$vvfat" = "yes" ; then ++ echo "CONFIG_VVFAT=y" >> $config_host_mak ++fi ++if test "$qed" = "yes" ; then ++ echo "CONFIG_QED=y" >> $config_host_mak ++fi ++if test "$parallels" = "yes" ; then ++ echo "CONFIG_PARALLELS=y" >> $config_host_mak ++fi ++if test "$sheepdog" = "yes" ; then ++ echo "CONFIG_SHEEPDOG=y" >> $config_host_mak ++fi ++ + if test "$tcg_interpreter" = "yes"; then + QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/tci $QEMU_INCLUDES" + elif test "$ARCH" = "sparc64" ; then +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Make-remaining-uses-of-qobject-input-visitor-m.patch b/SOURCES/kvm-block-Make-remaining-uses-of-qobject-input-visitor-m.patch new file mode 100644 index 0000000..dd40d92 --- /dev/null +++ b/SOURCES/kvm-block-Make-remaining-uses-of-qobject-input-visitor-m.patch @@ -0,0 +1,121 @@ +From d8f04a096c6c5ec035c17ef41e1af1801beb27a6 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:21 +0200 +Subject: [PATCH 023/268] block: Make remaining uses of qobject input visitor + more robust + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-15-armbru@redhat.com> +Patchwork-id: 80730 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 14/23] block: Make remaining uses of qobject input visitor more robust +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Remaining uses of qobject_input_visitor_new_keyval() in the block +subsystem: + +* block_crypto_open_opts_init() + Currently doesn't visit any non-string scalars, thus safe. It's + called from + - block_crypto_open_luks() + Creates the QDict with qemu_opts_to_qdict_filtered(), which + creates only string scalars, but has a TODO asking for other types. + - qcow_open() + - qcow2_open(), qcow2_co_invalidate_cache(), qcow2_reopen_prepare() + +* block_crypto_create_opts_init(), called from + - block_crypto_co_create_opts_luks() + Also creates the QDict with qemu_opts_to_qdict_filtered(). + +* vdi_co_create_opts() + Also creates the QDict with qemu_opts_to_qdict_filtered(). + +Replace these uses by qobject_input_visitor_new_flat_confused() for +robustness. This adds crumpling. Right now, that's a no-op, but if +we ever extend these things in non-flat ways, crumpling will be +needed. + +Signed-off-by: Markus Armbruster +Signed-off-by: Kevin Wolf +(cherry picked from commit f853465aacb45dbb07e4cc9815e39b55e10dc690) +Signed-off-by: Miroslav Rezanina +--- + block/crypto.c | 12 +++++++++--- + block/vdi.c | 8 ++++++-- + 2 files changed, 15 insertions(+), 5 deletions(-) + +diff --git a/block/crypto.c b/block/crypto.c +index 7e7ad2d..f5151f4 100644 +--- a/block/crypto.c ++++ b/block/crypto.c +@@ -21,11 +21,11 @@ + #include "qemu/osdep.h" + + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "crypto/block.h" + #include "qapi/opts-visitor.h" + #include "qapi/qapi-visit-crypto.h" +-#include "qapi/qmp/qdict.h" + #include "qapi/qobject-input-visitor.h" + #include "qapi/error.h" + #include "qemu/option.h" +@@ -159,7 +159,10 @@ block_crypto_open_opts_init(QCryptoBlockFormat format, + ret = g_new0(QCryptoBlockOpenOptions, 1); + ret->format = format; + +- v = qobject_input_visitor_new_keyval(QOBJECT(opts)); ++ v = qobject_input_visitor_new_flat_confused(opts, &local_err); ++ if (local_err) { ++ goto out; ++ } + + visit_start_struct(v, NULL, NULL, 0, &local_err); + if (local_err) { +@@ -210,7 +213,10 @@ block_crypto_create_opts_init(QCryptoBlockFormat format, + ret = g_new0(QCryptoBlockCreateOptions, 1); + ret->format = format; + +- v = qobject_input_visitor_new_keyval(QOBJECT(opts)); ++ v = qobject_input_visitor_new_flat_confused(opts, &local_err); ++ if (local_err) { ++ goto out; ++ } + + visit_start_struct(v, NULL, NULL, 0, &local_err); + if (local_err) { +diff --git a/block/vdi.c b/block/vdi.c +index 96a22b8..41859a8 100644 +--- a/block/vdi.c ++++ b/block/vdi.c +@@ -51,10 +51,10 @@ + + #include "qemu/osdep.h" + #include "qapi/error.h" +-#include "qapi/qmp/qdict.h" + #include "qapi/qobject-input-visitor.h" + #include "qapi/qapi-visit-block-core.h" + #include "block/block_int.h" ++#include "block/qdict.h" + #include "sysemu/block-backend.h" + #include "qemu/module.h" + #include "qemu/option.h" +@@ -933,7 +933,11 @@ static int coroutine_fn vdi_co_create_opts(const char *filename, QemuOpts *opts, + } + + /* Get the QAPI object */ +- v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); ++ v = qobject_input_visitor_new_flat_confused(qdict, errp); ++ if (!v) { ++ ret = -EINVAL; ++ goto done; ++ } + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Move-bdrv_drain_all_begin-out-of-coroutine-con.patch b/SOURCES/kvm-block-Move-bdrv_drain_all_begin-out-of-coroutine-con.patch new file mode 100644 index 0000000..c7d94ba --- /dev/null +++ b/SOURCES/kvm-block-Move-bdrv_drain_all_begin-out-of-coroutine-con.patch @@ -0,0 +1,78 @@ +From 3ba36d5627afb6009c91c71ae8709d5dac2cc03d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:46 +0100 +Subject: [PATCH 20/49] block: Move bdrv_drain_all_begin() out of coroutine + context + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-8-kwolf@redhat.com> +Patchwork-id: 82599 +O-Subject: [RHEL-8 qemu-kvm PATCH 17/44] block: Move bdrv_drain_all_begin() out of coroutine context +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Before we can introduce a single polling loop for all nodes in +bdrv_drain_all_begin(), we must make sure to run it outside of coroutine +context like we already do for bdrv_do_drained_begin(). + +Signed-off-by: Kevin Wolf +(cherry picked from commit c8ca33d06def97d909a8511377b82994ae4e5981) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 22 +++++++++++++++++----- + 1 file changed, 17 insertions(+), 5 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 23fe069..b58c91b 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -263,11 +263,16 @@ static void bdrv_co_drain_bh_cb(void *opaque) + Coroutine *co = data->co; + BlockDriverState *bs = data->bs; + +- bdrv_dec_in_flight(bs); +- if (data->begin) { +- bdrv_do_drained_begin(bs, data->recursive, data->parent, data->poll); ++ if (bs) { ++ bdrv_dec_in_flight(bs); ++ if (data->begin) { ++ bdrv_do_drained_begin(bs, data->recursive, data->parent, data->poll); ++ } else { ++ bdrv_do_drained_end(bs, data->recursive, data->parent); ++ } + } else { +- bdrv_do_drained_end(bs, data->recursive, data->parent); ++ assert(data->begin); ++ bdrv_drain_all_begin(); + } + + data->done = true; +@@ -293,7 +298,9 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + .parent = parent, + .poll = poll, + }; +- bdrv_inc_in_flight(bs); ++ if (bs) { ++ bdrv_inc_in_flight(bs); ++ } + aio_bh_schedule_oneshot(bdrv_get_aio_context(bs), + bdrv_co_drain_bh_cb, &data); + +@@ -463,6 +470,11 @@ void bdrv_drain_all_begin(void) + BlockDriverState *bs; + BdrvNextIterator it; + ++ if (qemu_in_coroutine()) { ++ bdrv_co_yield_to_drain(NULL, true, false, NULL, true); ++ return; ++ } ++ + /* BDRV_POLL_WHILE() for a node can only be called from its own I/O thread + * or the main loop AioContext. We potentially use BDRV_POLL_WHILE() on + * nodes in several different AioContexts, so make sure we're in the main +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Move-bdrv_truncate-implementation-to-io.c.patch b/SOURCES/kvm-block-Move-bdrv_truncate-implementation-to-io.c.patch new file mode 100644 index 0000000..1ab794f --- /dev/null +++ b/SOURCES/kvm-block-Move-bdrv_truncate-implementation-to-io.c.patch @@ -0,0 +1,297 @@ +From 713bea6a36ec53cffb617f431c71afd60766bbc5 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 14:42:56 +0200 +Subject: [PATCH 211/268] block: Move bdrv_truncate() implementation to io.c + +RH-Author: Kevin Wolf +Message-id: <20180712144258.17303-5-kwolf@redhat.com> +Patchwork-id: 81328 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 4/6] block: Move bdrv_truncate() implementation to io.c +Bugzilla: 1595173 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: John Snow + +This moves the bdrv_truncate() implementation from block.c to block/io.c +so it can have access to the tracked requests infrastructure. + +This involves making refresh_total_sectors() public (in block_int.h). + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 3d9f2d2af63fda5f0404fb85ea80161837a4e4e3) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block.c | 111 +--------------------------------------------- + block/io.c | 109 +++++++++++++++++++++++++++++++++++++++++++++ + include/block/block_int.h | 2 + + 3 files changed, 112 insertions(+), 110 deletions(-) + +diff --git a/block.c b/block.c +index 0af08ca..10a1ece 100644 +--- a/block.c ++++ b/block.c +@@ -721,7 +721,7 @@ static int find_image_format(BlockBackend *file, const char *filename, + * Set the current 'total_sectors' value + * Return 0 on success, -errno on error. + */ +-static int refresh_total_sectors(BlockDriverState *bs, int64_t hint) ++int refresh_total_sectors(BlockDriverState *bs, int64_t hint) + { + BlockDriver *drv = bs->drv; + +@@ -2200,16 +2200,6 @@ static void bdrv_parent_cb_change_media(BlockDriverState *bs, bool load) + } + } + +-static void bdrv_parent_cb_resize(BlockDriverState *bs) +-{ +- BdrvChild *c; +- QLIST_FOREACH(c, &bs->parents, next_parent) { +- if (c->role->resize) { +- c->role->resize(c); +- } +- } +-} +- + /* + * Sets the backing file link of a BDS. A new reference is created; callers + * which don't need their own reference any more must call bdrv_unref(). +@@ -3736,105 +3726,6 @@ exit: + } + + /** +- * Truncate file to 'offset' bytes (needed only for file protocols) +- */ +-int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, +- PreallocMode prealloc, Error **errp) +-{ +- BlockDriverState *bs = child->bs; +- BlockDriver *drv = bs->drv; +- int ret; +- +- assert(child->perm & BLK_PERM_RESIZE); +- +- /* if bs->drv == NULL, bs is closed, so there's nothing to do here */ +- if (!drv) { +- error_setg(errp, "No medium inserted"); +- return -ENOMEDIUM; +- } +- if (offset < 0) { +- error_setg(errp, "Image size cannot be negative"); +- return -EINVAL; +- } +- +- bdrv_inc_in_flight(bs); +- +- if (!drv->bdrv_co_truncate) { +- if (bs->file && drv->is_filter) { +- ret = bdrv_co_truncate(bs->file, offset, prealloc, errp); +- goto out; +- } +- error_setg(errp, "Image format driver does not support resize"); +- ret = -ENOTSUP; +- goto out; +- } +- if (bs->read_only) { +- error_setg(errp, "Image is read-only"); +- ret = -EACCES; +- goto out; +- } +- +- assert(!(bs->open_flags & BDRV_O_INACTIVE)); +- +- ret = drv->bdrv_co_truncate(bs, offset, prealloc, errp); +- if (ret < 0) { +- goto out; +- } +- ret = refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); +- if (ret < 0) { +- error_setg_errno(errp, -ret, "Could not refresh total sector count"); +- } else { +- offset = bs->total_sectors * BDRV_SECTOR_SIZE; +- } +- bdrv_dirty_bitmap_truncate(bs, offset); +- bdrv_parent_cb_resize(bs); +- atomic_inc(&bs->write_gen); +- +-out: +- bdrv_dec_in_flight(bs); +- return ret; +-} +- +-typedef struct TruncateCo { +- BdrvChild *child; +- int64_t offset; +- PreallocMode prealloc; +- Error **errp; +- int ret; +-} TruncateCo; +- +-static void coroutine_fn bdrv_truncate_co_entry(void *opaque) +-{ +- TruncateCo *tco = opaque; +- tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->prealloc, +- tco->errp); +-} +- +-int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, +- Error **errp) +-{ +- Coroutine *co; +- TruncateCo tco = { +- .child = child, +- .offset = offset, +- .prealloc = prealloc, +- .errp = errp, +- .ret = NOT_DONE, +- }; +- +- if (qemu_in_coroutine()) { +- /* Fast-path if already in coroutine context */ +- bdrv_truncate_co_entry(&tco); +- } else { +- co = qemu_coroutine_create(bdrv_truncate_co_entry, &tco); +- qemu_coroutine_enter(co); +- BDRV_POLL_WHILE(child->bs, tco.ret == NOT_DONE); +- } +- +- return tco.ret; +-} +- +-/** + * Length of a allocated file in bytes. Sparse files are counted by actual + * allocated space. Return < 0 if error or unknown. + */ +diff --git a/block/io.c b/block/io.c +index 5c043a4..32a82e3 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2929,3 +2929,112 @@ int coroutine_fn bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, + bdrv_dec_in_flight(dst_bs); + return ret; + } ++ ++static void bdrv_parent_cb_resize(BlockDriverState *bs) ++{ ++ BdrvChild *c; ++ QLIST_FOREACH(c, &bs->parents, next_parent) { ++ if (c->role->resize) { ++ c->role->resize(c); ++ } ++ } ++} ++ ++/** ++ * Truncate file to 'offset' bytes (needed only for file protocols) ++ */ ++int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, ++ PreallocMode prealloc, Error **errp) ++{ ++ BlockDriverState *bs = child->bs; ++ BlockDriver *drv = bs->drv; ++ int ret; ++ ++ assert(child->perm & BLK_PERM_RESIZE); ++ ++ /* if bs->drv == NULL, bs is closed, so there's nothing to do here */ ++ if (!drv) { ++ error_setg(errp, "No medium inserted"); ++ return -ENOMEDIUM; ++ } ++ if (offset < 0) { ++ error_setg(errp, "Image size cannot be negative"); ++ return -EINVAL; ++ } ++ ++ bdrv_inc_in_flight(bs); ++ ++ if (!drv->bdrv_co_truncate) { ++ if (bs->file && drv->is_filter) { ++ ret = bdrv_co_truncate(bs->file, offset, prealloc, errp); ++ goto out; ++ } ++ error_setg(errp, "Image format driver does not support resize"); ++ ret = -ENOTSUP; ++ goto out; ++ } ++ if (bs->read_only) { ++ error_setg(errp, "Image is read-only"); ++ ret = -EACCES; ++ goto out; ++ } ++ ++ assert(!(bs->open_flags & BDRV_O_INACTIVE)); ++ ++ ret = drv->bdrv_co_truncate(bs, offset, prealloc, errp); ++ if (ret < 0) { ++ goto out; ++ } ++ ret = refresh_total_sectors(bs, offset >> BDRV_SECTOR_BITS); ++ if (ret < 0) { ++ error_setg_errno(errp, -ret, "Could not refresh total sector count"); ++ } else { ++ offset = bs->total_sectors * BDRV_SECTOR_SIZE; ++ } ++ bdrv_dirty_bitmap_truncate(bs, offset); ++ bdrv_parent_cb_resize(bs); ++ atomic_inc(&bs->write_gen); ++ ++out: ++ bdrv_dec_in_flight(bs); ++ return ret; ++} ++ ++typedef struct TruncateCo { ++ BdrvChild *child; ++ int64_t offset; ++ PreallocMode prealloc; ++ Error **errp; ++ int ret; ++} TruncateCo; ++ ++static void coroutine_fn bdrv_truncate_co_entry(void *opaque) ++{ ++ TruncateCo *tco = opaque; ++ tco->ret = bdrv_co_truncate(tco->child, tco->offset, tco->prealloc, ++ tco->errp); ++} ++ ++int bdrv_truncate(BdrvChild *child, int64_t offset, PreallocMode prealloc, ++ Error **errp) ++{ ++ Coroutine *co; ++ TruncateCo tco = { ++ .child = child, ++ .offset = offset, ++ .prealloc = prealloc, ++ .errp = errp, ++ .ret = NOT_DONE, ++ }; ++ ++ if (qemu_in_coroutine()) { ++ /* Fast-path if already in coroutine context */ ++ bdrv_truncate_co_entry(&tco); ++ } else { ++ co = qemu_coroutine_create(bdrv_truncate_co_entry, &tco); ++ qemu_coroutine_enter(co); ++ BDRV_POLL_WHILE(child->bs, tco.ret == NOT_DONE); ++ } ++ ++ return tco.ret; ++} +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 46b2f87..6a844ec 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -1129,4 +1129,6 @@ int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags); + ++int refresh_total_sectors(BlockDriverState *bs, int64_t hint); ++ + #endif /* BLOCK_INT_H */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Move-request-tracking-to-children-in-copy-offl.patch b/SOURCES/kvm-block-Move-request-tracking-to-children-in-copy-offl.patch new file mode 100644 index 0000000..75cf9e7 --- /dev/null +++ b/SOURCES/kvm-block-Move-request-tracking-to-children-in-copy-offl.patch @@ -0,0 +1,124 @@ +From 49b0a4709b289f28c97633d8b3145213df1d8fc1 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 24 Jul 2018 12:43:07 +0200 +Subject: [PATCH 231/268] block: Move request tracking to children in copy + offloading + +RH-Author: John Snow +Message-id: <20180718225511.14878-14-jsnow@redhat.com> +Patchwork-id: 81393 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 13/35] block: Move request tracking to children in copy offloading +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Fam Zheng + +in_flight and tracked requests need to be tracked in every layer during +recursion. For now the only user is qemu-img convert where overlapping +requests and IOThreads don't exist, therefore this change doesn't make +much difference form user point of view, but it is incorrect as part of +the API. Fix it. + +Reported-by: Kevin Wolf +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Kevin Wolf +(cherry picked from commit 37aec7d75eb0d035a0db4f2cf9ad8b1b0c10f91b) +Signed-off-by: John Snow +--- + block/io.c | 60 +++++++++++++++++++++++++++++------------------------------- + 1 file changed, 29 insertions(+), 31 deletions(-) + +diff --git a/block/io.c b/block/io.c +index ac36d1c..136a5d0 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2847,6 +2847,9 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + BdrvRequestFlags flags, + bool recurse_src) + { ++ BdrvTrackedRequest src_req, dst_req; ++ BlockDriverState *src_bs = src->bs; ++ BlockDriverState *dst_bs = dst->bs; + int ret; + + if (!src || !dst || !src->bs || !dst->bs) { +@@ -2870,17 +2873,32 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + || src->bs->encrypted || dst->bs->encrypted) { + return -ENOTSUP; + } ++ ++ bdrv_inc_in_flight(src_bs); ++ bdrv_inc_in_flight(dst_bs); ++ tracked_request_begin(&src_req, src_bs, src_offset, ++ bytes, BDRV_TRACKED_READ); ++ tracked_request_begin(&dst_req, dst_bs, dst_offset, ++ bytes, BDRV_TRACKED_WRITE); ++ ++ wait_serialising_requests(&src_req); ++ wait_serialising_requests(&dst_req); + if (recurse_src) { +- return src->bs->drv->bdrv_co_copy_range_from(src->bs, +- src, src_offset, +- dst, dst_offset, +- bytes, flags); ++ ret = src->bs->drv->bdrv_co_copy_range_from(src->bs, ++ src, src_offset, ++ dst, dst_offset, ++ bytes, flags); + } else { +- return dst->bs->drv->bdrv_co_copy_range_to(dst->bs, +- src, src_offset, +- dst, dst_offset, +- bytes, flags); ++ ret = dst->bs->drv->bdrv_co_copy_range_to(dst->bs, ++ src, src_offset, ++ dst, dst_offset, ++ bytes, flags); + } ++ tracked_request_end(&src_req); ++ tracked_request_end(&dst_req); ++ bdrv_dec_in_flight(src_bs); ++ bdrv_dec_in_flight(dst_bs); ++ return ret; + } + + /* Copy range from @src to @dst. +@@ -2911,29 +2929,9 @@ int coroutine_fn bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, + uint64_t bytes, BdrvRequestFlags flags) + { +- BdrvTrackedRequest src_req, dst_req; +- BlockDriverState *src_bs = src->bs; +- BlockDriverState *dst_bs = dst->bs; +- int ret; +- +- bdrv_inc_in_flight(src_bs); +- bdrv_inc_in_flight(dst_bs); +- tracked_request_begin(&src_req, src_bs, src_offset, +- bytes, BDRV_TRACKED_READ); +- tracked_request_begin(&dst_req, dst_bs, dst_offset, +- bytes, BDRV_TRACKED_WRITE); +- +- wait_serialising_requests(&src_req); +- wait_serialising_requests(&dst_req); +- ret = bdrv_co_copy_range_from(src, src_offset, +- dst, dst_offset, +- bytes, flags); +- +- tracked_request_end(&src_req); +- tracked_request_end(&dst_req); +- bdrv_dec_in_flight(src_bs); +- bdrv_dec_in_flight(dst_bs); +- return ret; ++ return bdrv_co_copy_range_from(src, src_offset, ++ dst, dst_offset, ++ bytes, flags); + } + + static void bdrv_parent_cb_resize(BlockDriverState *bs) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Poll-after-drain-on-attaching-a-node.patch b/SOURCES/kvm-block-Poll-after-drain-on-attaching-a-node.patch new file mode 100644 index 0000000..1ace071 --- /dev/null +++ b/SOURCES/kvm-block-Poll-after-drain-on-attaching-a-node.patch @@ -0,0 +1,150 @@ +From b51c41bfbf422dd3e15b946faa1695571bcbe2df Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:50 +0100 +Subject: [PATCH 24/49] block: Poll after drain on attaching a node + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-12-kwolf@redhat.com> +Patchwork-id: 82601 +O-Subject: [RHEL-8 qemu-kvm PATCH 21/44] block: Poll after drain on attaching a node +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Commit dcf94a23b1 ('block: Don't poll in parent drain callbacks') +removed polling in bdrv_child_cb_drained_begin() on the grounds that the +original bdrv_drain() already will poll and BdrvChildRole.drained_begin +calls must not cause graph changes (and therefore must not call +aio_poll() or the recursion through the graph will break. + +This reasoning is correct for calls through bdrv_do_drained_begin(). +However, BdrvChildRole.drained_begin is also called when a node that is +already in a drained section (i.e. bdrv_do_drained_begin() has already +returned and therefore can't poll any more) is attached to a new parent. +In this case, we must explicitly poll to have all requests completed +before the drained new child can be attached to the parent. + +In bdrv_replace_child_noperm(), we know that we're not inside the +recursion of bdrv_do_drained_begin() because graph changes are not +allowed there, and bdrv_replace_child_noperm() is a graph change. The +call of BdrvChildRole.drained_begin() must therefore be followed by a +BDRV_POLL_WHILE() that waits for the completion of requests. + +Reported-by: Max Reitz +Signed-off-by: Kevin Wolf +(cherry picked from commit 4be6a6d118123340f16fb8b3bf45220d0f8ed6d4) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 2 +- + block/io.c | 26 ++++++++++++++++++++------ + include/block/block.h | 8 ++++++++ + include/block/block_int.h | 3 +++ + 4 files changed, 32 insertions(+), 7 deletions(-) + +diff --git a/block.c b/block.c +index eea9c6f..e89b5e3 100644 +--- a/block.c ++++ b/block.c +@@ -2072,7 +2072,7 @@ static void bdrv_replace_child_noperm(BdrvChild *child, + } + assert(num >= 0); + for (i = 0; i < num; i++) { +- child->role->drained_begin(child); ++ bdrv_parent_drained_begin_single(child, true); + } + } + +diff --git a/block/io.c b/block/io.c +index 38ae299..d404088 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -52,9 +52,7 @@ void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore, + if (c == ignore || (ignore_bds_parents && c->role->parent_is_bds)) { + continue; + } +- if (c->role->drained_begin) { +- c->role->drained_begin(c); +- } ++ bdrv_parent_drained_begin_single(c, false); + } + } + +@@ -73,6 +71,14 @@ void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore, + } + } + ++static bool bdrv_parent_drained_poll_single(BdrvChild *c) ++{ ++ if (c->role->drained_poll) { ++ return c->role->drained_poll(c); ++ } ++ return false; ++} ++ + static bool bdrv_parent_drained_poll(BlockDriverState *bs, BdrvChild *ignore, + bool ignore_bds_parents) + { +@@ -83,14 +89,22 @@ static bool bdrv_parent_drained_poll(BlockDriverState *bs, BdrvChild *ignore, + if (c == ignore || (ignore_bds_parents && c->role->parent_is_bds)) { + continue; + } +- if (c->role->drained_poll) { +- busy |= c->role->drained_poll(c); +- } ++ busy |= bdrv_parent_drained_poll_single(c); + } + + return busy; + } + ++void bdrv_parent_drained_begin_single(BdrvChild *c, bool poll) ++{ ++ if (c->role->drained_begin) { ++ c->role->drained_begin(c); ++ } ++ if (poll) { ++ BDRV_POLL_WHILE(c->bs, bdrv_parent_drained_poll_single(c)); ++ } ++} ++ + static void bdrv_merge_limits(BlockLimits *dst, const BlockLimits *src) + { + dst->opt_transfer = MAX(dst->opt_transfer, src->opt_transfer); +diff --git a/include/block/block.h b/include/block/block.h +index f9079ac..356712c 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -590,6 +590,14 @@ void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore, + bool ignore_bds_parents); + + /** ++ * bdrv_parent_drained_begin_single: ++ * ++ * Begin a quiesced section for the parent of @c. If @poll is true, wait for ++ * any pending activity to cease. ++ */ ++void bdrv_parent_drained_begin_single(BdrvChild *c, bool poll); ++ ++/** + * bdrv_parent_drained_end: + * + * End a quiesced section of all users of @bs. This is part of +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 9757d5e..b7806e3 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -610,6 +610,9 @@ struct BdrvChildRole { + * requests after returning from .drained_begin() until .drained_end() is + * called. + * ++ * These functions must not change the graph (and therefore also must not ++ * call aio_poll(), which could change the graph indirectly). ++ * + * Note that this can be nested. If drained_begin() was called twice, new + * I/O is allowed only after drained_end() was called twice, too. + */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Really-pause-block-jobs-on-drain.patch b/SOURCES/kvm-block-Really-pause-block-jobs-on-drain.patch new file mode 100644 index 0000000..b4ef133 --- /dev/null +++ b/SOURCES/kvm-block-Really-pause-block-jobs-on-drain.patch @@ -0,0 +1,360 @@ +From 6dca7cc6632a4b4632786e8a087289dff1ad6ef8 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:41 +0100 +Subject: [PATCH 10/49] block: Really pause block jobs on drain + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-8-kwolf@redhat.com> +Patchwork-id: 82587 +O-Subject: [RHEL-8 qemu-kvm PATCH 07/44] block: Really pause block jobs on drain +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +We already requested that block jobs be paused in .bdrv_drained_begin, +but no guarantee was made that the job was actually inactive at the +point where bdrv_drained_begin() returned. + +This introduces a new callback BdrvChildRole.bdrv_drained_poll() and +uses it to make bdrv_drain_poll() consider block jobs using the node to +be drained. + +For the test case to work as expected, we have to switch from +block_job_sleep_ns() to qemu_co_sleep_ns() so that the test job is even +considered active and must be waited for when draining the node. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 89bd030533e3592ca0a995450dcfc5d53e459e20) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 9 +++++++++ + block/io.c | 40 ++++++++++++++++++++++++++++++++++------ + block/mirror.c | 8 ++++++++ + blockjob.c | 23 +++++++++++++++++++++++ + include/block/block.h | 8 ++++++++ + include/block/block_int.h | 7 +++++++ + include/block/blockjob_int.h | 8 ++++++++ + tests/test-bdrv-drain.c | 18 ++++++++++-------- + 8 files changed, 107 insertions(+), 14 deletions(-) + +diff --git a/block.c b/block.c +index 10a1ece..0d9698a 100644 +--- a/block.c ++++ b/block.c +@@ -821,6 +821,12 @@ static void bdrv_child_cb_drained_begin(BdrvChild *child) + bdrv_drained_begin(bs); + } + ++static bool bdrv_child_cb_drained_poll(BdrvChild *child) ++{ ++ BlockDriverState *bs = child->opaque; ++ return bdrv_drain_poll(bs, NULL); ++} ++ + static void bdrv_child_cb_drained_end(BdrvChild *child) + { + BlockDriverState *bs = child->opaque; +@@ -905,6 +911,7 @@ const BdrvChildRole child_file = { + .get_parent_desc = bdrv_child_get_parent_desc, + .inherit_options = bdrv_inherited_options, + .drained_begin = bdrv_child_cb_drained_begin, ++ .drained_poll = bdrv_child_cb_drained_poll, + .drained_end = bdrv_child_cb_drained_end, + .attach = bdrv_child_cb_attach, + .detach = bdrv_child_cb_detach, +@@ -929,6 +936,7 @@ const BdrvChildRole child_format = { + .get_parent_desc = bdrv_child_get_parent_desc, + .inherit_options = bdrv_inherited_fmt_options, + .drained_begin = bdrv_child_cb_drained_begin, ++ .drained_poll = bdrv_child_cb_drained_poll, + .drained_end = bdrv_child_cb_drained_end, + .attach = bdrv_child_cb_attach, + .detach = bdrv_child_cb_detach, +@@ -1048,6 +1056,7 @@ const BdrvChildRole child_backing = { + .detach = bdrv_backing_detach, + .inherit_options = bdrv_backing_options, + .drained_begin = bdrv_child_cb_drained_begin, ++ .drained_poll = bdrv_child_cb_drained_poll, + .drained_end = bdrv_child_cb_drained_end, + .inactivate = bdrv_child_cb_inactivate, + .update_filename = bdrv_backing_update_filename, +diff --git a/block/io.c b/block/io.c +index 4d332c3..e260394 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -69,6 +69,23 @@ void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore) + } + } + ++static bool bdrv_parent_drained_poll(BlockDriverState *bs, BdrvChild *ignore) ++{ ++ BdrvChild *c, *next; ++ bool busy = false; ++ ++ QLIST_FOREACH_SAFE(c, &bs->parents, next_parent, next) { ++ if (c == ignore) { ++ continue; ++ } ++ if (c->role->drained_poll) { ++ busy |= c->role->drained_poll(c); ++ } ++ } ++ ++ return busy; ++} ++ + static void bdrv_merge_limits(BlockLimits *dst, const BlockLimits *src) + { + dst->opt_transfer = MAX(dst->opt_transfer, src->opt_transfer); +@@ -182,21 +199,32 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin) + } + + /* Returns true if BDRV_POLL_WHILE() should go into a blocking aio_poll() */ +-static bool bdrv_drain_poll(BlockDriverState *bs) ++bool bdrv_drain_poll(BlockDriverState *bs, BdrvChild *ignore_parent) ++{ ++ if (bdrv_parent_drained_poll(bs, ignore_parent)) { ++ return true; ++ } ++ ++ return atomic_read(&bs->in_flight); ++} ++ ++static bool bdrv_drain_poll_top_level(BlockDriverState *bs, ++ BdrvChild *ignore_parent) + { + /* Execute pending BHs first and check everything else only after the BHs + * have executed. */ + while (aio_poll(bs->aio_context, false)); +- return atomic_read(&bs->in_flight); ++ ++ return bdrv_drain_poll(bs, ignore_parent); + } + +-static bool bdrv_drain_recurse(BlockDriverState *bs) ++static bool bdrv_drain_recurse(BlockDriverState *bs, BdrvChild *parent) + { + BdrvChild *child, *tmp; + bool waited; + + /* Wait for drained requests to finish */ +- waited = BDRV_POLL_WHILE(bs, bdrv_drain_poll(bs)); ++ waited = BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, parent)); + + QLIST_FOREACH_SAFE(child, &bs->children, next, tmp) { + BlockDriverState *bs = child->bs; +@@ -213,7 +241,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs) + */ + bdrv_ref(bs); + } +- waited |= bdrv_drain_recurse(bs); ++ waited |= bdrv_drain_recurse(bs, child); + if (in_main_loop) { + bdrv_unref(bs); + } +@@ -289,7 +317,7 @@ void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + + bdrv_parent_drained_begin(bs, parent); + bdrv_drain_invoke(bs, true); +- bdrv_drain_recurse(bs); ++ bdrv_drain_recurse(bs, parent); + + if (recursive) { + bs->recursive_quiesce_counter++; +diff --git a/block/mirror.c b/block/mirror.c +index 313e6e9..4b27f71 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -976,6 +976,12 @@ static void mirror_pause(Job *job) + mirror_wait_for_all_io(s); + } + ++static bool mirror_drained_poll(BlockJob *job) ++{ ++ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); ++ return !!s->in_flight; ++} ++ + static void mirror_attached_aio_context(BlockJob *job, AioContext *new_context) + { + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); +@@ -1011,6 +1017,7 @@ static const BlockJobDriver mirror_job_driver = { + .pause = mirror_pause, + .complete = mirror_complete, + }, ++ .drained_poll = mirror_drained_poll, + .attached_aio_context = mirror_attached_aio_context, + .drain = mirror_drain, + }; +@@ -1028,6 +1035,7 @@ static const BlockJobDriver commit_active_job_driver = { + .pause = mirror_pause, + .complete = mirror_complete, + }, ++ .drained_poll = mirror_drained_poll, + .attached_aio_context = mirror_attached_aio_context, + .drain = mirror_drain, + }; +diff --git a/blockjob.c b/blockjob.c +index 0306533..be5903a 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -155,6 +155,28 @@ static void child_job_drained_begin(BdrvChild *c) + job_pause(&job->job); + } + ++static bool child_job_drained_poll(BdrvChild *c) ++{ ++ BlockJob *bjob = c->opaque; ++ Job *job = &bjob->job; ++ const BlockJobDriver *drv = block_job_driver(bjob); ++ ++ /* An inactive or completed job doesn't have any pending requests. Jobs ++ * with !job->busy are either already paused or have a pause point after ++ * being reentered, so no job driver code will run before they pause. */ ++ if (!job->busy || job_is_completed(job) || job->deferred_to_main_loop) { ++ return false; ++ } ++ ++ /* Otherwise, assume that it isn't fully stopped yet, but allow the job to ++ * override this assumption. */ ++ if (drv->drained_poll) { ++ return drv->drained_poll(bjob); ++ } else { ++ return true; ++ } ++} ++ + static void child_job_drained_end(BdrvChild *c) + { + BlockJob *job = c->opaque; +@@ -164,6 +186,7 @@ static void child_job_drained_end(BdrvChild *c) + static const BdrvChildRole child_job = { + .get_parent_desc = child_job_get_parent_desc, + .drained_begin = child_job_drained_begin, ++ .drained_poll = child_job_drained_poll, + .drained_end = child_job_drained_end, + .stay_at_node = true, + }; +diff --git a/include/block/block.h b/include/block/block.h +index 8f87eea..8c91d4c 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -596,6 +596,14 @@ void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore); + void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore); + + /** ++ * bdrv_drain_poll: ++ * ++ * Poll for pending requests in @bs and its parents (except for ++ * @ignore_parent). This is part of bdrv_drained_begin. ++ */ ++bool bdrv_drain_poll(BlockDriverState *bs, BdrvChild *ignore_parent); ++ ++/** + * bdrv_drained_begin: + * + * Begin a quiesced section for exclusive access to the BDS, by disabling +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 341cbe8..beeacde 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -610,6 +610,13 @@ struct BdrvChildRole { + void (*drained_begin)(BdrvChild *child); + void (*drained_end)(BdrvChild *child); + ++ /* ++ * Returns whether the parent has pending requests for the child. This ++ * callback is polled after .drained_begin() has been called until all ++ * activity on the child has stopped. ++ */ ++ bool (*drained_poll)(BdrvChild *child); ++ + /* Notifies the parent that the child has been activated/inactivated (e.g. + * when migration is completing) and it can start/stop requesting + * permissions and doing I/O on it. */ +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 5cd50c6..e4a318d 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -39,6 +39,14 @@ struct BlockJobDriver { + JobDriver job_driver; + + /* ++ * Returns whether the job has pending requests for the child or will ++ * submit new requests before the next pause point. This callback is polled ++ * in the context of draining a job node after requesting that the job be ++ * paused, until all activity on the child has stopped. ++ */ ++ bool (*drained_poll)(BlockJob *job); ++ ++ /* + * If the callback is not NULL, it will be invoked before the job is + * resumed in a new AioContext. This is the place to move any resources + * besides job->blk to the new AioContext. +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index f5d85c9..49786ea 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -681,7 +681,11 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) + + job_transition_to_ready(&s->common.job); + while (!s->should_complete) { +- job_sleep_ns(&s->common.job, 100000); ++ /* Avoid block_job_sleep_ns() because it marks the job as !busy. We ++ * want to emulate some actual activity (probably some I/O) here so ++ * that drain has to wait for this acitivity to stop. */ ++ qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); ++ job_pause_point(&s->common.job); + } + + return 0; +@@ -728,7 +732,7 @@ static void test_blockjob_common(enum drain_type drain_type) + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ ++ g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + + do_drain_begin(drain_type, src); + +@@ -738,15 +742,14 @@ static void test_blockjob_common(enum drain_type drain_type) + } else { + g_assert_cmpint(job->job.pause_count, ==, 1); + } +- /* XXX We don't wait until the job is actually paused. Is this okay? */ +- /* g_assert_true(job->job.paused); */ ++ g_assert_true(job->job.paused); + g_assert_false(job->job.busy); /* The job is paused */ + + do_drain_end(drain_type, src); + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ ++ g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + + do_drain_begin(drain_type, target); + +@@ -756,15 +759,14 @@ static void test_blockjob_common(enum drain_type drain_type) + } else { + g_assert_cmpint(job->job.pause_count, ==, 1); + } +- /* XXX We don't wait until the job is actually paused. Is this okay? */ +- /* g_assert_true(job->job.paused); */ ++ g_assert_true(job->job.paused); + g_assert_false(job->job.busy); /* The job is paused */ + + do_drain_end(drain_type, target); + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ ++ g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + + ret = job_complete_sync(&job->job, &error_abort); + g_assert_cmpint(ret, ==, 0); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Remove-aio_poll-in-bdrv_drain_poll-variants.patch b/SOURCES/kvm-block-Remove-aio_poll-in-bdrv_drain_poll-variants.patch new file mode 100644 index 0000000..99d7961 --- /dev/null +++ b/SOURCES/kvm-block-Remove-aio_poll-in-bdrv_drain_poll-variants.patch @@ -0,0 +1,64 @@ +From 30bdfc5373eab96cb1f3d62ab90b07becd885272 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:07 +0100 +Subject: [PATCH 41/49] block: Remove aio_poll() in bdrv_drain_poll variants + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-29-kwolf@redhat.com> +Patchwork-id: 82619 +O-Subject: [RHEL-8 qemu-kvm PATCH 38/44] block: Remove aio_poll() in bdrv_drain_poll variants +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +bdrv_drain_poll_top_level() was buggy because it didn't release the +AioContext lock of the node to be drained before calling aio_poll(). +This way, callbacks called by aio_poll() would possibly take the lock a +second time and run into a deadlock with a nested AIO_WAIT_WHILE() call. + +However, it turns out that the aio_poll() call isn't actually needed any +more. It was introduced in commit 91af091f923, which is effectively +reverted by this patch. The cases it was supposed to fix are now covered +by bdrv_drain_poll(), which waits for block jobs to reach a quiescent +state. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +Reviewed-by: Max Reitz +(cherry picked from commit 4cf077b59fc73eec29f8b7d082919dbb278bdc86) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 8 -------- + 1 file changed, 8 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 19db35e..3313958 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -266,10 +266,6 @@ bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, + static bool bdrv_drain_poll_top_level(BlockDriverState *bs, bool recursive, + BdrvChild *ignore_parent) + { +- /* Execute pending BHs first and check everything else only after the BHs +- * have executed. */ +- while (aio_poll(bs->aio_context, false)); +- + return bdrv_drain_poll(bs, recursive, ignore_parent, false); + } + +@@ -509,10 +505,6 @@ static bool bdrv_drain_all_poll(void) + BlockDriverState *bs = NULL; + bool result = false; + +- /* Execute pending BHs first (may modify the graph) and check everything +- * else only after the BHs have executed. */ +- while (aio_poll(qemu_get_aio_context(), false)); +- + /* bdrv_drain_poll() can't make changes to the graph and we are holding the + * main AioContext lock, so iterating bdrv_next_all_states() is safe. */ + while ((bs = bdrv_next_all_states(bs))) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Remove-bdrv_drain_recurse.patch b/SOURCES/kvm-block-Remove-bdrv_drain_recurse.patch new file mode 100644 index 0000000..7cc1c80 --- /dev/null +++ b/SOURCES/kvm-block-Remove-bdrv_drain_recurse.patch @@ -0,0 +1,91 @@ +From e6d8bc54a031c61c1fce2c9e1ccdb821b5991c1a Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:42 +0100 +Subject: [PATCH 11/49] block: Remove bdrv_drain_recurse() + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-9-kwolf@redhat.com> +Patchwork-id: 82588 +O-Subject: [RHEL-8 qemu-kvm PATCH 08/44] block: Remove bdrv_drain_recurse() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +For bdrv_drain(), recursively waiting for child node requests is +pointless because we didn't quiesce their parents, so new requests could +come in anyway. Letting the function work only on a single node makes it +more consistent. + +For subtree drains and drain_all, we already have the recursion in +bdrv_do_drained_begin(), so the extra recursion doesn't add anything +either. + +Remove the useless code. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit d30b8e64b7b282da785307504ada59efa8096fb1) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 36 +++--------------------------------- + 1 file changed, 3 insertions(+), 33 deletions(-) + +diff --git a/block/io.c b/block/io.c +index e260394..a0e3461 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -218,38 +218,6 @@ static bool bdrv_drain_poll_top_level(BlockDriverState *bs, + return bdrv_drain_poll(bs, ignore_parent); + } + +-static bool bdrv_drain_recurse(BlockDriverState *bs, BdrvChild *parent) +-{ +- BdrvChild *child, *tmp; +- bool waited; +- +- /* Wait for drained requests to finish */ +- waited = BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, parent)); +- +- QLIST_FOREACH_SAFE(child, &bs->children, next, tmp) { +- BlockDriverState *bs = child->bs; +- bool in_main_loop = +- qemu_get_current_aio_context() == qemu_get_aio_context(); +- assert(bs->refcnt > 0); +- if (in_main_loop) { +- /* In case the recursive bdrv_drain_recurse processes a +- * block_job_defer_to_main_loop BH and modifies the graph, +- * let's hold a reference to bs until we are done. +- * +- * IOThread doesn't have such a BH, and it is not safe to call +- * bdrv_unref without BQL, so skip doing it there. +- */ +- bdrv_ref(bs); +- } +- waited |= bdrv_drain_recurse(bs, child); +- if (in_main_loop) { +- bdrv_unref(bs); +- } +- } +- +- return waited; +-} +- + static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + BdrvChild *parent); + static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, +@@ -317,7 +285,9 @@ void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + + bdrv_parent_drained_begin(bs, parent); + bdrv_drain_invoke(bs, true); +- bdrv_drain_recurse(bs, parent); ++ ++ /* Wait for drained requests to finish */ ++ BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, parent)); + + if (recursive) { + bs->recursive_quiesce_counter++; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Remove-recursive-parameter-from-bdrv_drain_inv.patch b/SOURCES/kvm-block-Remove-recursive-parameter-from-bdrv_drain_inv.patch new file mode 100644 index 0000000..ce277e4 --- /dev/null +++ b/SOURCES/kvm-block-Remove-recursive-parameter-from-bdrv_drain_inv.patch @@ -0,0 +1,75 @@ +From 0e2a3eb5bd393ad4f22c5e492a7f81562eacb7d4 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:37 +0100 +Subject: [PATCH 06/49] block: Remove 'recursive' parameter from + bdrv_drain_invoke() + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-4-kwolf@redhat.com> +Patchwork-id: 82583 +O-Subject: [RHEL-8 qemu-kvm PATCH 03/44] block: Remove 'recursive' parameter from bdrv_drain_invoke() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +All callers pass false for the 'recursive' parameter now. Remove it. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 7d40d9ef9dfb4948a857bfc6ec8408eed1d1d9e7) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 13 +++---------- + 1 file changed, 3 insertions(+), 10 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 230b551..aa41f1e 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -167,9 +167,8 @@ static void coroutine_fn bdrv_drain_invoke_entry(void *opaque) + } + + /* Recursively call BlockDriver.bdrv_co_drain_begin/end callbacks */ +-static void bdrv_drain_invoke(BlockDriverState *bs, bool begin, bool recursive) ++static void bdrv_drain_invoke(BlockDriverState *bs, bool begin) + { +- BdrvChild *child, *tmp; + BdrvCoDrainData data = { .bs = bs, .done = false, .begin = begin}; + + if (!bs->drv || (begin && !bs->drv->bdrv_co_drain_begin) || +@@ -180,12 +179,6 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin, bool recursive) + data.co = qemu_coroutine_create(bdrv_drain_invoke_entry, &data); + bdrv_coroutine_enter(bs, data.co); + BDRV_POLL_WHILE(bs, !data.done); +- +- if (recursive) { +- QLIST_FOREACH_SAFE(child, &bs->children, next, tmp) { +- bdrv_drain_invoke(child->bs, begin, true); +- } +- } + } + + static bool bdrv_drain_recurse(BlockDriverState *bs) +@@ -286,7 +279,7 @@ void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + } + + bdrv_parent_drained_begin(bs, parent); +- bdrv_drain_invoke(bs, true, false); ++ bdrv_drain_invoke(bs, true); + bdrv_drain_recurse(bs); + + if (recursive) { +@@ -321,7 +314,7 @@ void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, + old_quiesce_counter = atomic_fetch_dec(&bs->quiesce_counter); + + /* Re-enable things in child-to-parent order */ +- bdrv_drain_invoke(bs, false, false); ++ bdrv_drain_invoke(bs, false); + bdrv_parent_drained_end(bs, parent); + if (old_quiesce_counter == 1) { + aio_enable_external(bdrv_get_aio_context(bs)); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Require-auto-read-only-for-existing-fallbacks.patch b/SOURCES/kvm-block-Require-auto-read-only-for-existing-fallbacks.patch new file mode 100644 index 0000000..ad6800b --- /dev/null +++ b/SOURCES/kvm-block-Require-auto-read-only-for-existing-fallbacks.patch @@ -0,0 +1,266 @@ +From 60065cb41272144762ceb38e9bcd1a203c29c064 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:34 +0000 +Subject: [PATCH 04/14] block: Require auto-read-only for existing fallbacks + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-5-kwolf@redhat.com> +Patchwork-id: 83954 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 04/12] block: Require auto-read-only for existing fallbacks +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +Some block drivers have traditionally changed their node to read-only +mode without asking the user. This behaviour has been marked deprecated +since 2.11, expecting users to provide an explicit read-only=on option. + +Now that we have auto-read-only=on, enable these drivers to make use of +the option. + +This is the only use of bdrv_set_read_only(), so we can make it a bit +more specific and turn it into a bdrv_apply_auto_read_only() that is +more convenient for drivers to use. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit eaa2410f1ea864609090c0a5fda9e0ce9499da79) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 42 +++++++++++++++++++++++++++--------------- + block/bochs.c | 17 ++++++----------- + block/cloop.c | 16 +++++----------- + block/dmg.c | 16 +++++----------- + block/rbd.c | 15 ++++----------- + block/vvfat.c | 10 ++-------- + include/block/block.h | 3 ++- + 7 files changed, 51 insertions(+), 68 deletions(-) + +diff --git a/block.c b/block.c +index f357975..268debe 100644 +--- a/block.c ++++ b/block.c +@@ -266,29 +266,41 @@ int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only, + return 0; + } + +-/* TODO Remove (deprecated since 2.11) +- * Block drivers are not supposed to automatically change bs->read_only. +- * Instead, they should just check whether they can provide what the user +- * explicitly requested and error out if read-write is requested, but they can +- * only provide read-only access. */ +-int bdrv_set_read_only(BlockDriverState *bs, bool read_only, Error **errp) ++/* ++ * Called by a driver that can only provide a read-only image. ++ * ++ * Returns 0 if the node is already read-only or it could switch the node to ++ * read-only because BDRV_O_AUTO_RDONLY is set. ++ * ++ * Returns -EACCES if the node is read-write and BDRV_O_AUTO_RDONLY is not set ++ * or bdrv_can_set_read_only() forbids making the node read-only. If @errmsg ++ * is not NULL, it is used as the error message for the Error object. ++ */ ++int bdrv_apply_auto_read_only(BlockDriverState *bs, const char *errmsg, ++ Error **errp) + { + int ret = 0; + +- ret = bdrv_can_set_read_only(bs, read_only, false, errp); +- if (ret < 0) { +- return ret; ++ if (!(bs->open_flags & BDRV_O_RDWR)) { ++ return 0; ++ } ++ if (!(bs->open_flags & BDRV_O_AUTO_RDONLY)) { ++ goto fail; + } + +- bs->read_only = read_only; +- +- if (read_only) { +- bs->open_flags &= ~BDRV_O_RDWR; +- } else { +- bs->open_flags |= BDRV_O_RDWR; ++ ret = bdrv_can_set_read_only(bs, true, false, NULL); ++ if (ret < 0) { ++ goto fail; + } + ++ bs->read_only = true; ++ bs->open_flags &= ~BDRV_O_RDWR; ++ + return 0; ++ ++fail: ++ error_setg(errp, "%s", errmsg ?: "Image is read-only"); ++ return -EACCES; + } + + void bdrv_get_full_backing_filename_from_filename(const char *backed, +diff --git a/block/bochs.c b/block/bochs.c +index 50c6300..22e7d44 100644 +--- a/block/bochs.c ++++ b/block/bochs.c +@@ -105,23 +105,18 @@ static int bochs_open(BlockDriverState *bs, QDict *options, int flags, + struct bochs_header bochs; + int ret; + ++ /* No write support yet */ ++ ret = bdrv_apply_auto_read_only(bs, NULL, errp); ++ if (ret < 0) { ++ return ret; ++ } ++ + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, + false, errp); + if (!bs->file) { + return -EINVAL; + } + +- if (!bdrv_is_read_only(bs)) { +- error_report("Opening bochs images without an explicit read-only=on " +- "option is deprecated. Future versions will refuse to " +- "open the image instead of automatically marking the " +- "image read-only."); +- ret = bdrv_set_read_only(bs, true, errp); /* no write support yet */ +- if (ret < 0) { +- return ret; +- } +- } +- + ret = bdrv_pread(bs->file, 0, &bochs, sizeof(bochs)); + if (ret < 0) { + return ret; +diff --git a/block/cloop.c b/block/cloop.c +index 2be6898..df2b85f 100644 +--- a/block/cloop.c ++++ b/block/cloop.c +@@ -67,23 +67,17 @@ static int cloop_open(BlockDriverState *bs, QDict *options, int flags, + uint32_t offsets_size, max_compressed_block_size = 1, i; + int ret; + ++ ret = bdrv_apply_auto_read_only(bs, NULL, errp); ++ if (ret < 0) { ++ return ret; ++ } ++ + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, + false, errp); + if (!bs->file) { + return -EINVAL; + } + +- if (!bdrv_is_read_only(bs)) { +- error_report("Opening cloop images without an explicit read-only=on " +- "option is deprecated. Future versions will refuse to " +- "open the image instead of automatically marking the " +- "image read-only."); +- ret = bdrv_set_read_only(bs, true, errp); +- if (ret < 0) { +- return ret; +- } +- } +- + /* read header */ + ret = bdrv_pread(bs->file, 128, &s->block_size, 4); + if (ret < 0) { +diff --git a/block/dmg.c b/block/dmg.c +index c9b3c51..1d9283b 100644 +--- a/block/dmg.c ++++ b/block/dmg.c +@@ -413,23 +413,17 @@ static int dmg_open(BlockDriverState *bs, QDict *options, int flags, + int64_t offset; + int ret; + ++ ret = bdrv_apply_auto_read_only(bs, NULL, errp); ++ if (ret < 0) { ++ return ret; ++ } ++ + bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, + false, errp); + if (!bs->file) { + return -EINVAL; + } + +- if (!bdrv_is_read_only(bs)) { +- error_report("Opening dmg images without an explicit read-only=on " +- "option is deprecated. Future versions will refuse to " +- "open the image instead of automatically marking the " +- "image read-only."); +- ret = bdrv_set_read_only(bs, true, errp); +- if (ret < 0) { +- return ret; +- } +- } +- + block_module_load_one("dmg-bz2"); + + s->n_chunks = 0; +diff --git a/block/rbd.c b/block/rbd.c +index dc369d0..8d74a29 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -773,17 +773,10 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + /* If we are using an rbd snapshot, we must be r/o, otherwise + * leave as-is */ + if (s->snap != NULL) { +- if (!bdrv_is_read_only(bs)) { +- error_report("Opening rbd snapshots without an explicit " +- "read-only=on option is deprecated. Future versions " +- "will refuse to open the image instead of " +- "automatically marking the image read-only."); +- r = bdrv_set_read_only(bs, true, &local_err); +- if (r < 0) { +- rbd_close(s->image); +- error_propagate(errp, local_err); +- goto failed_open; +- } ++ r = bdrv_apply_auto_read_only(bs, "rbd snapshots are read-only", errp); ++ if (r < 0) { ++ rbd_close(s->image); ++ goto failed_open; + } + } + +diff --git a/block/vvfat.c b/block/vvfat.c +index 3efce9e..a5a3fb9 100644 +--- a/block/vvfat.c ++++ b/block/vvfat.c +@@ -1262,15 +1262,9 @@ static int vvfat_open(BlockDriverState *bs, QDict *options, int flags, + "Unable to set VVFAT to 'rw' when drive is read-only"); + goto fail; + } +- } else if (!bdrv_is_read_only(bs)) { +- error_report("Opening non-rw vvfat images without an explicit " +- "read-only=on option is deprecated. Future versions " +- "will refuse to open the image instead of " +- "automatically marking the image read-only."); +- /* read only is the default for safety */ +- ret = bdrv_set_read_only(bs, true, &local_err); ++ } else { ++ ret = bdrv_apply_auto_read_only(bs, NULL, errp); + if (ret < 0) { +- error_propagate(errp, local_err); + goto fail; + } + } +diff --git a/include/block/block.h b/include/block/block.h +index 6ee8b2a..36a702c 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -433,7 +433,8 @@ int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base, + bool bdrv_is_read_only(BlockDriverState *bs); + int bdrv_can_set_read_only(BlockDriverState *bs, bool read_only, + bool ignore_allow_rdw, Error **errp); +-int bdrv_set_read_only(BlockDriverState *bs, bool read_only, Error **errp); ++int bdrv_apply_auto_read_only(BlockDriverState *bs, const char *errmsg, ++ Error **errp); + bool bdrv_is_writable(BlockDriverState *bs); + bool bdrv_is_sg(BlockDriverState *bs); + bool bdrv_is_inserted(BlockDriverState *bs); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Set-BDRV_REQ_WRITE_UNCHANGED-for-COR-writes.patch b/SOURCES/kvm-block-Set-BDRV_REQ_WRITE_UNCHANGED-for-COR-writes.patch new file mode 100644 index 0000000..646d030 --- /dev/null +++ b/SOURCES/kvm-block-Set-BDRV_REQ_WRITE_UNCHANGED-for-COR-writes.patch @@ -0,0 +1,52 @@ +From 4c433d800c5c5c31e72f69d8192b44dc3382ba02 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:07 +0200 +Subject: [PATCH 040/268] block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-6-mreitz@redhat.com> +Patchwork-id: 80766 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 05/10] block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Signed-off-by: Max Reitz +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Alberto Garcia +Message-id: 20180421132929.21610-5-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 7adcf59fecf3c8ce9330430187350b53f9e50cf7) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 134b2a4..fada4ef 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -1115,13 +1115,15 @@ static int coroutine_fn bdrv_co_do_copy_on_readv(BdrvChild *child, + /* FIXME: Should we (perhaps conditionally) be setting + * BDRV_REQ_MAY_UNMAP, if it will allow for a sparser copy + * that still correctly reads as zero? */ +- ret = bdrv_co_do_pwrite_zeroes(bs, cluster_offset, pnum, 0); ++ ret = bdrv_co_do_pwrite_zeroes(bs, cluster_offset, pnum, ++ BDRV_REQ_WRITE_UNCHANGED); + } else { + /* This does not change the data on the disk, it is not + * necessary to flush even in cache=writethrough mode. + */ + ret = bdrv_driver_pwritev(bs, cluster_offset, pnum, +- &local_qiov, 0); ++ &local_qiov, ++ BDRV_REQ_WRITE_UNCHANGED); + } + + if (ret < 0) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Support-BDRV_REQ_WRITE_UNCHANGED-in-filters.patch b/SOURCES/kvm-block-Support-BDRV_REQ_WRITE_UNCHANGED-in-filters.patch new file mode 100644 index 0000000..9ff4d00 --- /dev/null +++ b/SOURCES/kvm-block-Support-BDRV_REQ_WRITE_UNCHANGED-in-filters.patch @@ -0,0 +1,173 @@ +From 9cecd3548e9275e7de21801b90c277c7b04cabb4 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:09 +0200 +Subject: [PATCH 042/268] block: Support BDRV_REQ_WRITE_UNCHANGED in filters + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-8-mreitz@redhat.com> +Patchwork-id: 80767 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 07/10] block: Support BDRV_REQ_WRITE_UNCHANGED in filters +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Update the rest of the filter drivers to support +BDRV_REQ_WRITE_UNCHANGED. They already forward write request flags to +their children, so we just have to announce support for it. + +This patch does not cover the replication driver because that currently +does not support flags at all, and because it just grabs the WRITE +permission for its children when it can, so we should be fine just +submitting the incoming WRITE_UNCHANGED requests as normal writes. + +It also does not cover format drivers for similar reasons. They all use +bdrv_format_default_perms() as their .bdrv_child_perm() implementation +so they just always grab the WRITE permission for their file children +whenever possible. In addition, it often would be difficult to +ascertain whether incoming unchanging writes end up as unchanging writes +in their files. So we just leave them as normal potentially changing +writes. + +Signed-off-by: Max Reitz +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Alberto Garcia +Message-id: 20180421132929.21610-7-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 228345bf5db8bc97d1c64f062e138d389065d1ab) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/blkdebug.c | 9 +++++---- + block/blkreplay.c | 3 +++ + block/blkverify.c | 3 +++ + block/copy-on-read.c | 10 ++++++---- + block/mirror.c | 2 ++ + block/raw-format.c | 9 +++++---- + block/throttle.c | 6 ++++-- + 7 files changed, 28 insertions(+), 14 deletions(-) + +diff --git a/block/blkdebug.c b/block/blkdebug.c +index 053372c..526af2a 100644 +--- a/block/blkdebug.c ++++ b/block/blkdebug.c +@@ -398,10 +398,11 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags, + goto out; + } + +- bs->supported_write_flags = BDRV_REQ_FUA & +- bs->file->bs->supported_write_flags; +- bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & +- bs->file->bs->supported_zero_flags; ++ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | ++ (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); ++ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | ++ ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & ++ bs->file->bs->supported_zero_flags); + ret = -EINVAL; + + /* Set alignment overrides */ +diff --git a/block/blkreplay.c b/block/blkreplay.c +index fe5a9b4..b016dbe 100755 +--- a/block/blkreplay.c ++++ b/block/blkreplay.c +@@ -35,6 +35,9 @@ static int blkreplay_open(BlockDriverState *bs, QDict *options, int flags, + goto fail; + } + ++ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED; ++ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED; ++ + ret = 0; + fail: + return ret; +diff --git a/block/blkverify.c b/block/blkverify.c +index 754cc9e..da97ee5 100644 +--- a/block/blkverify.c ++++ b/block/blkverify.c +@@ -141,6 +141,9 @@ static int blkverify_open(BlockDriverState *bs, QDict *options, int flags, + goto fail; + } + ++ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED; ++ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED; ++ + ret = 0; + fail: + qemu_opts_del(opts); +diff --git a/block/copy-on-read.c b/block/copy-on-read.c +index 823ec75..6a97208 100644 +--- a/block/copy-on-read.c ++++ b/block/copy-on-read.c +@@ -33,11 +33,13 @@ static int cor_open(BlockDriverState *bs, QDict *options, int flags, + return -EINVAL; + } + +- bs->supported_write_flags = BDRV_REQ_FUA & +- bs->file->bs->supported_write_flags; ++ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | ++ (BDRV_REQ_FUA & ++ bs->file->bs->supported_write_flags); + +- bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & +- bs->file->bs->supported_zero_flags; ++ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | ++ ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & ++ bs->file->bs->supported_zero_flags); + + return 0; + } +diff --git a/block/mirror.c b/block/mirror.c +index 99da9c0..003f957 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -1152,6 +1152,8 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs, + mirror_top_bs->implicit = true; + } + mirror_top_bs->total_sectors = bs->total_sectors; ++ mirror_top_bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED; ++ mirror_top_bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED; + bdrv_set_aio_context(mirror_top_bs, bdrv_get_aio_context(bs)); + + /* bdrv_append takes ownership of the mirror_top_bs reference, need to keep +diff --git a/block/raw-format.c b/block/raw-format.c +index a378547..fe33693 100644 +--- a/block/raw-format.c ++++ b/block/raw-format.c +@@ -415,10 +415,11 @@ static int raw_open(BlockDriverState *bs, QDict *options, int flags, + } + + bs->sg = bs->file->bs->sg; +- bs->supported_write_flags = BDRV_REQ_FUA & +- bs->file->bs->supported_write_flags; +- bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & +- bs->file->bs->supported_zero_flags; ++ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED | ++ (BDRV_REQ_FUA & bs->file->bs->supported_write_flags); ++ bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED | ++ ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) & ++ bs->file->bs->supported_zero_flags); + + if (bs->probed && !bdrv_is_read_only(bs)) { + fprintf(stderr, +diff --git a/block/throttle.c b/block/throttle.c +index 95ed06a..e298827 100644 +--- a/block/throttle.c ++++ b/block/throttle.c +@@ -81,8 +81,10 @@ static int throttle_open(BlockDriverState *bs, QDict *options, + if (!bs->file) { + return -EINVAL; + } +- bs->supported_write_flags = bs->file->bs->supported_write_flags; +- bs->supported_zero_flags = bs->file->bs->supported_zero_flags; ++ bs->supported_write_flags = bs->file->bs->supported_write_flags | ++ BDRV_REQ_WRITE_UNCHANGED; ++ bs->supported_zero_flags = bs->file->bs->supported_zero_flags | ++ BDRV_REQ_WRITE_UNCHANGED; + + return throttle_configure_tgm(bs, tgm, options, errp); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Update-flags-in-bdrv_set_read_only.patch b/SOURCES/kvm-block-Update-flags-in-bdrv_set_read_only.patch new file mode 100644 index 0000000..501476f --- /dev/null +++ b/SOURCES/kvm-block-Update-flags-in-bdrv_set_read_only.patch @@ -0,0 +1,48 @@ +From 03d1ed6853d936e90ed1661433f822b4a360e5fa Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:31 +0000 +Subject: [PATCH 01/14] block: Update flags in bdrv_set_read_only() + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-2-kwolf@redhat.com> +Patchwork-id: 83949 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 01/12] block: Update flags in bdrv_set_read_only() +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +To fully change the read-only state of a node, we must not only change +bs->read_only, but also update bs->open_flags. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Alberto Garcia +(cherry picked from commit eeae6a596b0efc092f5101c67683053e245e6250) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/block.c b/block.c +index 73f55a1..6f1d53b 100644 +--- a/block.c ++++ b/block.c +@@ -281,6 +281,13 @@ int bdrv_set_read_only(BlockDriverState *bs, bool read_only, Error **errp) + } + + bs->read_only = read_only; ++ ++ if (read_only) { ++ bs->open_flags &= ~BDRV_O_RDWR; ++ } else { ++ bs->open_flags |= BDRV_O_RDWR; ++ } ++ + return 0; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Use-a-single-global-AioWait.patch b/SOURCES/kvm-block-Use-a-single-global-AioWait.patch new file mode 100644 index 0000000..cb91eb7 --- /dev/null +++ b/SOURCES/kvm-block-Use-a-single-global-AioWait.patch @@ -0,0 +1,367 @@ +From 6acc1d617c7db4c575dc56c6035ea00315dffa20 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:12 +0100 +Subject: [PATCH 46/49] block: Use a single global AioWait + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-34-kwolf@redhat.com> +Patchwork-id: 82623 +O-Subject: [RHEL-8 qemu-kvm PATCH 43/44] block: Use a single global AioWait +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +When draining a block node, we recurse to its parent and for subtree +drains also to its children. A single AIO_WAIT_WHILE() is then used to +wait for bdrv_drain_poll() to become true, which depends on all of the +nodes we recursed to. However, if the respective child or parent becomes +quiescent and calls bdrv_wakeup(), only the AioWait of the child/parent +is checked, while AIO_WAIT_WHILE() depends on the AioWait of the +original node. + +Fix this by using a single AioWait for all callers of AIO_WAIT_WHILE(). + +This may mean that the draining thread gets a few more unnecessary +wakeups because an unrelated operation got completed, but we already +wake it up when something _could_ have changed rather than only if it +has certainly changed. + +Apart from that, drain is a slow path anyway. In theory it would be +possible to use wakeups more selectively and still correctly, but the +gains are likely not worth the additional complexity. In fact, this +patch is a nice simplification for some places in the code. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +(cherry picked from commit cfe29d8294e06420e15d4938421ae006c8ac49e7) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 5 ----- + block/block-backend.c | 11 ++++------- + block/io.c | 7 ++----- + blockjob.c | 13 +------------ + include/block/aio-wait.h | 22 +++++++++++----------- + include/block/block.h | 6 +----- + include/block/block_int.h | 3 --- + include/block/blockjob.h | 10 ---------- + job.c | 3 +-- + util/aio-wait.c | 11 ++++++----- + 10 files changed, 26 insertions(+), 65 deletions(-) + +diff --git a/block.c b/block.c +index e89b5e3..fbd569c 100644 +--- a/block.c ++++ b/block.c +@@ -4847,11 +4847,6 @@ AioContext *bdrv_get_aio_context(BlockDriverState *bs) + return bs ? bs->aio_context : qemu_get_aio_context(); + } + +-AioWait *bdrv_get_aio_wait(BlockDriverState *bs) +-{ +- return bs ? &bs->wait : NULL; +-} +- + void bdrv_coroutine_enter(BlockDriverState *bs, Coroutine *co) + { + aio_co_enter(bdrv_get_aio_context(bs), co); +diff --git a/block/block-backend.c b/block/block-backend.c +index b8ea286..91abfe6 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -88,7 +88,6 @@ struct BlockBackend { + * Accessed with atomic ops. + */ + unsigned int in_flight; +- AioWait wait; + }; + + typedef struct BlockBackendAIOCB { +@@ -1300,7 +1299,7 @@ static void blk_inc_in_flight(BlockBackend *blk) + static void blk_dec_in_flight(BlockBackend *blk) + { + atomic_dec(&blk->in_flight); +- aio_wait_kick(&blk->wait); ++ aio_wait_kick(); + } + + static void error_callback_bh(void *opaque) +@@ -1601,9 +1600,8 @@ void blk_drain(BlockBackend *blk) + } + + /* We may have -ENOMEDIUM completions in flight */ +- AIO_WAIT_WHILE(&blk->wait, +- blk_get_aio_context(blk), +- atomic_mb_read(&blk->in_flight) > 0); ++ AIO_WAIT_WHILE(blk_get_aio_context(blk), ++ atomic_mb_read(&blk->in_flight) > 0); + + if (bs) { + bdrv_drained_end(bs); +@@ -1622,8 +1620,7 @@ void blk_drain_all(void) + aio_context_acquire(ctx); + + /* We may have -ENOMEDIUM completions in flight */ +- AIO_WAIT_WHILE(&blk->wait, ctx, +- atomic_mb_read(&blk->in_flight) > 0); ++ AIO_WAIT_WHILE(ctx, atomic_mb_read(&blk->in_flight) > 0); + + aio_context_release(ctx); + } +diff --git a/block/io.c b/block/io.c +index 3313958..7a99f7b 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -38,8 +38,6 @@ + /* Maximum bounce buffer for copy-on-read and write zeroes, in bytes */ + #define MAX_BOUNCE_BUFFER (32768 << BDRV_SECTOR_BITS) + +-static AioWait drain_all_aio_wait; +- + static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int bytes, BdrvRequestFlags flags); + +@@ -555,7 +553,7 @@ void bdrv_drain_all_begin(void) + } + + /* Now poll the in-flight requests */ +- AIO_WAIT_WHILE(&drain_all_aio_wait, NULL, bdrv_drain_all_poll()); ++ AIO_WAIT_WHILE(NULL, bdrv_drain_all_poll()); + + while ((bs = bdrv_next_all_states(bs))) { + bdrv_drain_assert_idle(bs); +@@ -709,8 +707,7 @@ void bdrv_inc_in_flight(BlockDriverState *bs) + + void bdrv_wakeup(BlockDriverState *bs) + { +- aio_wait_kick(bdrv_get_aio_wait(bs)); +- aio_wait_kick(&drain_all_aio_wait); ++ aio_wait_kick(); + } + + void bdrv_dec_in_flight(BlockDriverState *bs) +diff --git a/blockjob.c b/blockjob.c +index 617d86f..06f2429 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -221,20 +221,9 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, + return 0; + } + +-void block_job_wakeup_all_bdrv(BlockJob *job) +-{ +- GSList *l; +- +- for (l = job->nodes; l; l = l->next) { +- BdrvChild *c = l->data; +- bdrv_wakeup(c->bs); +- } +-} +- + static void block_job_on_idle(Notifier *n, void *opaque) + { +- BlockJob *job = opaque; +- block_job_wakeup_all_bdrv(job); ++ aio_wait_kick(); + } + + bool block_job_is_internal(BlockJob *job) +diff --git a/include/block/aio-wait.h b/include/block/aio-wait.h +index 600fad1..afd0ff7 100644 +--- a/include/block/aio-wait.h ++++ b/include/block/aio-wait.h +@@ -30,14 +30,15 @@ + /** + * AioWait: + * +- * An object that facilitates synchronous waiting on a condition. The main +- * loop can wait on an operation running in an IOThread as follows: ++ * An object that facilitates synchronous waiting on a condition. A single ++ * global AioWait object (global_aio_wait) is used internally. ++ * ++ * The main loop can wait on an operation running in an IOThread as follows: + * +- * AioWait *wait = ...; + * AioContext *ctx = ...; + * MyWork work = { .done = false }; + * schedule_my_work_in_iothread(ctx, &work); +- * AIO_WAIT_WHILE(wait, ctx, !work.done); ++ * AIO_WAIT_WHILE(ctx, !work.done); + * + * The IOThread must call aio_wait_kick() to notify the main loop when + * work.done changes: +@@ -46,7 +47,7 @@ + * { + * ... + * work.done = true; +- * aio_wait_kick(wait); ++ * aio_wait_kick(); + * } + */ + typedef struct { +@@ -54,9 +55,10 @@ typedef struct { + unsigned num_waiters; + } AioWait; + ++extern AioWait global_aio_wait; ++ + /** + * AIO_WAIT_WHILE: +- * @wait: the aio wait object + * @ctx: the aio context, or NULL if multiple aio contexts (for which the + * caller does not hold a lock) are involved in the polling condition. + * @cond: wait while this conditional expression is true +@@ -72,9 +74,9 @@ typedef struct { + * wait on conditions between two IOThreads since that could lead to deadlock, + * go via the main loop instead. + */ +-#define AIO_WAIT_WHILE(wait, ctx, cond) ({ \ ++#define AIO_WAIT_WHILE(ctx, cond) ({ \ + bool waited_ = false; \ +- AioWait *wait_ = (wait); \ ++ AioWait *wait_ = &global_aio_wait; \ + AioContext *ctx_ = (ctx); \ + /* Increment wait_->num_waiters before evaluating cond. */ \ + atomic_inc(&wait_->num_waiters); \ +@@ -102,14 +104,12 @@ typedef struct { + + /** + * aio_wait_kick: +- * @wait: the aio wait object that should re-evaluate its condition +- * + * Wake up the main thread if it is waiting on AIO_WAIT_WHILE(). During + * synchronous operations performed in an IOThread, the main thread lets the + * IOThread's event loop run, waiting for the operation to complete. A + * aio_wait_kick() call will wake up the main thread. + */ +-void aio_wait_kick(AioWait *wait); ++void aio_wait_kick(void); + + /** + * aio_wait_bh_oneshot: +diff --git a/include/block/block.h b/include/block/block.h +index 356712c..8e78daf 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -406,13 +406,9 @@ void bdrv_drain_all_begin(void); + void bdrv_drain_all_end(void); + void bdrv_drain_all(void); + +-/* Returns NULL when bs == NULL */ +-AioWait *bdrv_get_aio_wait(BlockDriverState *bs); +- + #define BDRV_POLL_WHILE(bs, cond) ({ \ + BlockDriverState *bs_ = (bs); \ +- AIO_WAIT_WHILE(bdrv_get_aio_wait(bs_), \ +- bdrv_get_aio_context(bs_), \ ++ AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ + cond); }) + + int bdrv_pdiscard(BlockDriverState *bs, int64_t offset, int bytes); +diff --git a/include/block/block_int.h b/include/block/block_int.h +index b7806e3..ff923b7 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -782,9 +782,6 @@ struct BlockDriverState { + unsigned int in_flight; + unsigned int serialising_in_flight; + +- /* Kicked to signal main loop when a request completes. */ +- AioWait wait; +- + /* counter for nested bdrv_io_plug. + * Accessed with atomic ops. + */ +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 2290bbb..ede0bd8 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -122,16 +122,6 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, + void block_job_remove_all_bdrv(BlockJob *job); + + /** +- * block_job_wakeup_all_bdrv: +- * @job: The block job +- * +- * Calls bdrv_wakeup() for all BlockDriverStates that have been added to the +- * job. This function is to be called whenever child_job_drained_poll() would +- * go from true to false to notify waiting drain requests. +- */ +-void block_job_wakeup_all_bdrv(BlockJob *job); +- +-/** + * block_job_set_speed: + * @job: The job to set the speed for. + * @speed: The new value +diff --git a/job.c b/job.c +index 5b53e43..3a7db59 100644 +--- a/job.c ++++ b/job.c +@@ -973,7 +973,6 @@ void job_complete(Job *job, Error **errp) + int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + { + Error *local_err = NULL; +- AioWait dummy_wait = {}; + int ret; + + job_ref(job); +@@ -987,7 +986,7 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + return -EBUSY; + } + +- AIO_WAIT_WHILE(&dummy_wait, job->aio_context, ++ AIO_WAIT_WHILE(job->aio_context, + (job_drain(job), !job_is_completed(job))); + + ret = (job_is_cancelled(job) && job->ret == 0) ? -ECANCELED : job->ret; +diff --git a/util/aio-wait.c b/util/aio-wait.c +index b8a8f86..b487749 100644 +--- a/util/aio-wait.c ++++ b/util/aio-wait.c +@@ -26,21 +26,22 @@ + #include "qemu/main-loop.h" + #include "block/aio-wait.h" + ++AioWait global_aio_wait; ++ + static void dummy_bh_cb(void *opaque) + { + /* The point is to make AIO_WAIT_WHILE()'s aio_poll() return */ + } + +-void aio_wait_kick(AioWait *wait) ++void aio_wait_kick(void) + { + /* The barrier (or an atomic op) is in the caller. */ +- if (atomic_read(&wait->num_waiters)) { ++ if (atomic_read(&global_aio_wait.num_waiters)) { + aio_bh_schedule_oneshot(qemu_get_aio_context(), dummy_bh_cb, NULL); + } + } + + typedef struct { +- AioWait wait; + bool done; + QEMUBHFunc *cb; + void *opaque; +@@ -54,7 +55,7 @@ static void aio_wait_bh(void *opaque) + data->cb(data->opaque); + + data->done = true; +- aio_wait_kick(&data->wait); ++ aio_wait_kick(); + } + + void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque) +@@ -67,5 +68,5 @@ void aio_wait_bh_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque) + assert(qemu_get_current_aio_context() == qemu_get_aio_context()); + + aio_bh_schedule_oneshot(ctx, aio_wait_bh, &data); +- AIO_WAIT_WHILE(&data.wait, ctx, !data.done); ++ AIO_WAIT_WHILE(ctx, !data.done); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Use-bdrv_do_drain_begin-end-in-bdrv_drain_all.patch b/SOURCES/kvm-block-Use-bdrv_do_drain_begin-end-in-bdrv_drain_all.patch new file mode 100644 index 0000000..bad010e --- /dev/null +++ b/SOURCES/kvm-block-Use-bdrv_do_drain_begin-end-in-bdrv_drain_all.patch @@ -0,0 +1,113 @@ +From c05f0a720ff742af5c010f5ab2c0661a10cfa536 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:36 +0100 +Subject: [PATCH 05/49] block: Use bdrv_do_drain_begin/end in bdrv_drain_all() + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-3-kwolf@redhat.com> +Patchwork-id: 82585 +O-Subject: [RHEL-8 qemu-kvm PATCH 02/44] block: Use bdrv_do_drain_begin/end in bdrv_drain_all() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +bdrv_do_drain_begin/end() implement already everything that +bdrv_drain_all_begin/end() need and currently still do manually: Disable +external events, call parent drain callbacks, call block driver +callbacks. + +It also does two more things: + +The first is incrementing bs->quiesce_counter. bdrv_drain_all() already +stood out in the test case by behaving different from the other drain +variants. Adding this is not only safe, but in fact a bug fix. + +The second is calling bdrv_drain_recurse(). We already do that later in +the same function in a loop, so basically doing an early first iteration +doesn't hurt. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 79ab8b21dc19c08adc407504e456ff64b9dacb66) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 10 ++-------- + tests/test-bdrv-drain.c | 14 ++++---------- + 2 files changed, 6 insertions(+), 18 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 7e0a169..230b551 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -412,11 +412,8 @@ void bdrv_drain_all_begin(void) + for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { + AioContext *aio_context = bdrv_get_aio_context(bs); + +- /* Stop things in parent-to-child order */ + aio_context_acquire(aio_context); +- aio_disable_external(aio_context); +- bdrv_parent_drained_begin(bs, NULL); +- bdrv_drain_invoke(bs, true, true); ++ bdrv_do_drained_begin(bs, true, NULL); + aio_context_release(aio_context); + + if (!g_slist_find(aio_ctxs, aio_context)) { +@@ -457,11 +454,8 @@ void bdrv_drain_all_end(void) + for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) { + AioContext *aio_context = bdrv_get_aio_context(bs); + +- /* Re-enable things in child-to-parent order */ + aio_context_acquire(aio_context); +- bdrv_drain_invoke(bs, false, true); +- bdrv_parent_drained_end(bs, NULL); +- aio_enable_external(aio_context); ++ bdrv_do_drained_end(bs, true, NULL); + aio_context_release(aio_context); + } + } +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index dee0a10..f1276a1 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -276,8 +276,7 @@ static void test_quiesce_common(enum drain_type drain_type, bool recursive) + + static void test_quiesce_drain_all(void) + { +- // XXX drain_all doesn't quiesce +- //test_quiesce_common(BDRV_DRAIN_ALL, true); ++ test_quiesce_common(BDRV_DRAIN_ALL, true); + } + + static void test_quiesce_drain(void) +@@ -319,12 +318,7 @@ static void test_nested(void) + + for (outer = 0; outer < DRAIN_TYPE_MAX; outer++) { + for (inner = 0; inner < DRAIN_TYPE_MAX; inner++) { +- /* XXX bdrv_drain_all() doesn't increase the quiesce_counter */ +- int bs_quiesce = (outer != BDRV_DRAIN_ALL) + +- (inner != BDRV_DRAIN_ALL); +- int backing_quiesce = (outer == BDRV_SUBTREE_DRAIN) + +- (inner == BDRV_SUBTREE_DRAIN); +- int backing_cb_cnt = (outer != BDRV_DRAIN) + ++ int backing_quiesce = (outer != BDRV_DRAIN) + + (inner != BDRV_DRAIN); + + g_assert_cmpint(bs->quiesce_counter, ==, 0); +@@ -335,10 +329,10 @@ static void test_nested(void) + do_drain_begin(outer, bs); + do_drain_begin(inner, bs); + +- g_assert_cmpint(bs->quiesce_counter, ==, bs_quiesce); ++ g_assert_cmpint(bs->quiesce_counter, ==, 2); + g_assert_cmpint(backing->quiesce_counter, ==, backing_quiesce); + g_assert_cmpint(s->drain_count, ==, 2); +- g_assert_cmpint(backing_s->drain_count, ==, backing_cb_cnt); ++ g_assert_cmpint(backing_s->drain_count, ==, backing_quiesce); + + do_drain_end(inner, bs); + do_drain_end(outer, bs); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-Use-tracked-request-for-truncate.patch b/SOURCES/kvm-block-Use-tracked-request-for-truncate.patch new file mode 100644 index 0000000..68a9e3d --- /dev/null +++ b/SOURCES/kvm-block-Use-tracked-request-for-truncate.patch @@ -0,0 +1,104 @@ +From fb4f98330be87d3272d91c354843eadc6db8fc73 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 14:42:57 +0200 +Subject: [PATCH 212/268] block: Use tracked request for truncate + +RH-Author: Kevin Wolf +Message-id: <20180712144258.17303-6-kwolf@redhat.com> +Patchwork-id: 81326 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 5/6] block: Use tracked request for truncate +Bugzilla: 1595173 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: John Snow + +When growing an image, block drivers (especially protocol drivers) may +initialise the newly added area. I/O requests to the same area need to +wait for this initialisation to be completed so that data writes don't +get overwritten and reads don't read uninitialised data. + +To avoid overhead in the fast I/O path by adding new locking in the +protocol drivers and to restrict the impact to requests that actually +touch the new area, reuse the existing tracked request infrastructure in +block/io.c and mark all discard requests as serialising. + +With this change, it is safe for protocol drivers to make +.bdrv_co_truncate actually asynchronous. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 1bc5f09f2e1b2be8f6f737b8d5352b438fc41492) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 25 +++++++++++++++++++++++++ + include/block/block_int.h | 1 + + 2 files changed, 26 insertions(+) + +diff --git a/block/io.c b/block/io.c +index 32a82e3..ad8afc0 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2948,6 +2948,8 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, + { + BlockDriverState *bs = child->bs; + BlockDriver *drv = bs->drv; ++ BdrvTrackedRequest req; ++ int64_t old_size, new_bytes; + int ret; + + assert(child->perm & BLK_PERM_RESIZE); +@@ -2962,7 +2964,28 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, + return -EINVAL; + } + ++ old_size = bdrv_getlength(bs); ++ if (old_size < 0) { ++ error_setg_errno(errp, -old_size, "Failed to get old image size"); ++ return old_size; ++ } ++ ++ if (offset > old_size) { ++ new_bytes = offset - old_size; ++ } else { ++ new_bytes = 0; ++ } ++ + bdrv_inc_in_flight(bs); ++ tracked_request_begin(&req, bs, offset, new_bytes, BDRV_TRACKED_TRUNCATE); ++ ++ /* If we are growing the image and potentially using preallocation for the ++ * new area, we need to make sure that no write requests are made to it ++ * concurrently or they might be overwritten by preallocation. */ ++ if (new_bytes) { ++ mark_request_serialising(&req, 1); ++ wait_serialising_requests(&req); ++ } + + if (!drv->bdrv_co_truncate) { + if (bs->file && drv->is_filter) { +@@ -2996,7 +3019,9 @@ int coroutine_fn bdrv_co_truncate(BdrvChild *child, int64_t offset, + atomic_inc(&bs->write_gen); + + out: ++ tracked_request_end(&req); + bdrv_dec_in_flight(bs); ++ + return ret; + } + +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 6a844ec..27e168f 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -63,6 +63,7 @@ enum BdrvTrackedRequestType { + BDRV_TRACKED_READ, + BDRV_TRACKED_WRITE, + BDRV_TRACKED_DISCARD, ++ BDRV_TRACKED_TRUNCATE, + }; + + typedef struct BdrvTrackedRequest { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-add-BDRV_REQ_SERIALISING-flag.patch b/SOURCES/kvm-block-add-BDRV_REQ_SERIALISING-flag.patch new file mode 100644 index 0000000..be2167b --- /dev/null +++ b/SOURCES/kvm-block-add-BDRV_REQ_SERIALISING-flag.patch @@ -0,0 +1,137 @@ +From 64a5cd71c48e81d7bf156b9987ad22a5058168fe Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:57 +0200 +Subject: [PATCH 239/268] block: add BDRV_REQ_SERIALISING flag + +RH-Author: John Snow +Message-id: <20180718225511.14878-22-jsnow@redhat.com> +Patchwork-id: 81421 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 21/35] block: add BDRV_REQ_SERIALISING flag +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Serialized writes should be used in copy-on-write of backup(sync=none) +for image fleecing scheme. + +We need to change an assert in bdrv_aligned_pwritev, added in +28de2dcd88de. The assert may fail now, because call to +wait_serialising_requests here may become first call to it for this +request with serializing flag set. It occurs if the request is aligned +(otherwise, we should already set serializing flag before calling +bdrv_aligned_pwritev and correspondingly waited for all intersecting +requests). However, for aligned requests, we should not care about +outdating of previously read data, as there no such data. Therefore, +let's just update an assert to not care about aligned requests. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit 09d2f948462f4979d18f573a0734d1daae8e67a9) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/io.c | 28 +++++++++++++++++++++++++++- + include/block/block.h | 14 +++++++++++++- + 2 files changed, 40 insertions(+), 2 deletions(-) + +diff --git a/block/io.c b/block/io.c +index 2d04289..bb617de 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -535,6 +535,18 @@ static void mark_request_serialising(BdrvTrackedRequest *req, uint64_t align) + req->overlap_bytes = MAX(req->overlap_bytes, overlap_bytes); + } + ++static bool is_request_serialising_and_aligned(BdrvTrackedRequest *req) ++{ ++ /* ++ * If the request is serialising, overlap_offset and overlap_bytes are set, ++ * so we can check if the request is aligned. Otherwise, don't care and ++ * return false. ++ */ ++ ++ return req->serialising && (req->offset == req->overlap_offset) && ++ (req->bytes == req->overlap_bytes); ++} ++ + /** + * Round a region to cluster boundaries + */ +@@ -1206,6 +1218,9 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child, + mark_request_serialising(req, bdrv_get_cluster_size(bs)); + } + ++ /* BDRV_REQ_SERIALISING is only for write operation */ ++ assert(!(flags & BDRV_REQ_SERIALISING)); ++ + if (!(flags & BDRV_REQ_NO_SERIALISING)) { + wait_serialising_requests(req); + } +@@ -1507,8 +1522,14 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child, + + /* BDRV_REQ_NO_SERIALISING is only for read operation */ + assert(!(flags & BDRV_REQ_NO_SERIALISING)); ++ ++ if (flags & BDRV_REQ_SERIALISING) { ++ mark_request_serialising(req, bdrv_get_cluster_size(bs)); ++ } ++ + waited = wait_serialising_requests(req); +- assert(!waited || !req->serialising); ++ assert(!waited || !req->serialising || ++ is_request_serialising_and_aligned(req)); + assert(req->overlap_offset <= offset); + assert(offset + bytes <= req->overlap_offset + req->overlap_bytes); + if (flags & BDRV_REQ_WRITE_UNCHANGED) { +@@ -2881,6 +2902,8 @@ static int coroutine_fn bdrv_co_copy_range_internal( + tracked_request_begin(&req, src->bs, src_offset, bytes, + BDRV_TRACKED_READ); + ++ /* BDRV_REQ_SERIALISING is only for write operation */ ++ assert(!(read_flags & BDRV_REQ_SERIALISING)); + if (!(read_flags & BDRV_REQ_NO_SERIALISING)) { + wait_serialising_requests(&req); + } +@@ -2900,6 +2923,9 @@ static int coroutine_fn bdrv_co_copy_range_internal( + + /* BDRV_REQ_NO_SERIALISING is only for read operation */ + assert(!(write_flags & BDRV_REQ_NO_SERIALISING)); ++ if (write_flags & BDRV_REQ_SERIALISING) { ++ mark_request_serialising(&req, bdrv_get_cluster_size(dst->bs)); ++ } + wait_serialising_requests(&req); + + ret = dst->bs->drv->bdrv_co_copy_range_to(dst->bs, +diff --git a/include/block/block.h b/include/block/block.h +index 409db21..8f87eea 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -70,8 +70,20 @@ typedef enum { + * content. */ + BDRV_REQ_WRITE_UNCHANGED = 0x40, + ++ /* ++ * BDRV_REQ_SERIALISING forces request serialisation for writes. ++ * It is used to ensure that writes to the backing file of a backup process ++ * target cannot race with a read of the backup target that defers to the ++ * backing file. ++ * ++ * Note, that BDRV_REQ_SERIALISING is _not_ opposite in meaning to ++ * BDRV_REQ_NO_SERIALISING. A more descriptive name for the latter might be ++ * _DO_NOT_WAIT_FOR_SERIALISING, except that is too long. ++ */ ++ BDRV_REQ_SERIALISING = 0x80, ++ + /* Mask of valid flags */ +- BDRV_REQ_MASK = 0x7f, ++ BDRV_REQ_MASK = 0xff, + } BdrvRequestFlags; + + typedef struct BlockSizes { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backend-Add-.drained_poll-callback.patch b/SOURCES/kvm-block-backend-Add-.drained_poll-callback.patch new file mode 100644 index 0000000..0b5603b --- /dev/null +++ b/SOURCES/kvm-block-backend-Add-.drained_poll-callback.patch @@ -0,0 +1,66 @@ +From 92a9a816dd93a361ad2e0751e503c5e12accf38b Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:03 +0100 +Subject: [PATCH 37/49] block-backend: Add .drained_poll callback + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-25-kwolf@redhat.com> +Patchwork-id: 82614 +O-Subject: [RHEL-8 qemu-kvm PATCH 34/44] block-backend: Add .drained_poll callback +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +A bdrv_drain operation must ensure that all parents are quiesced, this +includes BlockBackends. Otherwise, callbacks called by requests that are +completed on the BDS layer, but not quite yet on the BlockBackend layer +could still create new requests. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +Reviewed-by: Max Reitz +(cherry picked from commit fe5258a503a87e69be37c9ac48799e293809386e) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/block-backend.c | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/block/block-backend.c b/block/block-backend.c +index 3554b7e..dffc6f0 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -121,6 +121,7 @@ static void blk_root_inherit_options(int *child_flags, QDict *child_options, + abort(); + } + static void blk_root_drained_begin(BdrvChild *child); ++static bool blk_root_drained_poll(BdrvChild *child); + static void blk_root_drained_end(BdrvChild *child); + + static void blk_root_change_media(BdrvChild *child, bool load); +@@ -294,6 +295,7 @@ static const BdrvChildRole child_root = { + .get_parent_desc = blk_root_get_parent_desc, + + .drained_begin = blk_root_drained_begin, ++ .drained_poll = blk_root_drained_poll, + .drained_end = blk_root_drained_end, + + .activate = blk_root_activate, +@@ -2198,6 +2200,13 @@ static void blk_root_drained_begin(BdrvChild *child) + } + } + ++static bool blk_root_drained_poll(BdrvChild *child) ++{ ++ BlockBackend *blk = child->opaque; ++ assert(blk->quiesce_counter); ++ return !!blk->in_flight; ++} ++ + static void blk_root_drained_end(BdrvChild *child) + { + BlockBackend *blk = child->opaque; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backend-Add-blk_co_copy_range.patch b/SOURCES/kvm-block-backend-Add-blk_co_copy_range.patch new file mode 100644 index 0000000..406ed20 --- /dev/null +++ b/SOURCES/kvm-block-backend-Add-blk_co_copy_range.patch @@ -0,0 +1,70 @@ +From 1d94395d19ccfd501c5ce7ed2b300a1ac72cf0c0 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:49 +0200 +Subject: [PATCH 175/268] block-backend: Add blk_co_copy_range + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-10-famz@redhat.com> +Patchwork-id: 81161 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 09/13] block-backend: Add blk_co_copy_range +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +It's a BlockBackend wrapper of the BDS interface. + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Message-id: 20180601092648.24614-10-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit b5679fa49c9a70efa7bf01f6efad1a65e2349a0b) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/block-backend.c | 18 ++++++++++++++++++ + include/sysemu/block-backend.h | 4 ++++ + 2 files changed, 22 insertions(+) + +diff --git a/block/block-backend.c b/block/block-backend.c +index 681b240..5562ec4 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -2217,3 +2217,21 @@ void blk_unregister_buf(BlockBackend *blk, void *host) + { + bdrv_unregister_buf(blk_bs(blk), host); + } ++ ++int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, ++ BlockBackend *blk_out, int64_t off_out, ++ int bytes, BdrvRequestFlags flags) ++{ ++ int r; ++ r = blk_check_byte_request(blk_in, off_in, bytes); ++ if (r) { ++ return r; ++ } ++ r = blk_check_byte_request(blk_out, off_out, bytes); ++ if (r) { ++ return r; ++ } ++ return bdrv_co_copy_range(blk_in->root, off_in, ++ blk_out->root, off_out, ++ bytes, flags); ++} +diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h +index 92ab624..8d03d49 100644 +--- a/include/sysemu/block-backend.h ++++ b/include/sysemu/block-backend.h +@@ -232,4 +232,8 @@ void blk_set_force_allow_inactivate(BlockBackend *blk); + void blk_register_buf(BlockBackend *blk, void *host, size_t size); + void blk_unregister_buf(BlockBackend *blk, void *host); + ++int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, ++ BlockBackend *blk_out, int64_t off_out, ++ int bytes, BdrvRequestFlags flags); ++ + #endif +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backend-Decrease-in_flight-only-after-callback.patch b/SOURCES/kvm-block-backend-Decrease-in_flight-only-after-callback.patch new file mode 100644 index 0000000..ab05bbd --- /dev/null +++ b/SOURCES/kvm-block-backend-Decrease-in_flight-only-after-callback.patch @@ -0,0 +1,74 @@ +From 27fd652033779a16561160bef8aeda7f8f9c04be Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:05 +0100 +Subject: [PATCH 39/49] block-backend: Decrease in_flight only after callback + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-27-kwolf@redhat.com> +Patchwork-id: 82617 +O-Subject: [RHEL-8 qemu-kvm PATCH 36/44] block-backend: Decrease in_flight only after callback +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Request callbacks can do pretty much anything, including operations that +will yield from the coroutine (such as draining the backend). In that +case, a decreased in_flight would be visible to other code and could +lead to a drain completing while the callback hasn't actually completed +yet. + +Note that reordering these operations forbids calling drain directly +inside an AIO callback. As Paolo explains, indirectly calling it is +okay: + +- Calling it through a coroutine is okay, because then + bdrv_drained_begin() goes through bdrv_co_yield_to_drain() and you + have in_flight=2 when bdrv_co_yield_to_drain() yields, then soon + in_flight=1 when the aio_co_wake() in the AIO callback completes, then + in_flight=0 after the bottom half starts. + +- Calling it through a bottom half would be okay too, as long as the AIO + callback remembers to do inc_in_flight/dec_in_flight just like + bdrv_co_yield_to_drain() and bdrv_co_drain_bh_cb() do + +A few more important cases that come to mind: + +- A coroutine that yields because of I/O is okay, with a sequence + similar to bdrv_co_yield_to_drain(). + +- A coroutine that yields with no I/O pending will correctly decrease + in_flight to zero before yielding. + +- Calling more AIO from the callback won't overflow the counter just + because of mutual recursion, because AIO functions always yield at + least once before invoking the callback. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +Reviewed-by: Max Reitz +Reviewed-by: Paolo Bonzini +(cherry picked from commit 46aaf2a566e364a62315219255099cbf1c9b990d) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/block-backend.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/block-backend.c b/block/block-backend.c +index bfd0331..b8ea286 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -1341,8 +1341,8 @@ static const AIOCBInfo blk_aio_em_aiocb_info = { + static void blk_aio_complete(BlkAioEmAIOCB *acb) + { + if (acb->has_returned) { +- blk_dec_in_flight(acb->rwco.blk); + acb->common.cb(acb->common.opaque, acb->rwco.ret); ++ blk_dec_in_flight(acb->rwco.blk); + qemu_aio_unref(acb); + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backend-Fix-potential-double-blk_delete.patch b/SOURCES/kvm-block-backend-Fix-potential-double-blk_delete.patch new file mode 100644 index 0000000..684ef51 --- /dev/null +++ b/SOURCES/kvm-block-backend-Fix-potential-double-blk_delete.patch @@ -0,0 +1,67 @@ +From 0fd571f772eb449da398b029705f4cadf0129f60 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:04 +0100 +Subject: [PATCH 38/49] block-backend: Fix potential double blk_delete() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-26-kwolf@redhat.com> +Patchwork-id: 82615 +O-Subject: [RHEL-8 qemu-kvm PATCH 35/44] block-backend: Fix potential double blk_delete() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +blk_unref() first decreases the refcount of the BlockBackend and calls +blk_delete() if the refcount reaches zero. Requests can still be in +flight at this point, they are only drained during blk_delete(): + +At this point, arbitrary callbacks can run. If any callback takes a +temporary BlockBackend reference, it will first increase the refcount to +1 and then decrease it to 0 again, triggering another blk_delete(). This +will cause a use-after-free crash in the outer blk_delete(). + +Fix it by draining the BlockBackend before decreasing to refcount to 0. +Assert in blk_ref() that it never takes the first refcount (which would +mean that the BlockBackend is already being deleted). + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +Reviewed-by: Max Reitz +(cherry picked from commit 5ca9d21bd1c8eeb578d0964e31bd03d47c25773d) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/block-backend.c | 9 ++++++++- + 1 file changed, 8 insertions(+), 1 deletion(-) + +diff --git a/block/block-backend.c b/block/block-backend.c +index dffc6f0..bfd0331 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -436,6 +436,7 @@ int blk_get_refcnt(BlockBackend *blk) + */ + void blk_ref(BlockBackend *blk) + { ++ assert(blk->refcnt > 0); + blk->refcnt++; + } + +@@ -448,7 +449,13 @@ void blk_unref(BlockBackend *blk) + { + if (blk) { + assert(blk->refcnt > 0); +- if (!--blk->refcnt) { ++ if (blk->refcnt > 1) { ++ blk->refcnt--; ++ } else { ++ blk_drain(blk); ++ /* blk_drain() cannot resurrect blk, nobody held a reference */ ++ assert(blk->refcnt == 1); ++ blk->refcnt = 0; + blk_delete(blk); + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backend-Set-werror-rerror-defaults-in-blk_new.patch b/SOURCES/kvm-block-backend-Set-werror-rerror-defaults-in-blk_new.patch new file mode 100644 index 0000000..4f08d2c --- /dev/null +++ b/SOURCES/kvm-block-backend-Set-werror-rerror-defaults-in-blk_new.patch @@ -0,0 +1,69 @@ +From d0bc458b5072ac0e7ee4d27431d5f614f8dd4328 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:39 +0000 +Subject: [PATCH 14/15] block-backend: Set werror/rerror defaults in blk_new() + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-15-kwolf@redhat.com> +Patchwork-id: 83291 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 14/15] block-backend: Set werror/rerror defaults in blk_new() +Bugzilla: 1657637 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +Currently, the default values for werror and rerror have to be set +explicitly with blk_set_on_error() by the callers of blk_new(). The only +caller actually doing this is blockdev_init(), which is called for +BlockBackends created using -drive. + +In particular, anonymous BlockBackends created with +-device ...,drive= didn't get the correct default set and +instead defaulted to the integer value 0 (= BLOCKDEV_ON_ERROR_REPORT). +This is the intended default for rerror anyway, but the default for +werror should be BLOCKDEV_ON_ERROR_ENOSPC. + +Set the defaults in blk_new() instead so that they apply no matter what +way the BlockBackend was created. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Fam Zheng +(cherry picked from commit cb53460b708db3617ab73248374d071d5552c263) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/block-backend.c | 3 +++ + tests/qemu-iotests/067.out | 1 + + 2 files changed, 4 insertions(+) + +diff --git a/block/block-backend.c b/block/block-backend.c +index 91abfe6..7ae5832 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -325,6 +325,9 @@ BlockBackend *blk_new(uint64_t perm, uint64_t shared_perm) + blk->shared_perm = shared_perm; + blk_set_enable_write_cache(blk, true); + ++ blk->on_read_error = BLOCKDEV_ON_ERROR_REPORT; ++ blk->on_write_error = BLOCKDEV_ON_ERROR_ENOSPC; ++ + block_acct_init(&blk->stats); + + notifier_list_init(&blk->remove_bs_notifiers); +diff --git a/tests/qemu-iotests/067.out b/tests/qemu-iotests/067.out +index 2e71cff..b10c71d 100644 +--- a/tests/qemu-iotests/067.out ++++ b/tests/qemu-iotests/067.out +@@ -385,6 +385,7 @@ Testing: -device virtio-scsi -device scsi-cd,id=cd0 + { + "return": [ + { ++ "io-status": "ok", + "device": "", + "locked": false, + "removable": true, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backup-disable-copy-offloading-for-backup.patch b/SOURCES/kvm-block-backup-disable-copy-offloading-for-backup.patch new file mode 100644 index 0000000..8957e7b --- /dev/null +++ b/SOURCES/kvm-block-backup-disable-copy-offloading-for-backup.patch @@ -0,0 +1,43 @@ +From cfdd0c5695fc86b7f016dbe6e40342020a4931a4 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:53 +0200 +Subject: [PATCH 235/268] block/backup: disable copy offloading for backup + +RH-Author: John Snow +Message-id: <20180718225511.14878-18-jsnow@redhat.com> +Patchwork-id: 81417 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 17/35] block/backup: disable copy offloading for backup +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +Downstream only for 2.12.0; there are several upstream fixes that fix +the copy_range support for block/backup that are in 3.0 that we're not +prepared to backport to 2.12.0. + +For now, then, hardcode block/backup to use the old bounce buffer method +that works without additional caveats. + +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/backup.c b/block/backup.c +index d26eeb5..adb3cbd 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -727,7 +727,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + } else { + job->cluster_size = MAX(BACKUP_CLUSTER_SIZE_DEFAULT, bdi.cluster_size); + } +- job->use_copy_range = true; ++ job->use_copy_range = false; + job->copy_range_size = MIN_NON_ZERO(blk_get_max_transfer(job->common.blk), + blk_get_max_transfer(job->target)); + job->copy_range_size = MAX(job->cluster_size, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backup-fix-fleecing-scheme-use-serialized-writ.patch b/SOURCES/kvm-block-backup-fix-fleecing-scheme-use-serialized-writ.patch new file mode 100644 index 0000000..b406ba2 --- /dev/null +++ b/SOURCES/kvm-block-backup-fix-fleecing-scheme-use-serialized-writ.patch @@ -0,0 +1,141 @@ +From d9a55a5815a040032f85c20020b118dda54bba43 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:58 +0200 +Subject: [PATCH 240/268] block/backup: fix fleecing scheme: use serialized + writes + +RH-Author: John Snow +Message-id: <20180718225511.14878-23-jsnow@redhat.com> +Patchwork-id: 81396 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 22/35] block/backup: fix fleecing scheme: use serialized writes +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Fleecing scheme works as follows: we want a kind of temporary snapshot +of active drive A. We create temporary image B, with B->backing = A. +Then we start backup(sync=none) from A to B. From this point, B reads +as point-in-time snapshot of A (A continues to be active drive, +accepting guest IO). + +This scheme needs some additional synchronization between reads from B +and backup COW operations, otherwise, the following situation is +theoretically possible: + +(assume B is qcow2, client is NBD client, reading from B) + +1. client starts reading and take qcow2 mutex in qcow2_co_preadv, and + goes up to l2 table loading (assume cache miss) + +2) guest write => backup COW => qcow2 write => + try to take qcow2 mutex => waiting + +3. l2 table loaded, we see that cluster is UNALLOCATED, go to + "case QCOW2_CLUSTER_UNALLOCATED" and unlock mutex before + bdrv_co_preadv(bs->backing, ...) + +4) aha, mutex unlocked, backup COW continues, and we finally finish + guest write and change cluster in our active disk A + +5. actually, do bdrv_co_preadv(bs->backing, ...) and read + _new updated_ data. + +To avoid this, let's make backup writes serializing, to not intersect +with reads from B. + +Note: we expand range of handled cases from (sync=none and +B->backing = A) to just (A in backing chain of B), to finally allow +safe reading from B during backup for all cases when A in backing chain +of B, i.e. B formally looks like point-in-time snapshot of A. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit f8d59dfb40bbc6f5aeea57c8aac1e68c1d2454ee) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 20 ++++++++++++++------ + 1 file changed, 14 insertions(+), 6 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 369155a..4ba1a6a 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -47,6 +47,8 @@ typedef struct BackupBlockJob { + HBitmap *copy_bitmap; + bool use_copy_range; + int64_t copy_range_size; ++ ++ bool serialize_target_writes; + } BackupBlockJob; + + static const BlockJobDriver backup_job_driver; +@@ -102,6 +104,8 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, + QEMUIOVector qiov; + BlockBackend *blk = job->common.blk; + int nbytes; ++ int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; ++ int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0; + + hbitmap_reset(job->copy_bitmap, start / job->cluster_size, 1); + nbytes = MIN(job->cluster_size, job->len - start); +@@ -112,8 +116,7 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, + iov.iov_len = nbytes; + qemu_iovec_init_external(&qiov, &iov, 1); + +- ret = blk_co_preadv(blk, start, qiov.size, &qiov, +- is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); ++ ret = blk_co_preadv(blk, start, qiov.size, &qiov, read_flags); + if (ret < 0) { + trace_backup_do_cow_read_fail(job, start, ret); + if (error_is_read) { +@@ -124,11 +127,11 @@ static int coroutine_fn backup_cow_with_bounce_buffer(BackupBlockJob *job, + + if (qemu_iovec_is_zero(&qiov)) { + ret = blk_co_pwrite_zeroes(job->target, start, +- qiov.size, BDRV_REQ_MAY_UNMAP); ++ qiov.size, write_flags | BDRV_REQ_MAY_UNMAP); + } else { + ret = blk_co_pwritev(job->target, start, +- qiov.size, &qiov, +- job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0); ++ qiov.size, &qiov, write_flags | ++ (job->compress ? BDRV_REQ_WRITE_COMPRESSED : 0)); + } + if (ret < 0) { + trace_backup_do_cow_write_fail(job, start, ret); +@@ -156,6 +159,8 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, + int nr_clusters; + BlockBackend *blk = job->common.blk; + int nbytes; ++ int read_flags = is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0; ++ int write_flags = job->serialize_target_writes ? BDRV_REQ_SERIALISING : 0; + + assert(QEMU_IS_ALIGNED(job->copy_range_size, job->cluster_size)); + nbytes = MIN(job->copy_range_size, end - start); +@@ -163,7 +168,7 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, + hbitmap_reset(job->copy_bitmap, start / job->cluster_size, + nr_clusters); + ret = blk_co_copy_range(blk, start, job->target, start, nbytes, +- is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0, 0); ++ read_flags, write_flags); + if (ret < 0) { + trace_backup_do_cow_copy_range_fail(job, start, ret); + hbitmap_set(job->copy_bitmap, start / job->cluster_size, +@@ -701,6 +706,9 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + sync_bitmap : NULL; + job->compress = compress; + ++ /* Detect image-fleecing (and similar) schemes */ ++ job->serialize_target_writes = bdrv_chain_contains(target, bs); ++ + /* If there is no backing file on the target, we cannot rely on COW if our + * backup cluster size is smaller than the target cluster size. Even for + * targets with a backing file, try to avoid COW if possible. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backup-make-function-variables-consistently-na.patch b/SOURCES/kvm-block-backup-make-function-variables-consistently-na.patch new file mode 100644 index 0000000..00cb95b --- /dev/null +++ b/SOURCES/kvm-block-backup-make-function-variables-consistently-na.patch @@ -0,0 +1,166 @@ +From 2327f8fa6d0f917cf4ced7385aa27e1bb45b08ec Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:13 +0100 +Subject: [PATCH 10/28] block/backup: make function variables consistently + named + +RH-Author: John Snow +Message-id: <20180925223431.24791-8-jsnow@redhat.com> +Patchwork-id: 82272 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 07/25] block/backup: make function variables consistently named +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Rename opaque_job to job to be consistent with other job implementations. +Rename 'job', the BackupBlockJob object, to 's' to also be consistent. + +Suggested-by: Eric Blake +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180830015734.19765-8-jsnow@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 6870277535493fea31761d8d11ec23add2de0fb0) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/backup.c | 62 +++++++++++++++++++++++++++++----------------------------- + 1 file changed, 31 insertions(+), 31 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 08a5b74..524e0ff 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -468,59 +468,59 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) + bdrv_dirty_iter_free(dbi); + } + +-static int coroutine_fn backup_run(Job *opaque_job, Error **errp) ++static int coroutine_fn backup_run(Job *job, Error **errp) + { +- BackupBlockJob *job = container_of(opaque_job, BackupBlockJob, common.job); +- BlockDriverState *bs = blk_bs(job->common.blk); ++ BackupBlockJob *s = container_of(job, BackupBlockJob, common.job); ++ BlockDriverState *bs = blk_bs(s->common.blk); + int64_t offset, nb_clusters; + int ret = 0; + +- QLIST_INIT(&job->inflight_reqs); +- qemu_co_rwlock_init(&job->flush_rwlock); ++ QLIST_INIT(&s->inflight_reqs); ++ qemu_co_rwlock_init(&s->flush_rwlock); + +- nb_clusters = DIV_ROUND_UP(job->len, job->cluster_size); +- job_progress_set_remaining(&job->common.job, job->len); ++ nb_clusters = DIV_ROUND_UP(s->len, s->cluster_size); ++ job_progress_set_remaining(job, s->len); + +- job->copy_bitmap = hbitmap_alloc(nb_clusters, 0); +- if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { +- backup_incremental_init_copy_bitmap(job); ++ s->copy_bitmap = hbitmap_alloc(nb_clusters, 0); ++ if (s->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { ++ backup_incremental_init_copy_bitmap(s); + } else { +- hbitmap_set(job->copy_bitmap, 0, nb_clusters); ++ hbitmap_set(s->copy_bitmap, 0, nb_clusters); + } + + +- job->before_write.notify = backup_before_write_notify; +- bdrv_add_before_write_notifier(bs, &job->before_write); ++ s->before_write.notify = backup_before_write_notify; ++ bdrv_add_before_write_notifier(bs, &s->before_write); + +- if (job->sync_mode == MIRROR_SYNC_MODE_NONE) { ++ if (s->sync_mode == MIRROR_SYNC_MODE_NONE) { + /* All bits are set in copy_bitmap to allow any cluster to be copied. + * This does not actually require them to be copied. */ +- while (!job_is_cancelled(&job->common.job)) { ++ while (!job_is_cancelled(job)) { + /* Yield until the job is cancelled. We just let our before_write + * notify callback service CoW requests. */ +- job_yield(&job->common.job); ++ job_yield(job); + } +- } else if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { +- ret = backup_run_incremental(job); ++ } else if (s->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { ++ ret = backup_run_incremental(s); + } else { + /* Both FULL and TOP SYNC_MODE's require copying.. */ +- for (offset = 0; offset < job->len; +- offset += job->cluster_size) { ++ for (offset = 0; offset < s->len; ++ offset += s->cluster_size) { + bool error_is_read; + int alloced = 0; + +- if (yield_and_check(job)) { ++ if (yield_and_check(s)) { + break; + } + +- if (job->sync_mode == MIRROR_SYNC_MODE_TOP) { ++ if (s->sync_mode == MIRROR_SYNC_MODE_TOP) { + int i; + int64_t n; + + /* Check to see if these blocks are already in the + * backing file. */ + +- for (i = 0; i < job->cluster_size;) { ++ for (i = 0; i < s->cluster_size;) { + /* bdrv_is_allocated() only returns true/false based + * on the first set of sectors it comes across that + * are are all in the same state. +@@ -529,7 +529,7 @@ static int coroutine_fn backup_run(Job *opaque_job, Error **errp) + * needed but at some point that is always the case. */ + alloced = + bdrv_is_allocated(bs, offset + i, +- job->cluster_size - i, &n); ++ s->cluster_size - i, &n); + i += n; + + if (alloced || n == 0) { +@@ -547,29 +547,29 @@ static int coroutine_fn backup_run(Job *opaque_job, Error **errp) + if (alloced < 0) { + ret = alloced; + } else { +- ret = backup_do_cow(job, offset, job->cluster_size, ++ ret = backup_do_cow(s, offset, s->cluster_size, + &error_is_read, false); + } + if (ret < 0) { + /* Depending on error action, fail now or retry cluster */ + BlockErrorAction action = +- backup_error_action(job, error_is_read, -ret); ++ backup_error_action(s, error_is_read, -ret); + if (action == BLOCK_ERROR_ACTION_REPORT) { + break; + } else { +- offset -= job->cluster_size; ++ offset -= s->cluster_size; + continue; + } + } + } + } + +- notifier_with_return_remove(&job->before_write); ++ notifier_with_return_remove(&s->before_write); + + /* wait until pending backup_do_cow() calls have completed */ +- qemu_co_rwlock_wrlock(&job->flush_rwlock); +- qemu_co_rwlock_unlock(&job->flush_rwlock); +- hbitmap_free(job->copy_bitmap); ++ qemu_co_rwlock_wrlock(&s->flush_rwlock); ++ qemu_co_rwlock_unlock(&s->flush_rwlock); ++ hbitmap_free(s->copy_bitmap); + + return ret; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backup-prohibit-backup-from-using-in-use-bitma.patch b/SOURCES/kvm-block-backup-prohibit-backup-from-using-in-use-bitma.patch new file mode 100644 index 0000000..d4213b3 --- /dev/null +++ b/SOURCES/kvm-block-backup-prohibit-backup-from-using-in-use-bitma.patch @@ -0,0 +1,62 @@ +From 238718e949e885a511fbc9c2486bddec78682341 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:20 +0000 +Subject: [PATCH 26/35] block/backup: prohibit backup from using in use bitmaps + +RH-Author: John Snow +Message-id: <20181120181828.15132-17-jsnow@redhat.com> +Patchwork-id: 83076 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 16/24] block/backup: prohibit backup from using in use bitmaps +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +If the bitmap is frozen, we shouldn't touch it. + +Signed-off-by: John Snow +Reviewed-by: Vladimir Sementsov-Ogievskiy +Message-id: 20181002230218.13949-6-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit b27a6b8b329a8dcbab9dc1af45586f7585f3d47b) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index 916153e..56a3d0f 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3622,10 +3622,10 @@ static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn, + bdrv_unref(target_bs); + goto out; + } +- if (bdrv_dirty_bitmap_qmp_locked(bmap)) { ++ if (bdrv_dirty_bitmap_user_locked(bmap)) { + error_setg(errp, +- "Bitmap '%s' is currently locked and cannot be used for " +- "backup", backup->bitmap); ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be used for backup", backup->bitmap); + goto out; + } + } +@@ -3730,10 +3730,10 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap); + goto out; + } +- if (bdrv_dirty_bitmap_qmp_locked(bmap)) { ++ if (bdrv_dirty_bitmap_user_locked(bmap)) { + error_setg(errp, +- "Bitmap '%s' is currently locked and cannot be used for " +- "backup", backup->bitmap); ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be used for backup", backup->bitmap); + goto out; + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-backup-qapi-documentation-fixup.patch b/SOURCES/kvm-block-backup-qapi-documentation-fixup.patch new file mode 100644 index 0000000..b02e868 --- /dev/null +++ b/SOURCES/kvm-block-backup-qapi-documentation-fixup.patch @@ -0,0 +1,73 @@ +From 0d30ab413521525f60d9b45428157fe3ba998677 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:30 +0100 +Subject: [PATCH 27/28] block/backup: qapi documentation fixup + +RH-Author: John Snow +Message-id: <20180925223431.24791-25-jsnow@redhat.com> +Patchwork-id: 82284 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 24/25] block/backup: qapi documentation fixup +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Fix documentation to match the other jobs amended for 3.1. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-16-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit dfaff2c37dfa52ab045cf87503e60ea56317230a) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + qapi/block-core.json | 18 ++++++++++-------- + 1 file changed, 10 insertions(+), 8 deletions(-) + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index ba4bba0..602c028 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1249,13 +1249,14 @@ + # a different block device than @device). + # + # @auto-finalize: When false, this job will wait in a PENDING state after it has +-# finished its work, waiting for @block-job-finalize. +-# When true, this job will automatically perform its abort or +-# commit actions. ++# finished its work, waiting for @block-job-finalize before ++# making any block graph changes. ++# When true, this job will automatically ++# perform its abort or commit actions. + # Defaults to true. (Since 2.12) + # + # @auto-dismiss: When false, this job will wait in a CONCLUDED state after it +-# has completed ceased all work, and wait for @block-job-dismiss. ++# has completely ceased all work, and awaits @block-job-dismiss. + # When true, this job will automatically disappear from the query + # list without user intervention. + # Defaults to true. (Since 2.12) +@@ -1304,13 +1305,14 @@ + # a different block device than @device). + # + # @auto-finalize: When false, this job will wait in a PENDING state after it has +-# finished its work, waiting for @block-job-finalize. +-# When true, this job will automatically perform its abort or +-# commit actions. ++# finished its work, waiting for @block-job-finalize before ++# making any block graph changes. ++# When true, this job will automatically ++# perform its abort or commit actions. + # Defaults to true. (Since 2.12) + # + # @auto-dismiss: When false, this job will wait in a CONCLUDED state after it +-# has completed ceased all work, and wait for @block-job-dismiss. ++# has completely ceased all work, and awaits @block-job-dismiss. + # When true, this job will automatically disappear from the query + # list without user intervention. + # Defaults to true. (Since 2.12) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-commit-add-block-job-creation-flags.patch b/SOURCES/kvm-block-commit-add-block-job-creation-flags.patch new file mode 100644 index 0000000..d84e627 --- /dev/null +++ b/SOURCES/kvm-block-commit-add-block-job-creation-flags.patch @@ -0,0 +1,110 @@ +From 80f064aafcf344ba6c87b948b7bc8f9c13cd8ab1 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:16 +0100 +Subject: [PATCH 13/28] block/commit: add block job creation flags + +RH-Author: John Snow +Message-id: <20180925223431.24791-11-jsnow@redhat.com> +Patchwork-id: 82264 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 10/25] block/commit: add block job creation flags +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Add support for taking and passing forward job creation flags. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +Message-id: 20180906130225.5118-2-jsnow@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 5360782d0827854383097d560715d8d8027ee590) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/commit.c | 5 +++-- + blockdev.c | 7 ++++--- + include/block/block_int.h | 5 ++++- + 3 files changed, 11 insertions(+), 6 deletions(-) + +diff --git a/block/commit.c b/block/commit.c +index 25b3cb8..c737664 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -254,7 +254,8 @@ static BlockDriver bdrv_commit_top = { + }; + + void commit_start(const char *job_id, BlockDriverState *bs, +- BlockDriverState *base, BlockDriverState *top, int64_t speed, ++ BlockDriverState *base, BlockDriverState *top, ++ int creation_flags, int64_t speed, + BlockdevOnError on_error, const char *backing_file_str, + const char *filter_node_name, Error **errp) + { +@@ -272,7 +273,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, + } + + s = block_job_create(job_id, &commit_job_driver, NULL, bs, 0, BLK_PERM_ALL, +- speed, JOB_DEFAULT, NULL, NULL, errp); ++ speed, creation_flags, NULL, NULL, errp); + if (!s) { + return; + } +diff --git a/blockdev.c b/blockdev.c +index 0bdd3b5..4f96aa7 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3325,6 +3325,7 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + * BlockdevOnError change for blkmirror makes it in + */ + BlockdevOnError on_error = BLOCKDEV_ON_ERROR_REPORT; ++ int job_flags = JOB_DEFAULT; + + if (!has_speed) { + speed = 0; +@@ -3406,15 +3407,15 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + goto out; + } + commit_active_start(has_job_id ? job_id : NULL, bs, base_bs, +- JOB_DEFAULT, speed, on_error, ++ job_flags, speed, on_error, + filter_node_name, NULL, NULL, false, &local_err); + } else { + BlockDriverState *overlay_bs = bdrv_find_overlay(bs, top_bs); + if (bdrv_op_is_blocked(overlay_bs, BLOCK_OP_TYPE_COMMIT_TARGET, errp)) { + goto out; + } +- commit_start(has_job_id ? job_id : NULL, bs, base_bs, top_bs, speed, +- on_error, has_backing_file ? backing_file : NULL, ++ commit_start(has_job_id ? job_id : NULL, bs, base_bs, top_bs, job_flags, ++ speed, on_error, has_backing_file ? backing_file : NULL, + filter_node_name, &local_err); + } + if (local_err != NULL) { +diff --git a/include/block/block_int.h b/include/block/block_int.h +index b05cf11..25ad363 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -951,6 +951,8 @@ void stream_start(const char *job_id, BlockDriverState *bs, + * @bs: Active block device. + * @top: Top block device to be committed. + * @base: Block device that will be written into, and become the new top. ++ * @creation_flags: Flags that control the behavior of the Job lifetime. ++ * See @BlockJobCreateFlags + * @speed: The maximum speed, in bytes per second, or 0 for unlimited. + * @on_error: The action to take upon error. + * @backing_file_str: String to use as the backing file in @top's overlay +@@ -961,7 +963,8 @@ void stream_start(const char *job_id, BlockDriverState *bs, + * + */ + void commit_start(const char *job_id, BlockDriverState *bs, +- BlockDriverState *base, BlockDriverState *top, int64_t speed, ++ BlockDriverState *base, BlockDriverState *top, ++ int creation_flags, int64_t speed, + BlockdevOnError on_error, const char *backing_file_str, + const char *filter_node_name, Error **errp); + /** +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-commit-refactor-commit-to-use-job-callbacks.patch b/SOURCES/kvm-block-commit-refactor-commit-to-use-job-callbacks.patch new file mode 100644 index 0000000..a3a9c77 --- /dev/null +++ b/SOURCES/kvm-block-commit-refactor-commit-to-use-job-callbacks.patch @@ -0,0 +1,180 @@ +From e24c270ca2e6a0ebd01403c19703002239477243 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:19 +0100 +Subject: [PATCH 16/28] block/commit: refactor commit to use job callbacks + +RH-Author: John Snow +Message-id: <20180925223431.24791-14-jsnow@redhat.com> +Patchwork-id: 82279 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 13/25] block/commit: refactor commit to use job callbacks +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Use the component callbacks; prepare, abort, and clean. + +NB: prepare is only called when the job has not yet failed; +and abort can be called after prepare. + +complete -> prepare -> abort -> clean +complete -> abort -> clean + +During refactor, a potential problem with bdrv_drop_intermediate +was identified, the patched behavior is no worse than the pre-patch +behavior, so leave a FIXME for now to be fixed in a future patch. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-5-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 22dffcbec62ba918db690ed44beba4bd4e970bb9) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/commit.c | 92 ++++++++++++++++++++++++++++++++-------------------------- + 1 file changed, 51 insertions(+), 41 deletions(-) + +diff --git a/block/commit.c b/block/commit.c +index c737664..b387765 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -36,6 +36,7 @@ typedef struct CommitBlockJob { + BlockDriverState *commit_top_bs; + BlockBackend *top; + BlockBackend *base; ++ BlockDriverState *base_bs; + BlockdevOnError on_error; + int base_flags; + char *backing_file_str; +@@ -68,61 +69,67 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base, + return 0; + } + +-static void commit_exit(Job *job) ++static int commit_prepare(Job *job) + { + CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); +- BlockJob *bjob = &s->common; +- BlockDriverState *top = blk_bs(s->top); +- BlockDriverState *base = blk_bs(s->base); +- BlockDriverState *commit_top_bs = s->commit_top_bs; +- bool remove_commit_top_bs = false; +- +- /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */ +- bdrv_ref(top); +- bdrv_ref(commit_top_bs); + + /* Remove base node parent that still uses BLK_PERM_WRITE/RESIZE before + * the normal backing chain can be restored. */ + blk_unref(s->base); ++ s->base = NULL; ++ ++ /* FIXME: bdrv_drop_intermediate treats total failures and partial failures ++ * identically. Further work is needed to disambiguate these cases. */ ++ return bdrv_drop_intermediate(s->commit_top_bs, s->base_bs, ++ s->backing_file_str); ++} + +- if (!job_is_cancelled(job) && job->ret == 0) { +- /* success */ +- job->ret = bdrv_drop_intermediate(s->commit_top_bs, base, +- s->backing_file_str); +- } else { +- /* XXX Can (or should) we somehow keep 'consistent read' blocked even +- * after the failed/cancelled commit job is gone? If we already wrote +- * something to base, the intermediate images aren't valid any more. */ +- remove_commit_top_bs = true; ++static void commit_abort(Job *job) ++{ ++ CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); ++ BlockDriverState *top_bs = blk_bs(s->top); ++ ++ /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */ ++ bdrv_ref(top_bs); ++ bdrv_ref(s->commit_top_bs); ++ ++ if (s->base) { ++ blk_unref(s->base); + } + ++ /* free the blockers on the intermediate nodes so that bdrv_replace_nodes ++ * can succeed */ ++ block_job_remove_all_bdrv(&s->common); ++ ++ /* If bdrv_drop_intermediate() failed (or was not invoked), remove the ++ * commit filter driver from the backing chain now. Do this as the final ++ * step so that the 'consistent read' permission can be granted. ++ * ++ * XXX Can (or should) we somehow keep 'consistent read' blocked even ++ * after the failed/cancelled commit job is gone? If we already wrote ++ * something to base, the intermediate images aren't valid any more. */ ++ bdrv_child_try_set_perm(s->commit_top_bs->backing, 0, BLK_PERM_ALL, ++ &error_abort); ++ bdrv_replace_node(s->commit_top_bs, backing_bs(s->commit_top_bs), ++ &error_abort); ++ ++ bdrv_unref(s->commit_top_bs); ++ bdrv_unref(top_bs); ++} ++ ++static void commit_clean(Job *job) ++{ ++ CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); ++ + /* restore base open flags here if appropriate (e.g., change the base back + * to r/o). These reopens do not need to be atomic, since we won't abort + * even on failure here */ +- if (s->base_flags != bdrv_get_flags(base)) { +- bdrv_reopen(base, s->base_flags, NULL); ++ if (s->base_flags != bdrv_get_flags(s->base_bs)) { ++ bdrv_reopen(s->base_bs, s->base_flags, NULL); + } ++ + g_free(s->backing_file_str); + blk_unref(s->top); +- +- /* If there is more than one reference to the job (e.g. if called from +- * job_finish_sync()), job_completed() won't free it and therefore the +- * blockers on the intermediate nodes remain. This would cause +- * bdrv_set_backing_hd() to fail. */ +- block_job_remove_all_bdrv(bjob); +- +- /* If bdrv_drop_intermediate() didn't already do that, remove the commit +- * filter driver from the backing chain. Do this as the final step so that +- * the 'consistent read' permission can be granted. */ +- if (remove_commit_top_bs) { +- bdrv_child_try_set_perm(commit_top_bs->backing, 0, BLK_PERM_ALL, +- &error_abort); +- bdrv_replace_node(commit_top_bs, backing_bs(commit_top_bs), +- &error_abort); +- } +- +- bdrv_unref(commit_top_bs); +- bdrv_unref(top); + } + + static int coroutine_fn commit_run(Job *job, Error **errp) +@@ -211,7 +218,9 @@ static const BlockJobDriver commit_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = commit_run, +- .exit = commit_exit, ++ .prepare = commit_prepare, ++ .abort = commit_abort, ++ .clean = commit_clean + }, + }; + +@@ -350,6 +359,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, + if (ret < 0) { + goto fail; + } ++ s->base_bs = base; + + /* Required permissions are already taken with block_job_add_bdrv() */ + s->top = blk_new(0, BLK_PERM_ALL); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-commit-utilize-job_exit-shim.patch b/SOURCES/kvm-block-commit-utilize-job_exit-shim.patch new file mode 100644 index 0000000..8e22f61 --- /dev/null +++ b/SOURCES/kvm-block-commit-utilize-job_exit-shim.patch @@ -0,0 +1,115 @@ +From 1203019dd3f126849c6409db6a282342e7cf1280 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:10 +0100 +Subject: [PATCH 07/28] block/commit: utilize job_exit shim + +RH-Author: John Snow +Message-id: <20180925223431.24791-5-jsnow@redhat.com> +Patchwork-id: 82265 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 04/25] block/commit: utilize job_exit shim +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Change the manual deferment to commit_complete into the implicit +callback to job_exit, renaming commit_complete to commit_exit. + +This conversion does change the timing of when job_completed is +called to after the bdrv_replace_node and bdrv_unref calls, which +could have implications for bjob->blk which will now be put down +after this cleanup. + +Kevin highlights that we did not take any permissions for that backend +at job creation time, so it is safe to reorder these operations. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180830015734.19765-5-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit f369b48dc4095861223f9bc4329935599e03b1c5) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/commit.c | 22 +++++----------------- + 1 file changed, 5 insertions(+), 17 deletions(-) + +diff --git a/block/commit.c b/block/commit.c +index af7579d..25b3cb8 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -68,19 +68,13 @@ static int coroutine_fn commit_populate(BlockBackend *bs, BlockBackend *base, + return 0; + } + +-typedef struct { +- int ret; +-} CommitCompleteData; +- +-static void commit_complete(Job *job, void *opaque) ++static void commit_exit(Job *job) + { + CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); + BlockJob *bjob = &s->common; +- CommitCompleteData *data = opaque; + BlockDriverState *top = blk_bs(s->top); + BlockDriverState *base = blk_bs(s->base); + BlockDriverState *commit_top_bs = s->commit_top_bs; +- int ret = data->ret; + bool remove_commit_top_bs = false; + + /* Make sure commit_top_bs and top stay around until bdrv_replace_node() */ +@@ -91,10 +85,10 @@ static void commit_complete(Job *job, void *opaque) + * the normal backing chain can be restored. */ + blk_unref(s->base); + +- if (!job_is_cancelled(job) && ret == 0) { ++ if (!job_is_cancelled(job) && job->ret == 0) { + /* success */ +- ret = bdrv_drop_intermediate(s->commit_top_bs, base, +- s->backing_file_str); ++ job->ret = bdrv_drop_intermediate(s->commit_top_bs, base, ++ s->backing_file_str); + } else { + /* XXX Can (or should) we somehow keep 'consistent read' blocked even + * after the failed/cancelled commit job is gone? If we already wrote +@@ -117,9 +111,6 @@ static void commit_complete(Job *job, void *opaque) + * bdrv_set_backing_hd() to fail. */ + block_job_remove_all_bdrv(bjob); + +- job_completed(job, ret); +- g_free(data); +- + /* If bdrv_drop_intermediate() didn't already do that, remove the commit + * filter driver from the backing chain. Do this as the final step so that + * the 'consistent read' permission can be granted. */ +@@ -137,7 +128,6 @@ static void commit_complete(Job *job, void *opaque) + static int coroutine_fn commit_run(Job *job, Error **errp) + { + CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); +- CommitCompleteData *data; + int64_t offset; + uint64_t delay_ns = 0; + int ret = 0; +@@ -210,9 +200,6 @@ static int coroutine_fn commit_run(Job *job, Error **errp) + out: + qemu_vfree(buf); + +- data = g_malloc(sizeof(*data)); +- data->ret = ret; +- job_defer_to_main_loop(&s->common.job, commit_complete, data); + return ret; + } + +@@ -224,6 +211,7 @@ static const BlockJobDriver commit_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = commit_run, ++ .exit = commit_exit, + }, + }; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-create-Make-x-blockdev-create-a-job.patch b/SOURCES/kvm-block-create-Make-x-blockdev-create-a-job.patch new file mode 100644 index 0000000..e1d718c --- /dev/null +++ b/SOURCES/kvm-block-create-Make-x-blockdev-create-a-job.patch @@ -0,0 +1,217 @@ +From c8efd4c89062a362a9505fae8e1c8612dfc88e8a Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:40 +0200 +Subject: [PATCH 132/268] block/create: Make x-blockdev-create a job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-58-kwolf@redhat.com> +Patchwork-id: 81066 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 57/73] block/create: Make x-blockdev-create a job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This changes the x-blockdev-create QMP command so that it doesn't block +the monitor and the main loop any more, but starts a background job that +performs the image creation. + +The basic job as implemented here is all that is necessary to make image +creation asynchronous and to provide a QMP interface that can be marked +stable, but it still lacks a few features that jobs usually provide: The +job will ignore pause commands and it doesn't publish more than very +basic progress yet (total-progress is 1 and current-progress advances +from 0 to 1 when the driver callbacks returns). These features can be +added later without breaking compatibility. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit e5ab4347f9f53495e31fcef5e232c7c6be4a0567) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/create.c | 67 +++++++++++++++++++++++++++++++++--------------- + qapi/block-core.json | 14 ++++++---- + qapi/job.json | 4 ++- + tests/qemu-iotests/group | 14 +++++----- + 4 files changed, 66 insertions(+), 33 deletions(-) + +diff --git a/block/create.c b/block/create.c +index 8bd8a03..1a263e4 100644 +--- a/block/create.c ++++ b/block/create.c +@@ -24,28 +24,51 @@ + + #include "qemu/osdep.h" + #include "block/block_int.h" ++#include "qemu/job.h" + #include "qapi/qapi-commands-block-core.h" ++#include "qapi/qapi-visit-block-core.h" ++#include "qapi/clone-visitor.h" + #include "qapi/error.h" + +-typedef struct BlockdevCreateCo { ++typedef struct BlockdevCreateJob { ++ Job common; + BlockDriver *drv; + BlockdevCreateOptions *opts; + int ret; +- Error **errp; +-} BlockdevCreateCo; ++ Error *err; ++} BlockdevCreateJob; + +-static void coroutine_fn bdrv_co_create_co_entry(void *opaque) ++static void blockdev_create_complete(Job *job, void *opaque) + { +- BlockdevCreateCo *cco = opaque; +- cco->ret = cco->drv->bdrv_co_create(cco->opts, cco->errp); ++ BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common); ++ ++ job_completed(job, s->ret, s->err); + } + +-void qmp_x_blockdev_create(BlockdevCreateOptions *options, Error **errp) ++static void coroutine_fn blockdev_create_run(void *opaque) + { ++ BlockdevCreateJob *s = opaque; ++ ++ job_progress_set_remaining(&s->common, 1); ++ s->ret = s->drv->bdrv_co_create(s->opts, &s->err); ++ job_progress_update(&s->common, 1); ++ ++ qapi_free_BlockdevCreateOptions(s->opts); ++ job_defer_to_main_loop(&s->common, blockdev_create_complete, NULL); ++} ++ ++static const JobDriver blockdev_create_job_driver = { ++ .instance_size = sizeof(BlockdevCreateJob), ++ .job_type = JOB_TYPE_CREATE, ++ .start = blockdev_create_run, ++}; ++ ++void qmp_x_blockdev_create(const char *job_id, BlockdevCreateOptions *options, ++ Error **errp) ++{ ++ BlockdevCreateJob *s; + const char *fmt = BlockdevDriver_str(options->driver); + BlockDriver *drv = bdrv_find_format(fmt); +- Coroutine *co; +- BlockdevCreateCo cco; + + /* If the driver is in the schema, we know that it exists. But it may not + * be whitelisted. */ +@@ -55,22 +78,24 @@ void qmp_x_blockdev_create(BlockdevCreateOptions *options, Error **errp) + return; + } + +- /* Call callback if it exists */ ++ /* Error out if the driver doesn't support .bdrv_co_create */ + if (!drv->bdrv_co_create) { + error_setg(errp, "Driver does not support blockdev-create"); + return; + } + +- cco = (BlockdevCreateCo) { +- .drv = drv, +- .opts = options, +- .ret = -EINPROGRESS, +- .errp = errp, +- }; +- +- co = qemu_coroutine_create(bdrv_co_create_co_entry, &cco); +- qemu_coroutine_enter(co); +- while (cco.ret == -EINPROGRESS) { +- aio_poll(qemu_get_aio_context(), true); ++ /* Create the block job */ ++ /* TODO Running in the main context. Block drivers need to error out or add ++ * locking when they use a BDS in a different AioContext. */ ++ s = job_create(job_id, &blockdev_create_job_driver, NULL, ++ qemu_get_aio_context(), JOB_DEFAULT | JOB_MANUAL_DISMISS, ++ NULL, NULL, errp); ++ if (!s) { ++ return; + } ++ ++ s->drv = drv, ++ s->opts = QAPI_CLONE(BlockdevCreateOptions, options), ++ ++ job_start(&s->common); + } +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 96ddf87..050fbf3 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -4027,14 +4027,18 @@ + ## + # @x-blockdev-create: + # +-# Create an image format on a given node. +-# TODO Replace with something asynchronous (block job?) ++# Starts a job to create an image format on a given node. The job is ++# automatically finalized, but a manual job-dismiss is required. + # +-# Since: 2.12 ++# @job-id: Identifier for the newly created job. ++# ++# @options: Options for the image creation. ++# ++# Since: 3.0 + ## + { 'command': 'x-blockdev-create', +- 'data': 'BlockdevCreateOptions', +- 'boxed': true } ++ 'data': { 'job-id': 'str', ++ 'options': 'BlockdevCreateOptions' } } + + ## + # @blockdev-open-tray: +diff --git a/qapi/job.json b/qapi/job.json +index 970124d..69c1970 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -17,10 +17,12 @@ + # + # @backup: drive backup job type, see "drive-backup" + # ++# @create: image creation job type, see "x-blockdev-create" (since 3.0) ++# + # Since: 1.7 + ## + { 'enum': 'JobType', +- 'data': ['commit', 'stream', 'mirror', 'backup'] } ++ 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] } + + ## + # @JobStatus: +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 5c55adc..37ec5f1 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -204,14 +204,16 @@ + 203 rw auto + 204 rw auto quick + 205 rw auto quick +-206 rw auto +-207 rw auto ++# TODO The following commented out tests need to be reworked to work ++# with the x-blockdev-create job ++#206 rw auto ++#207 rw auto + 208 rw auto quick + 209 rw auto quick +-210 rw auto +-211 rw auto quick +-212 rw auto quick +-213 rw auto quick ++#210 rw auto ++#211 rw auto quick ++#212 rw auto quick ++#213 rw auto quick + 214 rw auto + 215 rw auto quick + 216 rw auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-create-Mark-blockdev-create-stable.patch b/SOURCES/kvm-block-create-Mark-blockdev-create-stable.patch new file mode 100644 index 0000000..4fe3aab --- /dev/null +++ b/SOURCES/kvm-block-create-Mark-blockdev-create-stable.patch @@ -0,0 +1,969 @@ +From 53c3256f19482b02a18c9402908df39ec498fdef Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:52 +0200 +Subject: [PATCH 144/268] block/create: Mark blockdev-create stable + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-70-kwolf@redhat.com> +Patchwork-id: 81104 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 69/73] block/create: Mark blockdev-create stable +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +We're ready to declare the blockdev-create job stable. This renames the +corresponding QMP command from x-blockdev-create to blockdev-create. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 3fb588a0f2c006122c34e1960a15c87ae2b927eb) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/create.c | 4 ++-- + qapi/block-core.json | 4 ++-- + qapi/job.json | 2 +- + tests/qemu-iotests/206 | 2 +- + tests/qemu-iotests/206.out | 54 +++++++++++++++++++++++----------------------- + tests/qemu-iotests/207 | 2 +- + tests/qemu-iotests/207.out | 18 ++++++++-------- + tests/qemu-iotests/210 | 2 +- + tests/qemu-iotests/210.out | 18 ++++++++-------- + tests/qemu-iotests/211 | 2 +- + tests/qemu-iotests/211.out | 24 ++++++++++----------- + tests/qemu-iotests/212 | 2 +- + tests/qemu-iotests/212.out | 42 ++++++++++++++++++------------------ + tests/qemu-iotests/213 | 2 +- + tests/qemu-iotests/213.out | 44 ++++++++++++++++++------------------- + 15 files changed, 111 insertions(+), 111 deletions(-) + +diff --git a/block/create.c b/block/create.c +index 1a263e4..915cd41 100644 +--- a/block/create.c ++++ b/block/create.c +@@ -63,8 +63,8 @@ static const JobDriver blockdev_create_job_driver = { + .start = blockdev_create_run, + }; + +-void qmp_x_blockdev_create(const char *job_id, BlockdevCreateOptions *options, +- Error **errp) ++void qmp_blockdev_create(const char *job_id, BlockdevCreateOptions *options, ++ Error **errp) + { + BlockdevCreateJob *s; + const char *fmt = BlockdevDriver_str(options->driver); +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 050fbf3..b0f41f1 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -4025,7 +4025,7 @@ + } } + + ## +-# @x-blockdev-create: ++# @blockdev-create: + # + # Starts a job to create an image format on a given node. The job is + # automatically finalized, but a manual job-dismiss is required. +@@ -4036,7 +4036,7 @@ + # + # Since: 3.0 + ## +-{ 'command': 'x-blockdev-create', ++{ 'command': 'blockdev-create', + 'data': { 'job-id': 'str', + 'options': 'BlockdevCreateOptions' } } + +diff --git a/qapi/job.json b/qapi/job.json +index 69c1970..17d1003 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -17,7 +17,7 @@ + # + # @backup: drive backup job type, see "drive-backup" + # +-# @create: image creation job type, see "x-blockdev-create" (since 3.0) ++# @create: image creation job type, see "blockdev-create" (since 3.0) + # + # Since: 1.7 + ## +diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206 +index b8cf2e7..128c334 100755 +--- a/tests/qemu-iotests/206 ++++ b/tests/qemu-iotests/206 +@@ -26,7 +26,7 @@ from iotests import imgfmt + iotests.verify_image_format(supported_fmts=['qcow2']) + + def blockdev_create(vm, options): +- result = vm.qmp_log('x-blockdev-create', job_id='job0', options=options) ++ result = vm.qmp_log('blockdev-create', job_id='job0', options=options) + + if 'return' in result: + assert result['return'] == {} +diff --git a/tests/qemu-iotests/206.out b/tests/qemu-iotests/206.out +index 34451a3..789eebe 100644 +--- a/tests/qemu-iotests/206.out ++++ b/tests/qemu-iotests/206.out +@@ -1,13 +1,13 @@ + === Successful image creation (defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + {'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}} + {u'return': {}} +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'imgfile', 'size': 134217728}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'imgfile', 'size': 134217728}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -24,12 +24,12 @@ Format specific information: + + === Successful image creation (inline blockdev-add, explicit defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'nocow': False, 'preallocation': 'off', 'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'nocow': False, 'preallocation': 'off', 'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'refcount-bits': 16, 'version': 'v3', 'preallocation': 'off', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': False, 'driver': 'qcow2', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'refcount-bits': 16, 'version': 'v3', 'preallocation': 'off', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': False, 'driver': 'qcow2', 'size': 67108864}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -46,12 +46,12 @@ Format specific information: + + === Successful image creation (v3 non-default options) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'nocow': True, 'preallocation': 'falloc', 'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'nocow': True, 'preallocation': 'falloc', 'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 2097152, 'refcount-bits': 1, 'version': 'v3', 'preallocation': 'metadata', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': True, 'driver': 'qcow2', 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 2097152, 'refcount-bits': 1, 'version': 'v3', 'preallocation': 'metadata', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'lazy-refcounts': True, 'driver': 'qcow2', 'size': 33554432}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -68,12 +68,12 @@ Format specific information: + + === Successful image creation (v2 non-default options) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'backing-fmt': 'qcow2', 'driver': 'qcow2', 'version': 'v2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'backing-file': 'TEST_DIR/PID-t.qcow2.base', 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'backing-fmt': 'qcow2', 'driver': 'qcow2', 'version': 'v2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'backing-file': 'TEST_DIR/PID-t.qcow2.base', 'size': 33554432}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -90,7 +90,7 @@ Format specific information: + + === Successful image creation (encrypted) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'encrypt': {'key-secret': 'keysec0', 'iter-time': 10, 'cipher-mode': 'ctr', 'ivgen-hash-alg': 'md5', 'cipher-alg': 'twofish-128', 'format': 'luks', 'ivgen-alg': 'plain64', 'hash-alg': 'sha1'}, 'driver': 'qcow2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'encrypt': {'key-secret': 'keysec0', 'iter-time': 10, 'cipher-mode': 'ctr', 'ivgen-hash-alg': 'md5', 'cipher-alg': 'twofish-128', 'format': 'luks', 'ivgen-alg': 'plain64', 'hash-alg': 'sha1'}, 'driver': 'qcow2', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.qcow2'}, 'size': 33554432}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -144,111 +144,111 @@ Format specific information: + + === Invalid BlockdevRef === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': "this doesn't exist", 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': "this doesn't exist", 'size': 33554432}}} + {u'return': {}} + Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + === Invalid sizes === +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 1234}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 1234}}} + {u'return': {}} + Job failed: Image size must be a multiple of 512 bytes + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 18446744073709551104L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 18446744073709551104L}}} + {u'return': {}} + Job failed: Could not resize image: Image size cannot be negative + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775808L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775808L}}} + {u'return': {}} + Job failed: Could not resize image: Image size cannot be negative + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775296}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'qcow2', 'file': 'node0', 'size': 9223372036854775296}}} + {u'return': {}} + Job failed: Could not resize image: Failed to grow the L1 table: File too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + === Invalid version === +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'version': 'v1', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'version': 'v1', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'error': {u'class': u'GenericError', u'desc': u"Invalid parameter 'v1'"}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'lazy-refcounts': True, 'version': 'v2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'lazy-refcounts': True, 'version': 'v2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Lazy refcounts only supported with compatibility level 1.1 and above (use version=v3 or greater) + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 8, 'version': 'v2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 8, 'version': 'v2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Different refcount widths than 16 bits require compatibility level 1.1 or above (use version=v3 or greater) + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + === Invalid backing file options === +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'full', 'driver': 'qcow2', 'backing-file': '/dev/null', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'full', 'driver': 'qcow2', 'backing-file': '/dev/null', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Backing file and preallocation cannot be used at the same time + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'backing-fmt': 'qcow2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'backing-fmt': 'qcow2', 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Backing format cannot be used without backing file + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + === Invalid cluster size === +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1234, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1234, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size must be a power of two between 512 and 2048k + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 128, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 128, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size must be a power of two between 512 and 2048k + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 4194304, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 4194304, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size must be a power of two between 512 and 2048k + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 0, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 0, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size must be a power of two between 512 and 2048k + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'driver': 'qcow2', 'file': 'node0', 'size': 281474976710656}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'driver': 'qcow2', 'file': 'node0', 'size': 281474976710656}}} + {u'return': {}} + Job failed: Could not resize image: Failed to grow the L1 table: File too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + === Invalid refcount width === +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 128, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 128, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Refcount width must be a power of two and may not exceed 64 bits + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 0, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 0, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Refcount width must be a power of two and may not exceed 64 bits + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 7, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'refcount-bits': 7, 'driver': 'qcow2', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Refcount width must be a power of two and may not exceed 64 bits + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207 +index b595c92..444ae23 100755 +--- a/tests/qemu-iotests/207 ++++ b/tests/qemu-iotests/207 +@@ -31,7 +31,7 @@ def filter_hash(msg): + return re.sub("'hash': '[0-9a-f]+'", "'hash': HASH", msg) + + def blockdev_create(vm, options): +- result = vm.qmp_log('x-blockdev-create', job_id='job0', options=options, ++ result = vm.qmp_log('blockdev-create', job_id='job0', options=options, + filters=[iotests.filter_testfiles, filter_hash]) + + if 'return' in result: +diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out +index 5eee17b..078b7e6 100644 +--- a/tests/qemu-iotests/207.out ++++ b/tests/qemu-iotests/207.out +@@ -1,6 +1,6 @@ + === Successful image creation (defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -16,7 +16,7 @@ virtual size: 4.0M (4194304 bytes) + + === Test host-key-check options === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -25,7 +25,7 @@ image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.po + file format: IMGFMT + virtual size: 8.0M (8388608 bytes) + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'known_hosts'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'known_hosts'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -34,13 +34,13 @@ image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.po + file format: IMGFMT + virtual size: 4.0M (4194304 bytes) + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} + {u'return': {}} + Job failed: remote host key does not match host_key_check 'wrong' + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -49,13 +49,13 @@ image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.po + file format: IMGFMT + virtual size: 8.0M (8388608 bytes) + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} + {u'return': {}} + Job failed: remote host key does not match host_key_check 'wrong' + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -66,13 +66,13 @@ virtual size: 4.0M (4194304 bytes) + + === Invalid path and user === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': '/this/is/not/an/existing/path', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': '/this/is/not/an/existing/path', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} + {u'return': {}} + Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31) + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'user': 'invalid user', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'user': 'invalid user', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} + {u'return': {}} + Job failed: failed to authenticate using publickey authentication and the identities held by your ssh-agent + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210 +index ff4fdde..d142841 100755 +--- a/tests/qemu-iotests/210 ++++ b/tests/qemu-iotests/210 +@@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=['luks']) + iotests.verify_protocol(supported=['file']) + + def blockdev_create(vm, options): +- result = vm.qmp_log('x-blockdev-create', job_id='job0', options=options) ++ result = vm.qmp_log('blockdev-create', job_id='job0', options=options) + + if 'return' in result: + assert result['return'] == {} +diff --git a/tests/qemu-iotests/210.out b/tests/qemu-iotests/210.out +index 0e6e5c0..078ba54 100644 +--- a/tests/qemu-iotests/210.out ++++ b/tests/qemu-iotests/210.out +@@ -1,13 +1,13 @@ + === Successful image creation (defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + {'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}} + {u'return': {}} +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'iter-time': 10, 'driver': 'luks', 'file': 'imgfile', 'size': 134217728}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'iter-time': 10, 'driver': 'luks', 'file': 'imgfile', 'size': 134217728}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -54,12 +54,12 @@ Format specific information: + + === Successful image creation (with non-default options) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'hash-alg': 'sha1', 'cipher-mode': 'ctr', 'cipher-alg': 'twofish-128', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}, 'iter-time': 10, 'ivgen-alg': 'plain64', 'ivgen-hash-alg': 'md5', 'driver': 'luks', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'hash-alg': 'sha1', 'cipher-mode': 'ctr', 'cipher-alg': 'twofish-128', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.luks'}, 'iter-time': 10, 'ivgen-alg': 'plain64', 'ivgen-hash-alg': 'md5', 'driver': 'luks', 'size': 67108864}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -106,7 +106,7 @@ Format specific information: + + === Invalid BlockdevRef === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': "this doesn't exist", 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'luks', 'file': "this doesn't exist", 'size': 67108864}}} + {u'return': {}} + Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -114,7 +114,7 @@ Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exi + + === Zero size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'iter-time': 10, 'driver': 'luks', 'file': 'node0', 'size': 0}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'iter-time': 10, 'driver': 'luks', 'file': 'node0', 'size': 0}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -161,19 +161,19 @@ Format specific information: + + === Invalid sizes === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 18446744073709551104L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 18446744073709551104L}}} + {u'return': {}} + Job failed: The requested file size is too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 9223372036854775808L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 9223372036854775808L}}} + {u'return': {}} + Job failed: The requested file size is too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 9223372036854775296}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'key-secret': 'keysec0', 'driver': 'luks', 'file': 'node0', 'size': 9223372036854775296}}} + {u'return': {}} + Job failed: The requested file size is too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +diff --git a/tests/qemu-iotests/211 b/tests/qemu-iotests/211 +index b45f886..7b7985d 100755 +--- a/tests/qemu-iotests/211 ++++ b/tests/qemu-iotests/211 +@@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=['vdi']) + iotests.verify_protocol(supported=['file']) + + def blockdev_create(vm, options): +- result = vm.qmp_log('x-blockdev-create', job_id='job0', options=options) ++ result = vm.qmp_log('blockdev-create', job_id='job0', options=options) + + if 'return' in result: + assert result['return'] == {} +diff --git a/tests/qemu-iotests/211.out b/tests/qemu-iotests/211.out +index 2bf1c4a..6feaea3 100644 +--- a/tests/qemu-iotests/211.out ++++ b/tests/qemu-iotests/211.out +@@ -1,13 +1,13 @@ + === Successful image creation (defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + {'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}} + {u'return': {}} +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'imgfile', 'size': 134217728}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'imgfile', 'size': 134217728}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -21,12 +21,12 @@ cluster_size: 1048576 + + === Successful image creation (explicit defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'off', 'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'off', 'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'size': 67108864}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -40,12 +40,12 @@ cluster_size: 1048576 + + === Successful image creation (with non-default options) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'metadata', 'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'preallocation': 'metadata', 'driver': 'vdi', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vdi'}, 'size': 33554432}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -60,7 +60,7 @@ cluster_size: 1048576 + + === Invalid BlockdevRef === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': "this doesn't exist", 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': "this doesn't exist", 'size': 33554432}}} + {u'return': {}} + Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -68,7 +68,7 @@ Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exi + + === Zero size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 0}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 0}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -80,7 +80,7 @@ cluster_size: 1048576 + + === Maximum size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 562949819203584}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 562949819203584}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -92,19 +92,19 @@ cluster_size: 1048576 + + === Invalid sizes === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 18446744073709551104L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 18446744073709551104L}}} + {u'return': {}} + Job failed: Unsupported VDI image size (size is 0xfffffffffffffe00, max supported is 0x1fffff8000000) + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 9223372036854775808L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 9223372036854775808L}}} + {u'return': {}} + Job failed: Unsupported VDI image size (size is 0x8000000000000000, max supported is 0x1fffff8000000) + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 562949819203585}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vdi', 'file': 'node0', 'size': 562949819203585}}} + {u'return': {}} + Job failed: Unsupported VDI image size (size is 0x1fffff8000001, max supported is 0x1fffff8000000) + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +diff --git a/tests/qemu-iotests/212 b/tests/qemu-iotests/212 +index 03cf41d..95c8810 100755 +--- a/tests/qemu-iotests/212 ++++ b/tests/qemu-iotests/212 +@@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=['parallels']) + iotests.verify_protocol(supported=['file']) + + def blockdev_create(vm, options): +- result = vm.qmp_log('x-blockdev-create', job_id='job0', options=options) ++ result = vm.qmp_log('blockdev-create', job_id='job0', options=options) + + if 'return' in result: + assert result['return'] == {} +diff --git a/tests/qemu-iotests/212.out b/tests/qemu-iotests/212.out +index 780bc30..9150da7 100644 +--- a/tests/qemu-iotests/212.out ++++ b/tests/qemu-iotests/212.out +@@ -1,13 +1,13 @@ + === Successful image creation (defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + {'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}} + {u'return': {}} +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'imgfile', 'size': 134217728}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'imgfile', 'size': 134217728}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -18,12 +18,12 @@ virtual size: 128M (134217728 bytes) + + === Successful image creation (explicit defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1048576, 'driver': 'parallels', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}, 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1048576, 'driver': 'parallels', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}, 'size': 67108864}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -34,12 +34,12 @@ virtual size: 64M (67108864 bytes) + + === Successful image creation (with non-default options) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'driver': 'parallels', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}, 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 65536, 'driver': 'parallels', 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.parallels'}, 'size': 33554432}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -50,7 +50,7 @@ virtual size: 32M (33554432 bytes) + + === Invalid BlockdevRef === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': "this doesn't exist", 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': "this doesn't exist", 'size': 33554432}}} + {u'return': {}} + Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -58,7 +58,7 @@ Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exi + + === Zero size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 0}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 0}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -69,7 +69,7 @@ virtual size: 0 (0 bytes) + + === Maximum size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 4503599627369984}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 4503599627369984}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -80,31 +80,31 @@ virtual size: 4096T (4503599627369984 bytes) + + === Invalid sizes === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 1234}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 1234}}} + {u'return': {}} + Job failed: Image size must be a multiple of 512 bytes + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 18446744073709551104L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 18446744073709551104L}}} + {u'return': {}} + Job failed: Image size is too large for this cluster size + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775808L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775808L}}} + {u'return': {}} + Job failed: Image size is too large for this cluster size + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775296}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 9223372036854775296}}} + {u'return': {}} + Job failed: Image size is too large for this cluster size + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 4503599627370497}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'parallels', 'file': 'node0', 'size': 4503599627370497}}} + {u'return': {}} + Job failed: Image size is too large for this cluster size + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -112,43 +112,43 @@ Job failed: Image size is too large for this cluster size + + === Invalid cluster size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1234, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 1234, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size must be a multiple of 512 bytes + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 128, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 128, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size must be a multiple of 512 bytes + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 4294967296, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 4294967296, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size is too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 9223372036854775808L, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 9223372036854775808L, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size is too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 18446744073709551104L, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 18446744073709551104L, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Cluster size is too large + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 0, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 0, 'driver': 'parallels', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Image size is too large for this cluster size + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'driver': 'parallels', 'file': 'node0', 'size': 281474976710656}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'cluster-size': 512, 'driver': 'parallels', 'file': 'node0', 'size': 281474976710656}}} + {u'return': {}} + Job failed: Image size is too large for this cluster size + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +diff --git a/tests/qemu-iotests/213 b/tests/qemu-iotests/213 +index 29d25bc..4054439 100755 +--- a/tests/qemu-iotests/213 ++++ b/tests/qemu-iotests/213 +@@ -27,7 +27,7 @@ iotests.verify_image_format(supported_fmts=['vhdx']) + iotests.verify_protocol(supported=['file']) + + def blockdev_create(vm, options): +- result = vm.qmp_log('x-blockdev-create', job_id='job0', options=options) ++ result = vm.qmp_log('blockdev-create', job_id='job0', options=options) + + if 'return' in result: + assert result['return'] == {} +diff --git a/tests/qemu-iotests/213.out b/tests/qemu-iotests/213.out +index f18a39a..e1dcd47 100644 +--- a/tests/qemu-iotests/213.out ++++ b/tests/qemu-iotests/213.out +@@ -1,13 +1,13 @@ + === Successful image creation (defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + + {'execute': 'blockdev-add', 'arguments': {'node_name': 'imgfile', 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}} + {u'return': {}} +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'imgfile', 'size': 134217728}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'imgfile', 'size': 134217728}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -19,12 +19,12 @@ cluster_size: 8388608 + + === Successful image creation (explicit defaults) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 8388608, 'driver': 'vhdx', 'subformat': 'dynamic', 'log-size': 1048576, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'block-state-zero': True, 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 8388608, 'driver': 'vhdx', 'subformat': 'dynamic', 'log-size': 1048576, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'block-state-zero': True, 'size': 67108864}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -36,12 +36,12 @@ cluster_size: 8388608 + + === Successful image creation (with non-default options) === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'size': 0, 'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 268435456, 'driver': 'vhdx', 'subformat': 'fixed', 'log-size': 8388608, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'block-state-zero': False, 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'block-size': 268435456, 'driver': 'vhdx', 'subformat': 'fixed', 'log-size': 8388608, 'file': {'driver': 'file', 'filename': 'TEST_DIR/PID-t.vhdx'}, 'block-state-zero': False, 'size': 33554432}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -53,7 +53,7 @@ cluster_size: 268435456 + + === Invalid BlockdevRef === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': "this doesn't exist", 'size': 33554432}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': "this doesn't exist", 'size': 33554432}}} + {u'return': {}} + Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exist + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -61,7 +61,7 @@ Job failed: Cannot find device=this doesn't exist nor node_name=this doesn't exi + + === Zero size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 0}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 0}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -73,7 +73,7 @@ cluster_size: 8388608 + + === Maximum size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 70368744177664}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 70368744177664}}} + {u'return': {}} + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} +@@ -85,25 +85,25 @@ cluster_size: 67108864 + + === Invalid sizes === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 18446744073709551104L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 18446744073709551104L}}} + {u'return': {}} + Job failed: Image size too large; max of 64TB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775808L}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775808L}}} + {u'return': {}} + Job failed: Image size too large; max of 64TB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775296}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 9223372036854775296}}} + {u'return': {}} + Job failed: Image size too large; max of 64TB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 70368744177665}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'file': 'node0', 'size': 70368744177665}}} + {u'return': {}} + Job failed: Image size too large; max of 64TB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -111,31 +111,31 @@ Job failed: Image size too large; max of 64TB + + === Invalid block size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 1234567, 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 1234567, 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Block size must be a multiple of 1 MB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 128, 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 128, 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Block size must be a multiple of 1 MB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 3145728, 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 3145728, 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Block size must be a power of two + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 536870912, 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 536870912, 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Block size must not exceed 268435456 + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 0, 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'vhdx', 'block-size': 0, 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Block size must be a multiple of 1 MB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +@@ -143,25 +143,25 @@ Job failed: Block size must be a multiple of 1 MB + + === Invalid log size === + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 1234567, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 1234567, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Log size must be a multiple of 1 MB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 128, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 128, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Log size must be a multiple of 1 MB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 4294967296, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 4294967296, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Log size must be smaller than 4 GB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} + {u'return': {}} + +-{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 0, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} ++{'execute': 'blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'log-size': 0, 'driver': 'vhdx', 'file': 'node0', 'size': 67108864}}} + {u'return': {}} + Job failed: Log size must be a multiple of 1 MB + {'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmap-Add-bdrv_dirty_iter_next_area.patch b/SOURCES/kvm-block-dirty-bitmap-Add-bdrv_dirty_iter_next_area.patch new file mode 100644 index 0000000..1953f04 --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmap-Add-bdrv_dirty_iter_next_area.patch @@ -0,0 +1,114 @@ +From 36323c9384909d2213fafb77b1fcf0ddbfcaaffc Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:09 +0000 +Subject: [PATCH 15/35] block/dirty-bitmap: Add bdrv_dirty_iter_next_area + +RH-Author: John Snow +Message-id: <20181120181828.15132-6-jsnow@redhat.com> +Patchwork-id: 83054 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 05/24] block/dirty-bitmap: Add bdrv_dirty_iter_next_area +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Max Reitz + +This new function allows to look for a consecutively dirty area in a +dirty bitmap. + +Signed-off-by: Max Reitz +Reviewed-by: Fam Zheng +Reviewed-by: John Snow +Message-id: 20180613181823.13618-10-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 72d10a94213a954ad569095cb4491f2ae0853c40) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 55 ++++++++++++++++++++++++++++++++++++++++++++ + include/block/dirty-bitmap.h | 2 ++ + 2 files changed, 57 insertions(+) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 236dce1..c9b8a6f 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -528,6 +528,61 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) + return hbitmap_iter_next(&iter->hbi, true); + } + ++/** ++ * Return the next consecutively dirty area in the dirty bitmap ++ * belonging to the given iterator @iter. ++ * ++ * @max_offset: Maximum value that may be returned for ++ * *offset + *bytes ++ * @offset: Will contain the start offset of the next dirty area ++ * @bytes: Will contain the length of the next dirty area ++ * ++ * Returns: True if a dirty area could be found before max_offset ++ * (which means that *offset and *bytes then contain valid ++ * values), false otherwise. ++ * ++ * Note that @iter is never advanced if false is returned. If an area ++ * is found (which means that true is returned), it will be advanced ++ * past that area. ++ */ ++bool bdrv_dirty_iter_next_area(BdrvDirtyBitmapIter *iter, uint64_t max_offset, ++ uint64_t *offset, int *bytes) ++{ ++ uint32_t granularity = bdrv_dirty_bitmap_granularity(iter->bitmap); ++ uint64_t gran_max_offset; ++ int64_t ret; ++ int size; ++ ++ if (max_offset == iter->bitmap->size) { ++ /* If max_offset points to the image end, round it up by the ++ * bitmap granularity */ ++ gran_max_offset = ROUND_UP(max_offset, granularity); ++ } else { ++ gran_max_offset = max_offset; ++ } ++ ++ ret = hbitmap_iter_next(&iter->hbi, false); ++ if (ret < 0 || ret + granularity > gran_max_offset) { ++ return false; ++ } ++ ++ *offset = ret; ++ size = 0; ++ ++ assert(granularity <= INT_MAX); ++ ++ do { ++ /* Advance iterator */ ++ ret = hbitmap_iter_next(&iter->hbi, true); ++ size += granularity; ++ } while (ret + granularity <= gran_max_offset && ++ hbitmap_iter_next(&iter->hbi, false) == ret + granularity && ++ size <= INT_MAX - granularity); ++ ++ *bytes = MIN(size, max_offset - *offset); ++ return true; ++} ++ + /* Called within bdrv_dirty_bitmap_lock..unlock */ + void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, + int64_t offset, int64_t bytes) +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index bf68dd7..259bd27 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -83,6 +83,8 @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, + void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap, + int64_t offset, int64_t bytes); + int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter); ++bool bdrv_dirty_iter_next_area(BdrvDirtyBitmapIter *iter, uint64_t max_offset, ++ uint64_t *offset, int *bytes); + void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t offset); + int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap); + int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmap-add-bdrv_enable_dirty_bitmap_lock.patch b/SOURCES/kvm-block-dirty-bitmap-add-bdrv_enable_dirty_bitmap_lock.patch new file mode 100644 index 0000000..8e3adda --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmap-add-bdrv_enable_dirty_bitmap_lock.patch @@ -0,0 +1,73 @@ +From 0f3b55a758bc4e5a4fd864d5b650557407278750 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:42 +0200 +Subject: [PATCH 224/268] block/dirty-bitmap: add + bdrv_enable_dirty_bitmap_locked + +RH-Author: John Snow +Message-id: <20180718225511.14878-7-jsnow@redhat.com> +Patchwork-id: 81405 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 06/35] block/dirty-bitmap: add bdrv_enable_dirty_bitmap_locked +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Add _locked version of bdrv_enable_dirty_bitmap, to fix dirty bitmap +migration in the following patch. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-id: 20180625165745.25259-2-vsementsov@virtuozzo.com +Signed-off-by: John Snow +(cherry picked from commit 92bcea40d3aac62853e60426bd109b748d4d1cd2) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/dirty-bitmap.c | 9 +++++++-- + include/block/dirty-bitmap.h | 1 + + 2 files changed, 8 insertions(+), 2 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 4159d39..4d6ae8b 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -250,6 +250,12 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs, + return 0; + } + ++void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap) ++{ ++ assert(!bdrv_dirty_bitmap_frozen(bitmap)); ++ bitmap->disabled = false; ++} ++ + /* Called with BQL taken. */ + void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap) + { +@@ -453,8 +459,7 @@ void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap) + void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap) + { + bdrv_dirty_bitmap_lock(bitmap); +- assert(!bdrv_dirty_bitmap_frozen(bitmap)); +- bitmap->disabled = false; ++ bdrv_enable_dirty_bitmap_locked(bitmap); + bdrv_dirty_bitmap_unlock(bitmap); + } + +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index 1e14743..af9ba3c 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -33,6 +33,7 @@ void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + Error **errp); + void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap); + void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap); ++void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap); + BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs); + uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs); + uint32_t bdrv_dirty_bitmap_granularity(const BdrvDirtyBitmap *bitmap); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmap-add-lock-to-bdrv_enable-disable_d.patch b/SOURCES/kvm-block-dirty-bitmap-add-lock-to-bdrv_enable-disable_d.patch new file mode 100644 index 0000000..c1080af --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmap-add-lock-to-bdrv_enable-disable_d.patch @@ -0,0 +1,62 @@ +From 7879107e37dd742ffeff6f943671ae6c49c75876 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:37 +0200 +Subject: [PATCH 219/268] block/dirty-bitmap: add lock to + bdrv_enable/disable_dirty_bitmap + +RH-Author: John Snow +Message-id: <20180718225511.14878-2-jsnow@redhat.com> +Patchwork-id: 81426 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 01/35] block/dirty-bitmap: add lock to bdrv_enable/disable_dirty_bitmap +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Add locks and remove comments about BQL accordingly to +dirty_bitmap_mutex definition in block_int.h. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Message-id: 20180606182449.1607-2-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 8b1402ce80d74dc02802f101a0f6c340462380d1) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/dirty-bitmap.c | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 9671594..5623425 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -442,18 +442,20 @@ void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + } + } + +-/* Called with BQL taken. */ + void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap) + { ++ bdrv_dirty_bitmap_lock(bitmap); + assert(!bdrv_dirty_bitmap_frozen(bitmap)); + bitmap->disabled = true; ++ bdrv_dirty_bitmap_unlock(bitmap); + } + +-/* Called with BQL taken. */ + void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap) + { ++ bdrv_dirty_bitmap_lock(bitmap); + assert(!bdrv_dirty_bitmap_frozen(bitmap)); + bitmap->disabled = false; ++ bdrv_dirty_bitmap_unlock(bitmap); + } + + BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmaps-add-user_locked-status-checker.patch b/SOURCES/kvm-block-dirty-bitmaps-add-user_locked-status-checker.patch new file mode 100644 index 0000000..5193cff --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmaps-add-user_locked-status-checker.patch @@ -0,0 +1,148 @@ +From e1a7b82bc7f0dc8f65c9bf07acbcdca1fb08bb17 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:16 +0000 +Subject: [PATCH 22/35] block/dirty-bitmaps: add user_locked status checker + +RH-Author: John Snow +Message-id: <20181120181828.15132-13-jsnow@redhat.com> +Patchwork-id: 83056 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 12/24] block/dirty-bitmaps: add user_locked status checker +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +Instead of both frozen and qmp_locked checks, wrap it into one check. +frozen implies the bitmap is split in two (for backup), and shouldn't +be modified. qmp_locked implies it's being used by another operation, +like being exported over NBD. In both cases it means we shouldn't allow +the user to modify it in any meaningful way. + +Replace any usages where we check both frozen and qmp_locked with the +new check. + +Signed-off-by: John Snow +Reviewed-by: Eric Blake +Message-id: 20181002230218.13949-2-jsnow@redhat.com +[w/edits Suggested-By: Vladimir Sementsov-Ogievskiy ] +Signed-off-by: John Snow +(cherry picked from commit 993edc0ce0c6f44deb8272a7a857e419417f5f84) +Signed-off-by: John Snow + +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 6 ++++++ + blockdev.c | 29 ++++++++--------------------- + include/block/dirty-bitmap.h | 1 + + migration/block-dirty-bitmap.c | 10 ++-------- + 4 files changed, 17 insertions(+), 29 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 8ac933c..9603cdd 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -176,6 +176,12 @@ bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap) + return bitmap->successor; + } + ++/* Both conditions disallow user-modification via QMP. */ ++bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) { ++ return bdrv_dirty_bitmap_frozen(bitmap) || ++ bdrv_dirty_bitmap_qmp_locked(bitmap); ++} ++ + void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked) + { + qemu_mutex_lock(bitmap->mutex); +diff --git a/blockdev.c b/blockdev.c +index d2e7e5a..5cdb608 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2119,11 +2119,8 @@ static void block_dirty_bitmap_clear_prepare(BlkActionState *common, + return; + } + +- if (bdrv_dirty_bitmap_frozen(state->bitmap)) { +- error_setg(errp, "Cannot modify a frozen bitmap"); +- return; +- } else if (bdrv_dirty_bitmap_qmp_locked(state->bitmap)) { +- error_setg(errp, "Cannot modify a locked bitmap"); ++ if (bdrv_dirty_bitmap_user_locked(state->bitmap)) { ++ error_setg(errp, "Cannot modify a bitmap in use by another operation"); + return; + } else if (!bdrv_dirty_bitmap_enabled(state->bitmap)) { + error_setg(errp, "Cannot clear a disabled bitmap"); +@@ -2992,15 +2989,10 @@ void qmp_block_dirty_bitmap_remove(const char *node, const char *name, + return; + } + +- if (bdrv_dirty_bitmap_frozen(bitmap)) { +- error_setg(errp, +- "Bitmap '%s' is currently frozen and cannot be removed", +- name); +- return; +- } else if (bdrv_dirty_bitmap_qmp_locked(bitmap)) { ++ if (bdrv_dirty_bitmap_user_locked(bitmap)) { + error_setg(errp, +- "Bitmap '%s' is currently locked and cannot be removed", +- name); ++ "Bitmap '%s' is currently in use by another operation and" ++ " cannot be removed", name); + return; + } + +@@ -3030,15 +3022,10 @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name, + return; + } + +- if (bdrv_dirty_bitmap_frozen(bitmap)) { ++ if (bdrv_dirty_bitmap_user_locked(bitmap)) { + error_setg(errp, +- "Bitmap '%s' is currently frozen and cannot be modified", +- name); +- return; +- } else if (bdrv_dirty_bitmap_qmp_locked(bitmap)) { +- error_setg(errp, +- "Bitmap '%s' is currently locked and cannot be modified", +- name); ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be cleared", name); + return; + } else if (!bdrv_dirty_bitmap_enabled(bitmap)) { + error_setg(errp, +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index 201ff7f..1463943 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -94,6 +94,7 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs); + bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap); + bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap); + bool bdrv_dirty_bitmap_qmp_locked(BdrvDirtyBitmap *bitmap); ++bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap); + bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs); + BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs, + BdrvDirtyBitmap *bitmap); +diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c +index fefbc6a..47251af 100644 +--- a/migration/block-dirty-bitmap.c ++++ b/migration/block-dirty-bitmap.c +@@ -301,14 +301,8 @@ static int init_dirty_bitmap_migration(void) + goto fail; + } + +- if (bdrv_dirty_bitmap_frozen(bitmap)) { +- error_report("Can't migrate frozen dirty bitmap: '%s", +- bdrv_dirty_bitmap_name(bitmap)); +- goto fail; +- } +- +- if (bdrv_dirty_bitmap_qmp_locked(bitmap)) { +- error_report("Can't migrate locked dirty bitmap: '%s", ++ if (bdrv_dirty_bitmap_user_locked(bitmap)) { ++ error_report("Can't migrate a bitmap that is in use by another operation: '%s'", + bdrv_dirty_bitmap_name(bitmap)); + goto fail; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmaps-allow-clear-on-disabled-bitmaps.patch b/SOURCES/kvm-block-dirty-bitmaps-allow-clear-on-disabled-bitmaps.patch new file mode 100644 index 0000000..5e71bc0 --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmaps-allow-clear-on-disabled-bitmaps.patch @@ -0,0 +1,72 @@ +From 461fe4b2f01c316ce2ca63fa81624fd09273dc02 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:18 +0000 +Subject: [PATCH 24/35] block/dirty-bitmaps: allow clear on disabled bitmaps + +RH-Author: John Snow +Message-id: <20181120181828.15132-15-jsnow@redhat.com> +Patchwork-id: 83062 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 14/24] block/dirty-bitmaps: allow clear on disabled bitmaps +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +Similarly to merge, it's OK to allow clear operations on disabled +bitmaps, as this condition only means that they are not recording +new writes. We are free to clear it if the user requests it. + +Signed-off-by: John Snow +Reviewed-by: Eric Blake +Reviewed-by: Vladimir Sementsov-Ogievskiy +Message-id: 20181002230218.13949-4-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 0be37c9e19f541643ef407bdafe0282b667ec23c) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 1 - + blockdev.c | 8 -------- + 2 files changed, 9 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index bfccb0e..9b9ebd7 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -625,7 +625,6 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, + + void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) + { +- assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); + bdrv_dirty_bitmap_lock(bitmap); + if (!out) { +diff --git a/blockdev.c b/blockdev.c +index 5cdb608..220b317 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2122,9 +2122,6 @@ static void block_dirty_bitmap_clear_prepare(BlkActionState *common, + if (bdrv_dirty_bitmap_user_locked(state->bitmap)) { + error_setg(errp, "Cannot modify a bitmap in use by another operation"); + return; +- } else if (!bdrv_dirty_bitmap_enabled(state->bitmap)) { +- error_setg(errp, "Cannot clear a disabled bitmap"); +- return; + } else if (bdrv_dirty_bitmap_readonly(state->bitmap)) { + error_setg(errp, "Cannot clear a readonly bitmap"); + return; +@@ -3027,11 +3024,6 @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name, + "Bitmap '%s' is currently in use by another operation" + " and cannot be cleared", name); + return; +- } else if (!bdrv_dirty_bitmap_enabled(bitmap)) { +- error_setg(errp, +- "Bitmap '%s' is currently disabled and cannot be cleared", +- name); +- return; + } else if (bdrv_dirty_bitmap_readonly(bitmap)) { + error_setg(errp, "Bitmap '%s' is readonly and cannot be cleared", name); + return; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmaps-fix-merge-permissions.patch b/SOURCES/kvm-block-dirty-bitmaps-fix-merge-permissions.patch new file mode 100644 index 0000000..fb6f17c --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmaps-fix-merge-permissions.patch @@ -0,0 +1,53 @@ +From c7289a21c999e767a12e4c8daff7e498167a3859 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:17 +0000 +Subject: [PATCH 23/35] block/dirty-bitmaps: fix merge permissions + +RH-Author: John Snow +Message-id: <20181120181828.15132-14-jsnow@redhat.com> +Patchwork-id: 83069 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 13/24] block/dirty-bitmaps: fix merge permissions +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +In prior commits that made merge transactionable, we removed the +assertion that merge cannot operate on disabled bitmaps. In addition, +we want to make sure that we are prohibiting merges to "locked" bitmaps. + +Use the new user_locked function to check. + +Reported-by: Eric Blake +Signed-off-by: John Snow +Reviewed-by: Eric Blake +Reviewed-by: Vladimir Sementsov-Ogievskiy +Message-id: 20181002230218.13949-3-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 283d7a04f2addcc51468635300208b60c19a0db3) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 9603cdd..bfccb0e 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -806,9 +806,9 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, + + qemu_mutex_lock(dest->mutex); + +- if (bdrv_dirty_bitmap_frozen(dest)) { +- error_setg(errp, "Bitmap '%s' is frozen and cannot be modified", +- dest->name); ++ if (bdrv_dirty_bitmap_user_locked(dest)) { ++ error_setg(errp, "Bitmap '%s' is currently in use by another" ++ " operation and cannot be modified", dest->name); + goto out; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-dirty-bitmaps-prohibit-enable-disable-on-locke.patch b/SOURCES/kvm-block-dirty-bitmaps-prohibit-enable-disable-on-locke.patch new file mode 100644 index 0000000..258f4bb --- /dev/null +++ b/SOURCES/kvm-block-dirty-bitmaps-prohibit-enable-disable-on-locke.patch @@ -0,0 +1,93 @@ +From c5dddad12032351514c74083854393390ebd64e2 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:19 +0000 +Subject: [PATCH 25/35] block/dirty-bitmaps: prohibit enable/disable on + locked/frozen bitmaps + +RH-Author: John Snow +Message-id: <20181120181828.15132-16-jsnow@redhat.com> +Patchwork-id: 83059 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 15/24] block/dirty-bitmaps: prohibit enable/disable on locked/frozen bitmaps +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +We're not being consistent about this. If it's in use by an operation, +the user should not be able to change the behavior of that bitmap. + +Signed-off-by: John Snow +Reviewed-by: Eric Blake +Reviewed-by: Vladimir Sementsov-Ogievskiy +Message-id: 20181002230218.13949-5-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit b053bb55738f35832f3d6472b12277a75c32a038) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 26 ++++++++++++++++++++------ + 1 file changed, 20 insertions(+), 6 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index 220b317..916153e 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2168,6 +2168,13 @@ static void block_dirty_bitmap_enable_prepare(BlkActionState *common, + return; + } + ++ if (bdrv_dirty_bitmap_user_locked(state->bitmap)) { ++ error_setg(errp, ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be enabled", action->name); ++ return; ++ } ++ + state->was_enabled = bdrv_dirty_bitmap_enabled(state->bitmap); + bdrv_enable_dirty_bitmap(state->bitmap); + } +@@ -2202,6 +2209,13 @@ static void block_dirty_bitmap_disable_prepare(BlkActionState *common, + return; + } + ++ if (bdrv_dirty_bitmap_user_locked(state->bitmap)) { ++ error_setg(errp, ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be disabled", action->name); ++ return; ++ } ++ + state->was_enabled = bdrv_dirty_bitmap_enabled(state->bitmap); + bdrv_disable_dirty_bitmap(state->bitmap); + } +@@ -3043,10 +3057,10 @@ void qmp_x_block_dirty_bitmap_enable(const char *node, const char *name, + return; + } + +- if (bdrv_dirty_bitmap_frozen(bitmap)) { ++ if (bdrv_dirty_bitmap_user_locked(bitmap)) { + error_setg(errp, +- "Bitmap '%s' is currently frozen and cannot be enabled", +- name); ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be enabled", name); + return; + } + +@@ -3064,10 +3078,10 @@ void qmp_x_block_dirty_bitmap_disable(const char *node, const char *name, + return; + } + +- if (bdrv_dirty_bitmap_frozen(bitmap)) { ++ if (bdrv_dirty_bitmap_user_locked(bitmap)) { + error_setg(errp, +- "Bitmap '%s' is currently frozen and cannot be disabled", +- name); ++ "Bitmap '%s' is currently in use by another operation" ++ " and cannot be disabled", name); + return; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-file-posix-File-locking-during-creation.patch b/SOURCES/kvm-block-file-posix-File-locking-during-creation.patch new file mode 100644 index 0000000..58f8741 --- /dev/null +++ b/SOURCES/kvm-block-file-posix-File-locking-during-creation.patch @@ -0,0 +1,102 @@ +From f815f89794a322fd3d0c7275ecdebce5a50e4307 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:53:36 +0200 +Subject: [PATCH 155/268] block/file-posix: File locking during creation + +RH-Author: Max Reitz +Message-id: <20180618145337.633-3-mreitz@redhat.com> +Patchwork-id: 80749 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/3] block/file-posix: File locking during creation +Bugzilla: 1519144 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng +RH-Acked-by: Miroslav Rezanina + +When creating a file, we should take the WRITE and RESIZE permissions. +We do not need either for the creation itself, but we do need them for +clearing and resizing it. So we can take the proper permissions by +replacing O_TRUNC with an explicit truncation to 0, and by taking the +appropriate file locks between those two steps. + +Signed-off-by: Max Reitz +Message-id: 20180509215336.31304-3-mreitz@redhat.com +Reviewed-by: Fam Zheng +Signed-off-by: Max Reitz +(cherry picked from commit b8cf1913a989b9ea6f248aaa233330047a62962e) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 37 +++++++++++++++++++++++++++++++++++-- + 1 file changed, 35 insertions(+), 2 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index c98a4a1..370a483 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -1992,6 +1992,7 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + { + BlockdevCreateOptionsFile *file_opts; + int fd; ++ int perm, shared; + int result = 0; + + /* Validate options and set default values */ +@@ -2006,14 +2007,44 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + } + + /* Create file */ +- fd = qemu_open(file_opts->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, +- 0644); ++ fd = qemu_open(file_opts->filename, O_RDWR | O_CREAT | O_BINARY, 0644); + if (fd < 0) { + result = -errno; + error_setg_errno(errp, -result, "Could not create file"); + goto out; + } + ++ /* Take permissions: We want to discard everything, so we need ++ * BLK_PERM_WRITE; and truncation to the desired size requires ++ * BLK_PERM_RESIZE. ++ * On the other hand, we cannot share the RESIZE permission ++ * because we promise that after this function, the file has the ++ * size given in the options. If someone else were to resize it ++ * concurrently, we could not guarantee that. ++ * Note that after this function, we can no longer guarantee that ++ * the file is not touched by a third party, so it may be resized ++ * then. */ ++ perm = BLK_PERM_WRITE | BLK_PERM_RESIZE; ++ shared = BLK_PERM_ALL & ~BLK_PERM_RESIZE; ++ ++ /* Step one: Take locks */ ++ result = raw_apply_lock_bytes(fd, perm, shared, false, errp); ++ if (result < 0) { ++ goto out_close; ++ } ++ ++ /* Step two: Check that nobody else has taken conflicting locks */ ++ result = raw_check_lock_bytes(fd, perm, shared, errp); ++ if (result < 0) { ++ goto out_close; ++ } ++ ++ /* Clear the file by truncating it to 0 */ ++ result = raw_regular_truncate(fd, 0, PREALLOC_MODE_OFF, errp); ++ if (result < 0) { ++ goto out_close; ++ } ++ + if (file_opts->nocow) { + #ifdef __linux__ + /* Set NOCOW flag to solve performance issue on fs like btrfs. +@@ -2029,6 +2060,8 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + #endif + } + ++ /* Resize and potentially preallocate the file to the desired ++ * final size */ + result = raw_regular_truncate(fd, file_opts->size, file_opts->preallocation, + errp); + if (result < 0) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-file-posix-Pass-FD-to-locking-helpers.patch b/SOURCES/kvm-block-file-posix-Pass-FD-to-locking-helpers.patch new file mode 100644 index 0000000..062e181 --- /dev/null +++ b/SOURCES/kvm-block-file-posix-Pass-FD-to-locking-helpers.patch @@ -0,0 +1,141 @@ +From 6ea0082c3ebf3259e1a3a479bee88d580dbbede7 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:53:35 +0200 +Subject: [PATCH 154/268] block/file-posix: Pass FD to locking helpers + +RH-Author: Max Reitz +Message-id: <20180618145337.633-2-mreitz@redhat.com> +Patchwork-id: 80751 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/3] block/file-posix: Pass FD to locking helpers +Bugzilla: 1519144 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng +RH-Acked-by: Miroslav Rezanina + +raw_apply_lock_bytes() and raw_check_lock_bytes() currently take a +BDRVRawState *, but they only use the lock_fd field. During image +creation, we do not have a BDRVRawState, but we do have an FD; so if we +want to reuse the functions there, we should modify them to receive only +the FD. + +Signed-off-by: Max Reitz +Reviewed-by: Fam Zheng +Message-id: 20180509215336.31304-2-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit d0a96155de099d388f5e1f46277a54bdcfbb0bb2) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 27 ++++++++++++++------------- + 1 file changed, 14 insertions(+), 13 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index 3794c00..c98a4a1 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -630,7 +630,7 @@ typedef enum { + * file; if @unlock == true, also unlock the unneeded bytes. + * @shared_perm_lock_bits is the mask of all permissions that are NOT shared. + */ +-static int raw_apply_lock_bytes(BDRVRawState *s, ++static int raw_apply_lock_bytes(int fd, + uint64_t perm_lock_bits, + uint64_t shared_perm_lock_bits, + bool unlock, Error **errp) +@@ -641,13 +641,13 @@ static int raw_apply_lock_bytes(BDRVRawState *s, + PERM_FOREACH(i) { + int off = RAW_LOCK_PERM_BASE + i; + if (perm_lock_bits & (1ULL << i)) { +- ret = qemu_lock_fd(s->lock_fd, off, 1, false); ++ ret = qemu_lock_fd(fd, off, 1, false); + if (ret) { + error_setg(errp, "Failed to lock byte %d", off); + return ret; + } + } else if (unlock) { +- ret = qemu_unlock_fd(s->lock_fd, off, 1); ++ ret = qemu_unlock_fd(fd, off, 1); + if (ret) { + error_setg(errp, "Failed to unlock byte %d", off); + return ret; +@@ -657,13 +657,13 @@ static int raw_apply_lock_bytes(BDRVRawState *s, + PERM_FOREACH(i) { + int off = RAW_LOCK_SHARED_BASE + i; + if (shared_perm_lock_bits & (1ULL << i)) { +- ret = qemu_lock_fd(s->lock_fd, off, 1, false); ++ ret = qemu_lock_fd(fd, off, 1, false); + if (ret) { + error_setg(errp, "Failed to lock byte %d", off); + return ret; + } + } else if (unlock) { +- ret = qemu_unlock_fd(s->lock_fd, off, 1); ++ ret = qemu_unlock_fd(fd, off, 1); + if (ret) { + error_setg(errp, "Failed to unlock byte %d", off); + return ret; +@@ -674,8 +674,7 @@ static int raw_apply_lock_bytes(BDRVRawState *s, + } + + /* Check "unshared" bytes implied by @perm and ~@shared_perm in the file. */ +-static int raw_check_lock_bytes(BDRVRawState *s, +- uint64_t perm, uint64_t shared_perm, ++static int raw_check_lock_bytes(int fd, uint64_t perm, uint64_t shared_perm, + Error **errp) + { + int ret; +@@ -685,7 +684,7 @@ static int raw_check_lock_bytes(BDRVRawState *s, + int off = RAW_LOCK_SHARED_BASE + i; + uint64_t p = 1ULL << i; + if (perm & p) { +- ret = qemu_lock_fd_test(s->lock_fd, off, 1, true); ++ ret = qemu_lock_fd_test(fd, off, 1, true); + if (ret) { + char *perm_name = bdrv_perm_names(p); + error_setg(errp, +@@ -702,7 +701,7 @@ static int raw_check_lock_bytes(BDRVRawState *s, + int off = RAW_LOCK_PERM_BASE + i; + uint64_t p = 1ULL << i; + if (!(shared_perm & p)) { +- ret = qemu_lock_fd_test(s->lock_fd, off, 1, true); ++ ret = qemu_lock_fd_test(fd, off, 1, true); + if (ret) { + char *perm_name = bdrv_perm_names(p); + error_setg(errp, +@@ -739,11 +738,11 @@ static int raw_handle_perm_lock(BlockDriverState *bs, + + switch (op) { + case RAW_PL_PREPARE: +- ret = raw_apply_lock_bytes(s, s->perm | new_perm, ++ ret = raw_apply_lock_bytes(s->lock_fd, s->perm | new_perm, + ~s->shared_perm | ~new_shared, + false, errp); + if (!ret) { +- ret = raw_check_lock_bytes(s, new_perm, new_shared, errp); ++ ret = raw_check_lock_bytes(s->lock_fd, new_perm, new_shared, errp); + if (!ret) { + return 0; + } +@@ -751,7 +750,8 @@ static int raw_handle_perm_lock(BlockDriverState *bs, + op = RAW_PL_ABORT; + /* fall through to unlock bytes. */ + case RAW_PL_ABORT: +- raw_apply_lock_bytes(s, s->perm, ~s->shared_perm, true, &local_err); ++ raw_apply_lock_bytes(s->lock_fd, s->perm, ~s->shared_perm, ++ true, &local_err); + if (local_err) { + /* Theoretically the above call only unlocks bytes and it cannot + * fail. Something weird happened, report it. +@@ -760,7 +760,8 @@ static int raw_handle_perm_lock(BlockDriverState *bs, + } + break; + case RAW_PL_COMMIT: +- raw_apply_lock_bytes(s, new_perm, ~new_shared, true, &local_err); ++ raw_apply_lock_bytes(s->lock_fd, new_perm, ~new_shared, ++ true, &local_err); + if (local_err) { + /* Theoretically the above call only unlocks bytes and it cannot + * fail. Something weird happened, report it. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-fix-QEMU-crash-with-scsi-hd-and-drive_del.patch b/SOURCES/kvm-block-fix-QEMU-crash-with-scsi-hd-and-drive_del.patch new file mode 100644 index 0000000..ad51aec --- /dev/null +++ b/SOURCES/kvm-block-fix-QEMU-crash-with-scsi-hd-and-drive_del.patch @@ -0,0 +1,90 @@ +From c800bff089dec124e622397583abfc28308d1c42 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 16:06:19 +0200 +Subject: [PATCH 215/268] block: fix QEMU crash with scsi-hd and drive_del + +RH-Author: Kevin Wolf +Message-id: <20180712160619.30712-2-kwolf@redhat.com> +Patchwork-id: 81334 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/1] block: fix QEMU crash with scsi-hd and drive_del +Bugzilla: 1599515 +RH-Acked-by: Fam Zheng +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Greg Kurz + +Removing a drive with drive_del while it is being used to run an I/O +intensive workload can cause QEMU to crash. + +An AIO flush can yield at some point: + +blk_aio_flush_entry() + blk_co_flush(blk) + bdrv_co_flush(blk->root->bs) + ... + qemu_coroutine_yield() + +and let the HMP command to run, free blk->root and give control +back to the AIO flush: + + hmp_drive_del() + blk_remove_bs() + bdrv_root_unref_child(blk->root) + child_bs = blk->root->bs + bdrv_detach_child(blk->root) + bdrv_replace_child(blk->root, NULL) + blk->root->bs = NULL + g_free(blk->root) <============== blk->root becomes stale + bdrv_unref(child_bs) + bdrv_delete(child_bs) + bdrv_close() + bdrv_drained_begin() + bdrv_do_drained_begin() + bdrv_drain_recurse() + aio_poll() + ... + qemu_coroutine_switch() + +and the AIO flush completion ends up dereferencing blk->root: + + blk_aio_complete() + scsi_aio_complete() + blk_get_aio_context(blk) + bs = blk_bs(blk) + ie, bs = blk->root ? blk->root->bs : NULL + ^^^^^ + stale + +The problem is that we should avoid making block driver graph +changes while we have in-flight requests. Let's drain all I/O +for this BB before calling bdrv_root_unref_child(). + +Signed-off-by: Greg Kurz +Signed-off-by: Kevin Wolf +(cherry picked from commit f45280cbf66d8e58224f6a253d0ae2aa72cc6280) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/block-backend.c | 5 +++++ + 1 file changed, 5 insertions(+) + +diff --git a/block/block-backend.c b/block/block-backend.c +index 5562ec4..f34e4c3 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -768,6 +768,11 @@ void blk_remove_bs(BlockBackend *blk) + + blk_update_root_state(blk); + ++ /* bdrv_root_unref_child() will cause blk->root to become stale and may ++ * switch to a completion coroutine later on. Let's drain all I/O here ++ * to avoid that and a potential QEMU crash. ++ */ ++ blk_drain(blk); + bdrv_root_unref_child(blk->root); + blk->root = NULL; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-for-jobs-do-not-clear-user_paused-until-after-.patch b/SOURCES/kvm-block-for-jobs-do-not-clear-user_paused-until-after-.patch new file mode 100644 index 0000000..2b322e6 --- /dev/null +++ b/SOURCES/kvm-block-for-jobs-do-not-clear-user_paused-until-after-.patch @@ -0,0 +1,60 @@ +From e582296e39bcca12dbfb267b3ffc511f3f64c242 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:50:58 +0100 +Subject: [PATCH 1/3] block: for jobs, do not clear user_paused until after the + resume + +RH-Author: John Snow +Message-id: <20181010205100.17689-2-jsnow@redhat.com> +Patchwork-id: 82631 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/3] block: for jobs, do not clear user_paused until after the resume +Bugzilla: 1635583 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +From: Jeff Cody + +The function job_cancel_async() will always cause an assert for blockjob +user resume. We set job->user_paused to false, and then call +job->driver->user_resume(). In the case of blockjobs, this is the +block_job_user_resume() function. + +In that function, we assert that job.user_paused is set to true. +Unfortunately, right before calling this function, it has explicitly +been set to false. + +The fix is pretty simple: set job->user_paused to false only after the +job user_resume() function has been called. + +Reviewed-by: John Snow +Reviewed-by: Eric Blake +Signed-off-by: Jeff Cody +Message-id: bb183b77d8f2dd6bd67b8da559a90ac1e74b2052.1534868459.git.jcody@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit e321c0597c7590499bacab239d7f86e257f96bcd) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + job.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/job.c b/job.c +index 3ab8ce1..dfba4bc 100644 +--- a/job.c ++++ b/job.c +@@ -700,10 +700,10 @@ static void job_cancel_async(Job *job, bool force) + { + if (job->user_paused) { + /* Do not call job_enter here, the caller will handle it. */ +- job->user_paused = false; + if (job->driver->user_resume) { + job->driver->user_resume(job); + } ++ job->user_paused = false; + assert(job->pause_count > 0); + job->pause_count--; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-ignore_bds_parents-parameter-for-drain-functio.patch b/SOURCES/kvm-block-ignore_bds_parents-parameter-for-drain-functio.patch new file mode 100644 index 0000000..e096c44 --- /dev/null +++ b/SOURCES/kvm-block-ignore_bds_parents-parameter-for-drain-functio.patch @@ -0,0 +1,481 @@ +From b4a41557a3fd97307e750c84916f9d2237f08242 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:47 +0100 +Subject: [PATCH 21/49] block: ignore_bds_parents parameter for drain functions +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-9-kwolf@redhat.com> +Patchwork-id: 82598 +O-Subject: [RHEL-8 qemu-kvm PATCH 18/44] block: ignore_bds_parents parameter for drain functions +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +In the future, bdrv_drained_all_begin/end() will drain all invidiual +nodes separately rather than whole subtrees. This means that we don't +want to propagate the drain to all parents any more: If the parent is a +BDS, it will already be drained separately. Recursing to all parents is +unnecessary work and would make it an O(n²) operation. + +Prepare the drain function for the changed drain_all by adding an +ignore_bds_parents parameter to the internal implementation that +prevents the propagation of the drain to BDS parents. We still (have to) +propagate it to non-BDS parents like BlockBackends or Jobs because those +are not drained separately. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 6cd5c9d7b2df93ef54144f170d4c908934a4767f) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 11 +++--- + block/io.c | 88 ++++++++++++++++++++++++++++------------------- + block/vvfat.c | 1 + + include/block/block.h | 16 ++++++--- + include/block/block_int.h | 6 ++++ + 5 files changed, 78 insertions(+), 44 deletions(-) + +diff --git a/block.c b/block.c +index 5a50de6..519be6e 100644 +--- a/block.c ++++ b/block.c +@@ -818,13 +818,13 @@ static char *bdrv_child_get_parent_desc(BdrvChild *c) + static void bdrv_child_cb_drained_begin(BdrvChild *child) + { + BlockDriverState *bs = child->opaque; +- bdrv_do_drained_begin_quiesce(bs, NULL); ++ bdrv_do_drained_begin_quiesce(bs, NULL, false); + } + + static bool bdrv_child_cb_drained_poll(BdrvChild *child) + { + BlockDriverState *bs = child->opaque; +- return bdrv_drain_poll(bs, false, NULL); ++ return bdrv_drain_poll(bs, false, NULL, false); + } + + static void bdrv_child_cb_drained_end(BdrvChild *child) +@@ -908,6 +908,7 @@ static void bdrv_inherited_options(int *child_flags, QDict *child_options, + } + + const BdrvChildRole child_file = { ++ .parent_is_bds = true, + .get_parent_desc = bdrv_child_get_parent_desc, + .inherit_options = bdrv_inherited_options, + .drained_begin = bdrv_child_cb_drained_begin, +@@ -933,6 +934,7 @@ static void bdrv_inherited_fmt_options(int *child_flags, QDict *child_options, + } + + const BdrvChildRole child_format = { ++ .parent_is_bds = true, + .get_parent_desc = bdrv_child_get_parent_desc, + .inherit_options = bdrv_inherited_fmt_options, + .drained_begin = bdrv_child_cb_drained_begin, +@@ -1051,6 +1053,7 @@ static int bdrv_backing_update_filename(BdrvChild *c, BlockDriverState *base, + } + + const BdrvChildRole child_backing = { ++ .parent_is_bds = true, + .get_parent_desc = bdrv_child_get_parent_desc, + .attach = bdrv_backing_attach, + .detach = bdrv_backing_detach, +@@ -4901,7 +4904,7 @@ void bdrv_set_aio_context(BlockDriverState *bs, AioContext *new_context) + AioContext *ctx = bdrv_get_aio_context(bs); + + aio_disable_external(ctx); +- bdrv_parent_drained_begin(bs, NULL); ++ bdrv_parent_drained_begin(bs, NULL, false); + bdrv_drain(bs); /* ensure there are no in-flight requests */ + + while (aio_poll(ctx, false)) { +@@ -4915,7 +4918,7 @@ void bdrv_set_aio_context(BlockDriverState *bs, AioContext *new_context) + */ + aio_context_acquire(new_context); + bdrv_attach_aio_context(bs, new_context); +- bdrv_parent_drained_end(bs, NULL); ++ bdrv_parent_drained_end(bs, NULL, false); + aio_enable_external(ctx); + aio_context_release(new_context); + } +diff --git a/block/io.c b/block/io.c +index b58c91b..0021fefd 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -41,12 +41,13 @@ + static int coroutine_fn bdrv_co_do_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int bytes, BdrvRequestFlags flags); + +-void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore) ++void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore, ++ bool ignore_bds_parents) + { + BdrvChild *c, *next; + + QLIST_FOREACH_SAFE(c, &bs->parents, next_parent, next) { +- if (c == ignore) { ++ if (c == ignore || (ignore_bds_parents && c->role->parent_is_bds)) { + continue; + } + if (c->role->drained_begin) { +@@ -55,12 +56,13 @@ void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore) + } + } + +-void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore) ++void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore, ++ bool ignore_bds_parents) + { + BdrvChild *c, *next; + + QLIST_FOREACH_SAFE(c, &bs->parents, next_parent, next) { +- if (c == ignore) { ++ if (c == ignore || (ignore_bds_parents && c->role->parent_is_bds)) { + continue; + } + if (c->role->drained_end) { +@@ -69,13 +71,14 @@ void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore) + } + } + +-static bool bdrv_parent_drained_poll(BlockDriverState *bs, BdrvChild *ignore) ++static bool bdrv_parent_drained_poll(BlockDriverState *bs, BdrvChild *ignore, ++ bool ignore_bds_parents) + { + BdrvChild *c, *next; + bool busy = false; + + QLIST_FOREACH_SAFE(c, &bs->parents, next_parent, next) { +- if (c == ignore) { ++ if (c == ignore || (ignore_bds_parents && c->role->parent_is_bds)) { + continue; + } + if (c->role->drained_poll) { +@@ -166,6 +169,7 @@ typedef struct { + bool recursive; + bool poll; + BdrvChild *parent; ++ bool ignore_bds_parents; + } BdrvCoDrainData; + + static void coroutine_fn bdrv_drain_invoke_entry(void *opaque) +@@ -219,11 +223,11 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin) + + /* Returns true if BDRV_POLL_WHILE() should go into a blocking aio_poll() */ + bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, +- BdrvChild *ignore_parent) ++ BdrvChild *ignore_parent, bool ignore_bds_parents) + { + BdrvChild *child, *next; + +- if (bdrv_parent_drained_poll(bs, ignore_parent)) { ++ if (bdrv_parent_drained_poll(bs, ignore_parent, ignore_bds_parents)) { + return true; + } + +@@ -232,8 +236,9 @@ bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, + } + + if (recursive) { ++ assert(!ignore_bds_parents); + QLIST_FOREACH_SAFE(child, &bs->children, next, next) { +- if (bdrv_drain_poll(child->bs, recursive, child)) { ++ if (bdrv_drain_poll(child->bs, recursive, child, false)) { + return true; + } + } +@@ -249,13 +254,14 @@ static bool bdrv_drain_poll_top_level(BlockDriverState *bs, bool recursive, + * have executed. */ + while (aio_poll(bs->aio_context, false)); + +- return bdrv_drain_poll(bs, recursive, ignore_parent); ++ return bdrv_drain_poll(bs, recursive, ignore_parent, false); + } + + static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, +- BdrvChild *parent, bool poll); ++ BdrvChild *parent, bool ignore_bds_parents, ++ bool poll); + static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, +- BdrvChild *parent); ++ BdrvChild *parent, bool ignore_bds_parents); + + static void bdrv_co_drain_bh_cb(void *opaque) + { +@@ -266,9 +272,11 @@ static void bdrv_co_drain_bh_cb(void *opaque) + if (bs) { + bdrv_dec_in_flight(bs); + if (data->begin) { +- bdrv_do_drained_begin(bs, data->recursive, data->parent, data->poll); ++ bdrv_do_drained_begin(bs, data->recursive, data->parent, ++ data->ignore_bds_parents, data->poll); + } else { +- bdrv_do_drained_end(bs, data->recursive, data->parent); ++ bdrv_do_drained_end(bs, data->recursive, data->parent, ++ data->ignore_bds_parents); + } + } else { + assert(data->begin); +@@ -281,7 +289,9 @@ static void bdrv_co_drain_bh_cb(void *opaque) + + static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + bool begin, bool recursive, +- BdrvChild *parent, bool poll) ++ BdrvChild *parent, ++ bool ignore_bds_parents, ++ bool poll) + { + BdrvCoDrainData data; + +@@ -296,6 +306,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + .begin = begin, + .recursive = recursive, + .parent = parent, ++ .ignore_bds_parents = ignore_bds_parents, + .poll = poll, + }; + if (bs) { +@@ -311,7 +322,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs, + } + + void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, +- BdrvChild *parent) ++ BdrvChild *parent, bool ignore_bds_parents) + { + assert(!qemu_in_coroutine()); + +@@ -320,26 +331,30 @@ void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, + aio_disable_external(bdrv_get_aio_context(bs)); + } + +- bdrv_parent_drained_begin(bs, parent); ++ bdrv_parent_drained_begin(bs, parent, ignore_bds_parents); + bdrv_drain_invoke(bs, true); + } + + static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, +- BdrvChild *parent, bool poll) ++ BdrvChild *parent, bool ignore_bds_parents, ++ bool poll) + { + BdrvChild *child, *next; + + if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(bs, true, recursive, parent, poll); ++ bdrv_co_yield_to_drain(bs, true, recursive, parent, ignore_bds_parents, ++ poll); + return; + } + +- bdrv_do_drained_begin_quiesce(bs, parent); ++ bdrv_do_drained_begin_quiesce(bs, parent, ignore_bds_parents); + + if (recursive) { ++ assert(!ignore_bds_parents); + bs->recursive_quiesce_counter++; + QLIST_FOREACH_SAFE(child, &bs->children, next, next) { +- bdrv_do_drained_begin(child->bs, true, child, false); ++ bdrv_do_drained_begin(child->bs, true, child, ignore_bds_parents, ++ false); + } + } + +@@ -353,28 +368,30 @@ static void bdrv_do_drained_begin(BlockDriverState *bs, bool recursive, + * nodes. + */ + if (poll) { ++ assert(!ignore_bds_parents); + BDRV_POLL_WHILE(bs, bdrv_drain_poll_top_level(bs, recursive, parent)); + } + } + + void bdrv_drained_begin(BlockDriverState *bs) + { +- bdrv_do_drained_begin(bs, false, NULL, true); ++ bdrv_do_drained_begin(bs, false, NULL, false, true); + } + + void bdrv_subtree_drained_begin(BlockDriverState *bs) + { +- bdrv_do_drained_begin(bs, true, NULL, true); ++ bdrv_do_drained_begin(bs, true, NULL, false, true); + } + +-void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, +- BdrvChild *parent) ++static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, ++ BdrvChild *parent, bool ignore_bds_parents) + { + BdrvChild *child, *next; + int old_quiesce_counter; + + if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(bs, false, recursive, parent, false); ++ bdrv_co_yield_to_drain(bs, false, recursive, parent, ignore_bds_parents, ++ false); + return; + } + assert(bs->quiesce_counter > 0); +@@ -382,27 +399,28 @@ void bdrv_do_drained_end(BlockDriverState *bs, bool recursive, + + /* Re-enable things in child-to-parent order */ + bdrv_drain_invoke(bs, false); +- bdrv_parent_drained_end(bs, parent); ++ bdrv_parent_drained_end(bs, parent, ignore_bds_parents); + if (old_quiesce_counter == 1) { + aio_enable_external(bdrv_get_aio_context(bs)); + } + + if (recursive) { ++ assert(!ignore_bds_parents); + bs->recursive_quiesce_counter--; + QLIST_FOREACH_SAFE(child, &bs->children, next, next) { +- bdrv_do_drained_end(child->bs, true, child); ++ bdrv_do_drained_end(child->bs, true, child, ignore_bds_parents); + } + } + } + + void bdrv_drained_end(BlockDriverState *bs) + { +- bdrv_do_drained_end(bs, false, NULL); ++ bdrv_do_drained_end(bs, false, NULL, false); + } + + void bdrv_subtree_drained_end(BlockDriverState *bs) + { +- bdrv_do_drained_end(bs, true, NULL); ++ bdrv_do_drained_end(bs, true, NULL, false); + } + + void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_parent) +@@ -410,7 +428,7 @@ void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_parent) + int i; + + for (i = 0; i < new_parent->recursive_quiesce_counter; i++) { +- bdrv_do_drained_begin(child->bs, true, child, true); ++ bdrv_do_drained_begin(child->bs, true, child, false, true); + } + } + +@@ -419,7 +437,7 @@ void bdrv_unapply_subtree_drain(BdrvChild *child, BlockDriverState *old_parent) + int i; + + for (i = 0; i < old_parent->recursive_quiesce_counter; i++) { +- bdrv_do_drained_end(child->bs, true, child); ++ bdrv_do_drained_end(child->bs, true, child, false); + } + } + +@@ -471,7 +489,7 @@ void bdrv_drain_all_begin(void) + BdrvNextIterator it; + + if (qemu_in_coroutine()) { +- bdrv_co_yield_to_drain(NULL, true, false, NULL, true); ++ bdrv_co_yield_to_drain(NULL, true, false, NULL, false, true); + return; + } + +@@ -485,7 +503,7 @@ void bdrv_drain_all_begin(void) + AioContext *aio_context = bdrv_get_aio_context(bs); + + aio_context_acquire(aio_context); +- bdrv_do_drained_begin(bs, true, NULL, true); ++ bdrv_do_drained_begin(bs, true, NULL, false, true); + aio_context_release(aio_context); + } + +@@ -503,7 +521,7 @@ void bdrv_drain_all_end(void) + AioContext *aio_context = bdrv_get_aio_context(bs); + + aio_context_acquire(aio_context); +- bdrv_do_drained_end(bs, true, NULL); ++ bdrv_do_drained_end(bs, true, NULL, false); + aio_context_release(aio_context); + } + } +diff --git a/block/vvfat.c b/block/vvfat.c +index 4595f33..c7d2ed2 100644 +--- a/block/vvfat.c ++++ b/block/vvfat.c +@@ -3134,6 +3134,7 @@ static void vvfat_qcow_options(int *child_flags, QDict *child_options, + } + + static const BdrvChildRole child_vvfat_qcow = { ++ .parent_is_bds = true, + .inherit_options = vvfat_qcow_options, + }; + +diff --git a/include/block/block.h b/include/block/block.h +index 43f29b5..6e91803 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -585,7 +585,8 @@ void bdrv_io_unplug(BlockDriverState *bs); + * Begin a quiesced section of all users of @bs. This is part of + * bdrv_drained_begin. + */ +-void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore); ++void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore, ++ bool ignore_bds_parents); + + /** + * bdrv_parent_drained_end: +@@ -593,18 +594,23 @@ void bdrv_parent_drained_begin(BlockDriverState *bs, BdrvChild *ignore); + * End a quiesced section of all users of @bs. This is part of + * bdrv_drained_end. + */ +-void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore); ++void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore, ++ bool ignore_bds_parents); + + /** + * bdrv_drain_poll: + * + * Poll for pending requests in @bs, its parents (except for @ignore_parent), +- * and if @recursive is true its children as well. ++ * and if @recursive is true its children as well (used for subtree drain). ++ * ++ * If @ignore_bds_parents is true, parents that are BlockDriverStates must ++ * ignore the drain request because they will be drained separately (used for ++ * drain_all). + * + * This is part of bdrv_drained_begin. + */ + bool bdrv_drain_poll(BlockDriverState *bs, bool recursive, +- BdrvChild *ignore_parent); ++ BdrvChild *ignore_parent, bool ignore_bds_parents); + + /** + * bdrv_drained_begin: +@@ -625,7 +631,7 @@ void bdrv_drained_begin(BlockDriverState *bs); + * running requests to complete. + */ + void bdrv_do_drained_begin_quiesce(BlockDriverState *bs, +- BdrvChild *parent); ++ BdrvChild *parent, bool ignore_bds_parents); + + /** + * Like bdrv_drained_begin, but recursively begins a quiesced section for +diff --git a/include/block/block_int.h b/include/block/block_int.h +index beeacde..0ad8a76 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -582,6 +582,12 @@ struct BdrvChildRole { + * points to. */ + bool stay_at_node; + ++ /* If true, the parent is a BlockDriverState and bdrv_next_all_states() ++ * will return it. This information is used for drain_all, where every node ++ * will be drained separately, so the drain only needs to be propagated to ++ * non-BDS parents. */ ++ bool parent_is_bds; ++ + void (*inherit_options)(int *child_flags, QDict *child_options, + int parent_flags, QDict *parent_options); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-io-fix-copy_range.patch b/SOURCES/kvm-block-io-fix-copy_range.patch new file mode 100644 index 0000000..ce0f79d --- /dev/null +++ b/SOURCES/kvm-block-io-fix-copy_range.patch @@ -0,0 +1,152 @@ +From 58bdc00e94d7a965f91881022386ec73fe081c2f Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 24 Jul 2018 12:59:12 +0200 +Subject: [PATCH 237/268] block/io: fix copy_range + +RH-Author: John Snow +Message-id: <20180718225511.14878-20-jsnow@redhat.com> +Patchwork-id: 81420 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 19/35] block/io: fix copy_range +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Here two things are fixed: + +1. Architecture + +On each recursion step, we go to the child of src or dst, only for one +of them. So, it's wrong to create tracked requests for both on each +step. It leads to tracked requests duplication. + +2. Wait for serializing requests on write path independently of + BDRV_REQ_NO_SERIALISING + +Before commit 9ded4a01149 "backup: Use copy offloading", +BDRV_REQ_NO_SERIALISING was used for only one case: read in +copy-on-write operation during backup. Also, the flag was handled only +on read path (in bdrv_co_preadv and bdrv_aligned_preadv). + +After 9ded4a01149, flag is used for not waiting serializing operations +on backup target (in same case of copy-on-write operation). This +behavior change is unsubstantiated and potentially dangerous, let's +drop it and add additional asserts and documentation. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit 999658a05e61a8d87b65827da665302bb44ce8c9) +Signed-off-by: John Snow +--- + block/io.c | 43 +++++++++++++++++++++++++++---------------- + include/block/block.h | 12 ++++++++++++ + 2 files changed, 39 insertions(+), 16 deletions(-) + +diff --git a/block/io.c b/block/io.c +index b6754f3..f8de42f 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -1505,6 +1505,8 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child, + max_transfer = QEMU_ALIGN_DOWN(MIN_NON_ZERO(bs->bl.max_transfer, INT_MAX), + align); + ++ /* BDRV_REQ_NO_SERIALISING is only for read operation */ ++ assert(!(flags & BDRV_REQ_NO_SERIALISING)); + waited = wait_serialising_requests(req); + assert(!waited || !req->serialising); + assert(req->overlap_offset <= offset); +@@ -2847,7 +2849,7 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + BdrvRequestFlags flags, + bool recurse_src) + { +- BdrvTrackedRequest src_req, dst_req; ++ BdrvTrackedRequest req; + int ret; + + if (!dst || !dst->bs) { +@@ -2876,32 +2878,41 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + return -ENOTSUP; + } + +- bdrv_inc_in_flight(src->bs); +- bdrv_inc_in_flight(dst->bs); +- tracked_request_begin(&src_req, src->bs, src_offset, +- bytes, BDRV_TRACKED_READ); +- tracked_request_begin(&dst_req, dst->bs, dst_offset, +- bytes, BDRV_TRACKED_WRITE); +- +- if (!(flags & BDRV_REQ_NO_SERIALISING)) { +- wait_serialising_requests(&src_req); +- wait_serialising_requests(&dst_req); +- } + if (recurse_src) { ++ bdrv_inc_in_flight(src->bs); ++ tracked_request_begin(&req, src->bs, src_offset, bytes, ++ BDRV_TRACKED_READ); ++ ++ if (!(flags & BDRV_REQ_NO_SERIALISING)) { ++ wait_serialising_requests(&req); ++ } ++ + ret = src->bs->drv->bdrv_co_copy_range_from(src->bs, + src, src_offset, + dst, dst_offset, + bytes, flags); ++ ++ tracked_request_end(&req); ++ bdrv_dec_in_flight(src->bs); + } else { ++ bdrv_inc_in_flight(dst->bs); ++ tracked_request_begin(&req, dst->bs, dst_offset, bytes, ++ BDRV_TRACKED_WRITE); ++ ++ /* BDRV_REQ_NO_SERIALISING is only for read operation, ++ * so we ignore it in flags. ++ */ ++ wait_serialising_requests(&req); ++ + ret = dst->bs->drv->bdrv_co_copy_range_to(dst->bs, + src, src_offset, + dst, dst_offset, + bytes, flags); ++ ++ tracked_request_end(&req); ++ bdrv_dec_in_flight(dst->bs); + } +- tracked_request_end(&src_req); +- tracked_request_end(&dst_req); +- bdrv_dec_in_flight(src->bs); +- bdrv_dec_in_flight(dst->bs); ++ + return ret; + } + +diff --git a/include/block/block.h b/include/block/block.h +index e1d5e47..716fb5b 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -50,6 +50,18 @@ typedef enum { + * opened with BDRV_O_UNMAP. + */ + BDRV_REQ_MAY_UNMAP = 0x4, ++ ++ /* ++ * The BDRV_REQ_NO_SERIALISING flag is only valid for reads and means that ++ * we don't want wait_serialising_requests() during the read operation. ++ * ++ * This flag is used for backup copy-on-write operations, when we need to ++ * read old data before write (write notifier triggered). It is okay since ++ * we already waited for other serializing requests in the initiating write ++ * (see bdrv_aligned_pwritev), and it is necessary if the initiating write ++ * is already serializing (without the flag, the read would deadlock ++ * waiting for the serialising write to complete). ++ */ + BDRV_REQ_NO_SERIALISING = 0x8, + BDRV_REQ_FUA = 0x10, + BDRV_REQ_WRITE_COMPRESSED = 0x20, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-iotest-to-catch-abort-on-forced-blockjob-cance.patch b/SOURCES/kvm-block-iotest-to-catch-abort-on-forced-blockjob-cance.patch new file mode 100644 index 0000000..3d39d23 --- /dev/null +++ b/SOURCES/kvm-block-iotest-to-catch-abort-on-forced-blockjob-cance.patch @@ -0,0 +1,173 @@ +From 171becce35c675a68b5a9b7d4ecea9de88906346 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:51:00 +0100 +Subject: [PATCH 3/3] block: iotest to catch abort on forced blockjob cancel + +RH-Author: John Snow +Message-id: <20181010205100.17689-4-jsnow@redhat.com> +Patchwork-id: 82632 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 3/3] block: iotest to catch abort on forced blockjob cancel +Bugzilla: 1635583 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +From: Jeff Cody + +Signed-off-by: Jeff Cody +Reviewed-by: John Snow +Message-id: df317f617fbe5affcf699cb8560e7b0c2e028a64.1534868459.git.jcody@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit 26bf474ba92c76e61bea51726e22da6dfd185296) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/229 | 95 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/229.out | 23 +++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 119 insertions(+) + create mode 100755 tests/qemu-iotests/229 + create mode 100644 tests/qemu-iotests/229.out + +diff --git a/tests/qemu-iotests/229 b/tests/qemu-iotests/229 +new file mode 100755 +index 0000000..ff851ec +--- /dev/null ++++ b/tests/qemu-iotests/229 +@@ -0,0 +1,95 @@ ++#!/bin/bash ++# ++# Test for force canceling a running blockjob that is paused in ++# an error state. ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++# creator ++owner=jcody@redhat.com ++ ++seq="$(basename $0)" ++echo "QA output created by $seq" ++ ++here="$PWD" ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ _cleanup_qemu ++ _cleanup_test_img ++ rm -f "$TEST_IMG" "$DEST_IMG" ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++. ./common.qemu ++ ++# Needs backing file and backing format support ++_supported_fmt qcow2 qed ++_supported_proto file ++_supported_os Linux ++ ++ ++DEST_IMG="$TEST_DIR/d.$IMGFMT" ++TEST_IMG="$TEST_DIR/b.$IMGFMT" ++ ++_make_test_img 2M ++ ++# destination for mirror will be too small, causing error ++TEST_IMG=$DEST_IMG _make_test_img 1M ++ ++$QEMU_IO -c 'write 0 2M' "$TEST_IMG" | _filter_qemu_io ++ ++_launch_qemu -drive id=testdisk,file="$TEST_IMG",format="$IMGFMT" ++ ++_send_qemu_cmd $QEMU_HANDLE \ ++ "{'execute': 'qmp_capabilities'}" \ ++ 'return' ++ ++echo ++echo '=== Starting drive-mirror, causing error & stop ===' ++echo ++ ++_send_qemu_cmd $QEMU_HANDLE \ ++ "{'execute': 'drive-mirror', ++ 'arguments': {'device': 'testdisk', ++ 'mode': 'absolute-paths', ++ 'format': '$IMGFMT', ++ 'target': '$DEST_IMG', ++ 'sync': 'full', ++ 'mode': 'existing', ++ 'on-source-error': 'stop', ++ 'on-target-error': 'stop' }}" \ ++ "JOB_STATUS_CHANGE.*pause" ++ ++echo ++echo '=== Force cancel job paused in error state ===' ++echo ++ ++success_or_failure="y" _send_qemu_cmd $QEMU_HANDLE \ ++ "{'execute': 'block-job-cancel', ++ 'arguments': { 'device': 'testdisk', ++ 'force': true}}" \ ++ "BLOCK_JOB_CANCELLED" "Assertion" ++ ++# success, all done ++echo "*** done" ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/229.out b/tests/qemu-iotests/229.out +new file mode 100644 +index 0000000..4c41128 +--- /dev/null ++++ b/tests/qemu-iotests/229.out +@@ -0,0 +1,23 @@ ++QA output created by 229 ++Formatting 'TEST_DIR/b.IMGFMT', fmt=IMGFMT size=2097152 ++Formatting 'TEST_DIR/d.IMGFMT', fmt=IMGFMT size=1048576 ++wrote 2097152/2097152 bytes at offset 0 ++2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++{"return": {}} ++ ++=== Starting drive-mirror, causing error & stop === ++ ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "testdisk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "testdisk"}} ++{"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "testdisk", "operation": "write", "action": "stop"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "paused", "id": "testdisk"}} ++ ++=== Force cancel job paused in error state === ++ ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "testdisk"}} ++{"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "testdisk", "operation": "write", "action": "stop"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "testdisk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "testdisk", "len": 2097152, "offset": 1048576, "speed": 0, "type": "mirror"}} ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 401258f..0242b2f 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -221,3 +221,4 @@ + 222 rw auto quick + 223 rw auto quick + 226 auto quick ++229 auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-linux-aio-acquire-AioContext-before-qemu_laio_.patch b/SOURCES/kvm-block-linux-aio-acquire-AioContext-before-qemu_laio_.patch new file mode 100644 index 0000000..dd2800c --- /dev/null +++ b/SOURCES/kvm-block-linux-aio-acquire-AioContext-before-qemu_laio_.patch @@ -0,0 +1,134 @@ +From 24c1b53802b14ce45767d17b6dec88a917d24a70 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:52 +0100 +Subject: [PATCH 26/49] block/linux-aio: acquire AioContext before + qemu_laio_process_completions + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-14-kwolf@redhat.com> +Patchwork-id: 82603 +O-Subject: [RHEL-8 qemu-kvm PATCH 23/44] block/linux-aio: acquire AioContext before qemu_laio_process_completions +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +From: Sergio Lopez + +In qemu_laio_process_completions_and_submit, the AioContext is acquired +before the ioq_submit iteration and after qemu_laio_process_completions, +but the latter is not thread safe either. + +This change avoids a number of random crashes when the Main Thread and +an IO Thread collide processing completions for the same AioContext. +This is an example of such crash: + + - The IO Thread is trying to acquire the AioContext at aio_co_enter, + which evidences that it didn't lock it before: + +Thread 3 (Thread 0x7fdfd8bd8700 (LWP 36743)): + #0 0x00007fdfe0dd542d in __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 + #1 0x00007fdfe0dd0de6 in _L_lock_870 () at /lib64/libpthread.so.0 + #2 0x00007fdfe0dd0cdf in __GI___pthread_mutex_lock (mutex=mutex@entry=0x5631fde0e6c0) + at ../nptl/pthread_mutex_lock.c:114 + #3 0x00005631fc0603a7 in qemu_mutex_lock_impl (mutex=0x5631fde0e6c0, file=0x5631fc23520f "util/async.c", line=511) at util/qemu-thread-posix.c:66 + #4 0x00005631fc05b558 in aio_co_enter (ctx=0x5631fde0e660, co=0x7fdfcc0c2b40) at util/async.c:493 + #5 0x00005631fc05b5ac in aio_co_wake (co=) at util/async.c:478 + #6 0x00005631fbfc51ad in qemu_laio_process_completion (laiocb=) at block/linux-aio.c:104 + #7 0x00005631fbfc523c in qemu_laio_process_completions (s=s@entry=0x7fdfc0297670) + at block/linux-aio.c:222 + #8 0x00005631fbfc5499 in qemu_laio_process_completions_and_submit (s=0x7fdfc0297670) + at block/linux-aio.c:237 + #9 0x00005631fc05d978 in aio_dispatch_handlers (ctx=ctx@entry=0x5631fde0e660) at util/aio-posix.c:406 + #10 0x00005631fc05e3ea in aio_poll (ctx=0x5631fde0e660, blocking=blocking@entry=true) + at util/aio-posix.c:693 + #11 0x00005631fbd7ad96 in iothread_run (opaque=0x5631fde0e1c0) at iothread.c:64 + #12 0x00007fdfe0dcee25 in start_thread (arg=0x7fdfd8bd8700) at pthread_create.c:308 + #13 0x00007fdfe0afc34d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113 + + - The Main Thread is also processing completions from the same + AioContext, and crashes due to failed assertion at util/iov.c:78: + +Thread 1 (Thread 0x7fdfeb5eac80 (LWP 36740)): + #0 0x00007fdfe0a391f7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 + #1 0x00007fdfe0a3a8e8 in __GI_abort () at abort.c:90 + #2 0x00007fdfe0a32266 in __assert_fail_base (fmt=0x7fdfe0b84e68 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5631fc238ccb "offset == 0", file=file@entry=0x5631fc23698e "util/iov.c", line=line@entry=78, function=function@entry=0x5631fc236adc <__PRETTY_FUNCTION__.15220> "iov_memset") + at assert.c:92 + #3 0x00007fdfe0a32312 in __GI___assert_fail (assertion=assertion@entry=0x5631fc238ccb "offset == 0", file=file@entry=0x5631fc23698e "util/iov.c", line=line@entry=78, function=function@entry=0x5631fc236adc <__PRETTY_FUNCTION__.15220> "iov_memset") at assert.c:101 + #4 0x00005631fc065287 in iov_memset (iov=, iov_cnt=, offset=, offset@entry=65536, fillc=fillc@entry=0, bytes=15515191315812405248) at util/iov.c:78 + #5 0x00005631fc065a63 in qemu_iovec_memset (qiov=, offset=offset@entry=65536, fillc=fillc@entry=0, bytes=) at util/iov.c:410 + #6 0x00005631fbfc5178 in qemu_laio_process_completion (laiocb=0x7fdd920df630) at block/linux-aio.c:88 + #7 0x00005631fbfc523c in qemu_laio_process_completions (s=s@entry=0x7fdfc0297670) + at block/linux-aio.c:222 + #8 0x00005631fbfc5499 in qemu_laio_process_completions_and_submit (s=0x7fdfc0297670) + at block/linux-aio.c:237 + #9 0x00005631fbfc54ed in qemu_laio_poll_cb (opaque=) at block/linux-aio.c:272 + #10 0x00005631fc05d85e in run_poll_handlers_once (ctx=ctx@entry=0x5631fde0e660) at util/aio-posix.c:497 + #11 0x00005631fc05e2ca in aio_poll (blocking=false, ctx=0x5631fde0e660) at util/aio-posix.c:574 + #12 0x00005631fc05e2ca in aio_poll (ctx=0x5631fde0e660, blocking=blocking@entry=false) + at util/aio-posix.c:604 + #13 0x00005631fbfcb8a3 in bdrv_do_drained_begin (ignore_parent=, recursive=, bs=) at block/io.c:273 + #14 0x00005631fbfcb8a3 in bdrv_do_drained_begin (bs=0x5631fe8b6200, recursive=, parent=0x0, ignore_bds_parents=, poll=) at block/io.c:390 + #15 0x00005631fbfbcd2e in blk_drain (blk=0x5631fe83ac80) at block/block-backend.c:1590 + #16 0x00005631fbfbe138 in blk_remove_bs (blk=blk@entry=0x5631fe83ac80) at block/block-backend.c:774 + #17 0x00005631fbfbe3d6 in blk_unref (blk=0x5631fe83ac80) at block/block-backend.c:401 + #18 0x00005631fbfbe3d6 in blk_unref (blk=0x5631fe83ac80) at block/block-backend.c:449 + #19 0x00005631fbfc9a69 in commit_complete (job=0x5631fe8b94b0, opaque=0x7fdfcc1bb080) + at block/commit.c:92 + #20 0x00005631fbf7d662 in job_defer_to_main_loop_bh (opaque=0x7fdfcc1b4560) at job.c:973 + #21 0x00005631fc05ad41 in aio_bh_poll (bh=0x7fdfcc01ad90) at util/async.c:90 + #22 0x00005631fc05ad41 in aio_bh_poll (ctx=ctx@entry=0x5631fddffdb0) at util/async.c:118 + #23 0x00005631fc05e210 in aio_dispatch (ctx=0x5631fddffdb0) at util/aio-posix.c:436 + #24 0x00005631fc05ac1e in aio_ctx_dispatch (source=, callback=, user_data=) at util/async.c:261 + #25 0x00007fdfeaae44c9 in g_main_context_dispatch (context=0x5631fde00140) at gmain.c:3201 + #26 0x00007fdfeaae44c9 in g_main_context_dispatch (context=context@entry=0x5631fde00140) at gmain.c:3854 + #27 0x00005631fc05d503 in main_loop_wait () at util/main-loop.c:215 + #28 0x00005631fc05d503 in main_loop_wait (timeout=) at util/main-loop.c:238 + #29 0x00005631fc05d503 in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:497 + #30 0x00005631fbd81412 in main_loop () at vl.c:1866 + #31 0x00005631fbc18ff3 in main (argc=, argv=, envp=) + at vl.c:4647 + + - A closer examination shows that s->io_q.in_flight appears to have + gone backwards: + +(gdb) frame 7 + #7 0x00005631fbfc523c in qemu_laio_process_completions (s=s@entry=0x7fdfc0297670) + at block/linux-aio.c:222 +222 qemu_laio_process_completion(laiocb); +(gdb) p s +$2 = (LinuxAioState *) 0x7fdfc0297670 +(gdb) p *s +$3 = {aio_context = 0x5631fde0e660, ctx = 0x7fdfeb43b000, e = {rfd = 33, wfd = 33}, io_q = {plugged = 0, + in_queue = 0, in_flight = 4294967280, blocked = false, pending = {sqh_first = 0x0, + sqh_last = 0x7fdfc0297698}}, completion_bh = 0x7fdfc0280ef0, event_idx = 21, event_max = 241} +(gdb) p/x s->io_q.in_flight +$4 = 0xfffffff0 + +Signed-off-by: Sergio Lopez +Signed-off-by: Kevin Wolf +(cherry picked from commit e091f0e905a4481f347913420f327d427f18d9d4) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/linux-aio.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/linux-aio.c b/block/linux-aio.c +index 88b8d55..abd8886 100644 +--- a/block/linux-aio.c ++++ b/block/linux-aio.c +@@ -233,9 +233,9 @@ static void qemu_laio_process_completions(LinuxAioState *s) + + static void qemu_laio_process_completions_and_submit(LinuxAioState *s) + { ++ aio_context_acquire(s->aio_context); + qemu_laio_process_completions(s); + +- aio_context_acquire(s->aio_context); + if (!s->io_q.plugged && !QSIMPLEQ_EMPTY(&s->io_q.pending)) { + ioq_submit(s); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-mirror-Make-cancel-always-cancel-pre-READY.patch b/SOURCES/kvm-block-mirror-Make-cancel-always-cancel-pre-READY.patch new file mode 100644 index 0000000..8ddc317 --- /dev/null +++ b/SOURCES/kvm-block-mirror-Make-cancel-always-cancel-pre-READY.patch @@ -0,0 +1,59 @@ +From a31ab535355600b06654345542f388d855c4569c Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:47:35 +0200 +Subject: [PATCH 034/268] block/mirror: Make cancel always cancel pre-READY + +RH-Author: Max Reitz +Message-id: <20180618144736.29873-3-mreitz@redhat.com> +Patchwork-id: 80748 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/3] block/mirror: Make cancel always cancel pre-READY +Bugzilla: 1572856 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +Commit b76e4458b1eb3c32e9824fe6aa51f67d2b251748 made the mirror block +job respect block-job-cancel's @force flag: With that flag set, it would +now always really cancel, even post-READY. + +Unfortunately, it had a side effect: Without that flag set, it would now +never cancel, not even before READY. Considering that is an +incompatible change and not noted anywhere in the commit or the +description of block-job-cancel's @force parameter, this seems +unintentional and we should revert to the previous behavior, which is to +immediately cancel the job when block-job-cancel is called before source +and target are in sync (i.e. before the READY event). + +Cc: qemu-stable@nongnu.org +Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1572856 +Reported-by: Yanan Fu +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180501220509.14152-2-mreitz@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit eb36639f7bbc16055e551593b81365e8ae3b0b05) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/mirror.c | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 9436a8d..99da9c0 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -874,7 +874,9 @@ static void coroutine_fn mirror_run(void *opaque) + } + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); + block_job_sleep_ns(&s->common, delay_ns); +- if (block_job_is_cancelled(&s->common) && s->common.force) { ++ if (block_job_is_cancelled(&s->common) && ++ (!s->synced || s->common.force)) ++ { + break; + } + s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-mirror-add-block-job-creation-flags.patch b/SOURCES/kvm-block-mirror-add-block-job-creation-flags.patch new file mode 100644 index 0000000..a405b90 --- /dev/null +++ b/SOURCES/kvm-block-mirror-add-block-job-creation-flags.patch @@ -0,0 +1,100 @@ +From dca12aacc8c49d23415968098f888aaefb52f747 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:17 +0100 +Subject: [PATCH 14/28] block/mirror: add block job creation flags + +RH-Author: John Snow +Message-id: <20180925223431.24791-12-jsnow@redhat.com> +Patchwork-id: 82268 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 11/25] block/mirror: add block job creation flags +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Add support for taking and passing forward job creation flags. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +Message-id: 20180906130225.5118-3-jsnow@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit a1999b33488daba68a1bcd7c6fdf314ddeacc6a2) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/mirror.c | 5 +++-- + blockdev.c | 3 ++- + include/block/block_int.h | 5 ++++- + 3 files changed, 9 insertions(+), 4 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 2604f61..9bed603 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -1265,7 +1265,8 @@ fail: + + void mirror_start(const char *job_id, BlockDriverState *bs, + BlockDriverState *target, const char *replaces, +- int64_t speed, uint32_t granularity, int64_t buf_size, ++ int creation_flags, int64_t speed, ++ uint32_t granularity, int64_t buf_size, + MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, + BlockdevOnError on_source_error, + BlockdevOnError on_target_error, +@@ -1280,7 +1281,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs, + } + is_none_mode = mode == MIRROR_SYNC_MODE_NONE; + base = mode == MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL; +- mirror_start_job(job_id, bs, JOB_DEFAULT, target, replaces, ++ mirror_start_job(job_id, bs, creation_flags, target, replaces, + speed, granularity, buf_size, backing_mode, + on_source_error, on_target_error, unmap, NULL, NULL, + &mirror_job_driver, is_none_mode, base, false, +diff --git a/blockdev.c b/blockdev.c +index 4f96aa7..a265dc7 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3700,6 +3700,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs, + const char *filter_node_name, + Error **errp) + { ++ int job_flags = JOB_DEFAULT; + + if (!has_speed) { + speed = 0; +@@ -3749,7 +3750,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs, + * and will allow to check whether the node still exist at mirror completion + */ + mirror_start(job_id, bs, target, +- has_replaces ? replaces : NULL, ++ has_replaces ? replaces : NULL, job_flags, + speed, granularity, buf_size, sync, backing_mode, + on_source_error, on_target_error, unmap, filter_node_name, + errp); +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 25ad363..07517cf 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -1000,6 +1000,8 @@ void commit_active_start(const char *job_id, BlockDriverState *bs, + * @target: Block device to write to. + * @replaces: Block graph node name to replace once the mirror is done. Can + * only be used when full mirroring is selected. ++ * @creation_flags: Flags that control the behavior of the Job lifetime. ++ * See @BlockJobCreateFlags + * @speed: The maximum speed, in bytes per second, or 0 for unlimited. + * @granularity: The chosen granularity for the dirty bitmap. + * @buf_size: The amount of data that can be in flight at one time. +@@ -1020,7 +1022,8 @@ void commit_active_start(const char *job_id, BlockDriverState *bs, + */ + void mirror_start(const char *job_id, BlockDriverState *bs, + BlockDriverState *target, const char *replaces, +- int64_t speed, uint32_t granularity, int64_t buf_size, ++ int creation_flags, int64_t speed, ++ uint32_t granularity, int64_t buf_size, + MirrorSyncMode mode, BlockMirrorBackingMode backing_mode, + BlockdevOnError on_source_error, + BlockdevOnError on_target_error, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-mirror-conservative-mirror_exit-refactor.patch b/SOURCES/kvm-block-mirror-conservative-mirror_exit-refactor.patch new file mode 100644 index 0000000..f74ea38 --- /dev/null +++ b/SOURCES/kvm-block-mirror-conservative-mirror_exit-refactor.patch @@ -0,0 +1,141 @@ +From 6b97fd0b69ecf0e1bebafa6bc607ae60b9ab252e Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:21 +0100 +Subject: [PATCH 18/28] block/mirror: conservative mirror_exit refactor + +RH-Author: John Snow +Message-id: <20180925223431.24791-16-jsnow@redhat.com> +Patchwork-id: 82270 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 15/25] block/mirror: conservative mirror_exit refactor +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +For purposes of minimum code movement, refactor the mirror_exit +callback to use the post-finalization callbacks in a trivial way. + +Signed-off-by: John Snow +Message-id: 20180906130225.5118-7-jsnow@redhat.com +Reviewed-by: Jeff Cody +Reviewed-by: Max Reitz +[mreitz: Added comment for the mirror_exit() function] +Signed-off-by: Max Reitz +(cherry picked from commit 737efc1eda23b904fbe0e66b37715fb0e5c3e58b) +Signed-off-by: John Snow + +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + block/mirror.c: context conflict on job properties +--- + block/mirror.c | 44 +++++++++++++++++++++++++++++++++----------- + 1 file changed, 33 insertions(+), 11 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 057db7c..163b1d4 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -71,6 +71,7 @@ typedef struct MirrorBlockJob { + int target_cluster_size; + int max_iov; + bool initial_zeroing_ongoing; ++ bool prepared; + } MirrorBlockJob; + + typedef struct MirrorOp { +@@ -480,7 +481,12 @@ static void mirror_wait_for_all_io(MirrorBlockJob *s) + } + } + +-static void mirror_exit(Job *job) ++/** ++ * mirror_exit_common: handle both abort() and prepare() cases. ++ * for .prepare, returns 0 on success and -errno on failure. ++ * for .abort cases, denoted by abort = true, MUST return 0. ++ */ ++static int mirror_exit_common(Job *job) + { + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); + BlockJob *bjob = &s->common; +@@ -489,7 +495,13 @@ static void mirror_exit(Job *job) + BlockDriverState *target_bs = blk_bs(s->target); + BlockDriverState *mirror_top_bs = s->mirror_top_bs; + Error *local_err = NULL; +- int ret = job->ret; ++ bool abort = job->ret < 0; ++ int ret = 0; ++ ++ if (s->prepared) { ++ return 0; ++ } ++ s->prepared = true; + + bdrv_release_dirty_bitmap(src, s->dirty_bitmap); + +@@ -514,7 +526,7 @@ static void mirror_exit(Job *job) + * required before it could become a backing file of target_bs. */ + bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, + &error_abort); +- if (ret == 0 && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) { ++ if (!abort && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) { + BlockDriverState *backing = s->is_none_mode ? src : s->base; + if (backing_bs(target_bs) != backing) { + bdrv_set_backing_hd(target_bs, backing, &local_err); +@@ -530,11 +542,8 @@ static void mirror_exit(Job *job) + aio_context_acquire(replace_aio_context); + } + +- if (s->should_complete && ret == 0) { +- BlockDriverState *to_replace = src; +- if (s->to_replace) { +- to_replace = s->to_replace; +- } ++ if (s->should_complete && !abort) { ++ BlockDriverState *to_replace = s->to_replace ?: src; + + if (bdrv_get_flags(target_bs) != bdrv_get_flags(to_replace)) { + bdrv_reopen(target_bs, bdrv_get_flags(to_replace), NULL); +@@ -581,7 +590,18 @@ static void mirror_exit(Job *job) + bdrv_unref(mirror_top_bs); + bdrv_unref(src); + +- job->ret = ret; ++ return ret; ++} ++ ++static int mirror_prepare(Job *job) ++{ ++ return mirror_exit_common(job); ++} ++ ++static void mirror_abort(Job *job) ++{ ++ int ret = mirror_exit_common(job); ++ assert(ret == 0); + } + + static void mirror_throttle(MirrorBlockJob *s) +@@ -986,7 +1006,8 @@ static const BlockJobDriver mirror_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = mirror_run, +- .exit = mirror_exit, ++ .prepare = mirror_prepare, ++ .abort = mirror_abort, + .pause = mirror_pause, + .complete = mirror_complete, + }, +@@ -1002,7 +1023,8 @@ static const BlockJobDriver commit_active_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = mirror_run, +- .exit = mirror_exit, ++ .prepare = mirror_prepare, ++ .abort = mirror_abort, + .pause = mirror_pause, + .complete = mirror_complete, + }, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-mirror-don-t-install-backing-chain-on-abort.patch b/SOURCES/kvm-block-mirror-don-t-install-backing-chain-on-abort.patch new file mode 100644 index 0000000..c066522 --- /dev/null +++ b/SOURCES/kvm-block-mirror-don-t-install-backing-chain-on-abort.patch @@ -0,0 +1,45 @@ +From 2bc98e9eaddaa3a7381b0f245ec37bb47f8baf81 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:20 +0100 +Subject: [PATCH 17/28] block/mirror: don't install backing chain on abort + +RH-Author: John Snow +Message-id: <20180925223431.24791-15-jsnow@redhat.com> +Patchwork-id: 82277 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 14/25] block/mirror: don't install backing chain on abort +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +In cases where we abort the block/mirror job, there's no point in +installing the new backing chain before we finish aborting. + +Signed-off-by: John Snow +Message-id: 20180906130225.5118-6-jsnow@redhat.com +Reviewed-by: Jeff Cody +Reviewed-by: Max Reitz +Signed-off-by: Max Reitz +(cherry picked from commit c2924ceaa7f1866148e2847c969fc1902a2524fa) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/mirror.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 9bed603..057db7c 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -514,7 +514,7 @@ static void mirror_exit(Job *job) + * required before it could become a backing file of target_bs. */ + bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, + &error_abort); +- if (s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) { ++ if (ret == 0 && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) { + BlockDriverState *backing = s->is_none_mode ? src : s->base; + if (backing_bs(target_bs) != backing) { + bdrv_set_backing_hd(target_bs, backing, &local_err); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-mirror-honor-ratelimit-again.patch b/SOURCES/kvm-block-mirror-honor-ratelimit-again.patch new file mode 100644 index 0000000..08f2bfe --- /dev/null +++ b/SOURCES/kvm-block-mirror-honor-ratelimit-again.patch @@ -0,0 +1,87 @@ +From c2e95d22694197ccf1fd74b8135f0f73f8ac623c Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:47:34 +0200 +Subject: [PATCH 033/268] block/mirror: honor ratelimit again + +RH-Author: Max Reitz +Message-id: <20180618144736.29873-2-mreitz@redhat.com> +Patchwork-id: 80747 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/3] block/mirror: honor ratelimit again +Bugzilla: 1572856 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +From: Stefan Hajnoczi + +Commit b76e4458b1eb3c32e9824fe6aa51f67d2b251748 ("block/mirror: change +the semantic of 'force' of block-job-cancel") accidentally removed the +ratelimit in the mirror job. + +Reintroduce the ratelimit but keep the block-job-cancel force=true +behavior that was added in commit +b76e4458b1eb3c32e9824fe6aa51f67d2b251748. + +Note that block_job_sleep_ns() returns immediately when the job is +cancelled. Therefore it's safe to unconditionally call +block_job_sleep_ns() - a cancelled job does not sleep. + +This commit fixes the non-deterministic qemu-iotests 185 output. The +test relies on the ratelimit to make the job sleep until the 'quit' +command is processed. Previously the job could complete before the +'quit' command was received since there was no ratelimit. + +Cc: Liang Li +Cc: Jeff Cody +Cc: Kevin Wolf +Signed-off-by: Stefan Hajnoczi +Message-id: 20180424123527.19168-1-stefanha@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit ddc4115efdfa6619689fe18871aa2d37890b3463) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/mirror.c | 8 +++++--- + tests/qemu-iotests/185.out | 2 +- + 2 files changed, 6 insertions(+), 4 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 820f512..9436a8d 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -868,12 +868,14 @@ static void coroutine_fn mirror_run(void *opaque) + } + + ret = 0; ++ ++ if (s->synced && !should_complete) { ++ delay_ns = (s->in_flight == 0 && cnt == 0 ? SLICE_TIME : 0); ++ } + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); ++ block_job_sleep_ns(&s->common, delay_ns); + if (block_job_is_cancelled(&s->common) && s->common.force) { + break; +- } else if (!should_complete) { +- delay_ns = (s->in_flight == 0 && cnt == 0 ? SLICE_TIME : 0); +- block_job_sleep_ns(&s->common, delay_ns); + } + s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + } +diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out +index 2c4b04d..992162f 100644 +--- a/tests/qemu-iotests/185.out ++++ b/tests/qemu-iotests/185.out +@@ -36,9 +36,9 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q + {"return": {}} + Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 + {"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} + + === Start backup job and exit qemu === +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-mirror-utilize-job_exit-shim.patch b/SOURCES/kvm-block-mirror-utilize-job_exit-shim.patch new file mode 100644 index 0000000..5448f5d --- /dev/null +++ b/SOURCES/kvm-block-mirror-utilize-job_exit-shim.patch @@ -0,0 +1,159 @@ +From 792e5326750b8e22627d993552af4a7906b49111 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:11 +0100 +Subject: [PATCH 08/28] block/mirror: utilize job_exit shim + +RH-Author: John Snow +Message-id: <20180925223431.24791-6-jsnow@redhat.com> +Patchwork-id: 82269 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 05/25] block/mirror: utilize job_exit shim +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Change the manual deferment to mirror_exit into the implicit +callback to job_exit and the mirror_exit callback. + +This does change the order of some bdrv_unref calls and job_completed, +but thanks to the new context in which we call .exit, this is safe to +defer the possible flushing of any nodes to the job_finalize_single +cleanup stage. + +Signed-off-by: John Snow +Message-id: 20180830015734.19765-6-jsnow@redhat.com +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 7b508f6b7a38a8d9729772fa6e525da883fb120b) +Signed-off-by: John Snow +--- + block/mirror.c | 30 +++++++++++------------------- + 1 file changed, 11 insertions(+), 19 deletions(-) + +-- +2.14.4 + +Signed-off-by: Danilo C. L. de Paula +--- + block/mirror.c | 30 +++++++++++------------------- + 1 file changed, 11 insertions(+), 19 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 3c330ee..2604f61 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -480,25 +480,21 @@ static void mirror_wait_for_all_io(MirrorBlockJob *s) + } + } + +-typedef struct { +- int ret; +-} MirrorExitData; +- +-static void mirror_exit(Job *job, void *opaque) ++static void mirror_exit(Job *job) + { + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); + BlockJob *bjob = &s->common; +- MirrorExitData *data = opaque; + AioContext *replace_aio_context = NULL; + BlockDriverState *src = s->source; + BlockDriverState *target_bs = blk_bs(s->target); + BlockDriverState *mirror_top_bs = s->mirror_top_bs; + Error *local_err = NULL; ++ int ret = job->ret; + + bdrv_release_dirty_bitmap(src, s->dirty_bitmap); + +- /* Make sure that the source BDS doesn't go away before we called +- * job_completed(). */ ++ /* Make sure that the source BDS doesn't go away during bdrv_replace_node, ++ * before we can call bdrv_drained_end */ + bdrv_ref(src); + bdrv_ref(mirror_top_bs); + bdrv_ref(target_bs); +@@ -524,7 +520,7 @@ static void mirror_exit(Job *job, void *opaque) + bdrv_set_backing_hd(target_bs, backing, &local_err); + if (local_err) { + error_report_err(local_err); +- data->ret = -EPERM; ++ ret = -EPERM; + } + } + } +@@ -534,7 +530,7 @@ static void mirror_exit(Job *job, void *opaque) + aio_context_acquire(replace_aio_context); + } + +- if (s->should_complete && data->ret == 0) { ++ if (s->should_complete && ret == 0) { + BlockDriverState *to_replace = src; + if (s->to_replace) { + to_replace = s->to_replace; +@@ -551,7 +547,7 @@ static void mirror_exit(Job *job, void *opaque) + bdrv_drained_end(target_bs); + if (local_err) { + error_report_err(local_err); +- data->ret = -EPERM; ++ ret = -EPERM; + } + } + if (s->to_replace) { +@@ -581,12 +577,11 @@ static void mirror_exit(Job *job, void *opaque) + blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); + blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort); + +- job_completed(job, data->ret); +- +- g_free(data); + bdrv_drained_end(src); + bdrv_unref(mirror_top_bs); + bdrv_unref(src); ++ ++ job->ret = ret; + } + + static void mirror_throttle(MirrorBlockJob *s) +@@ -686,7 +681,6 @@ static int mirror_flush(MirrorBlockJob *s) + static int coroutine_fn mirror_run(Job *job, Error **errp) + { + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); +- MirrorExitData *data; + BlockDriverState *bs = s->source; + BlockDriverState *target_bs = blk_bs(s->target); + bool need_drain = true; +@@ -896,14 +890,10 @@ immediate_exit: + g_free(s->in_flight_bitmap); + bdrv_dirty_iter_free(s->dbi); + +- data = g_malloc(sizeof(*data)); +- data->ret = ret; +- + if (need_drain) { + bdrv_drained_begin(bs); + } + +- job_defer_to_main_loop(&s->common.job, mirror_exit, data); + return ret; + } + +@@ -996,6 +986,7 @@ static const BlockJobDriver mirror_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = mirror_run, ++ .exit = mirror_exit, + .pause = mirror_pause, + .complete = mirror_complete, + }, +@@ -1011,6 +1002,7 @@ static const BlockJobDriver commit_active_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = mirror_run, ++ .exit = mirror_exit, + .pause = mirror_pause, + .complete = mirror_complete, + }, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-qcow2-bitmap-fix-free_bitmap_clusters.patch b/SOURCES/kvm-block-qcow2-bitmap-fix-free_bitmap_clusters.patch new file mode 100644 index 0000000..9e6084f --- /dev/null +++ b/SOURCES/kvm-block-qcow2-bitmap-fix-free_bitmap_clusters.patch @@ -0,0 +1,46 @@ +From ea69d5b7e4b59362f9d42b4e7e40f5c2fcc7b31b Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:44 +0200 +Subject: [PATCH 226/268] block/qcow2-bitmap: fix free_bitmap_clusters + +RH-Author: John Snow +Message-id: <20180718225511.14878-9-jsnow@redhat.com> +Patchwork-id: 81392 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 08/35] block/qcow2-bitmap: fix free_bitmap_clusters +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +This assert may fail, because bitmap_table is not initialized. Just +drop it, as it's obvious, that bitmap_table_load sets bitmap_table +parameter only when returning zero. + +Reported-by: Pavel Butsykin +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-id: 20180608101225.2575-1-vsementsov@virtuozzo.com +Signed-off-by: Max Reitz +(cherry picked from commit 7eb24009dbf8102fc27d5459bec1cb8a932540c4) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/qcow2-bitmap.c | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c +index 6e93ec4..3e4e4e4 100644 +--- a/block/qcow2-bitmap.c ++++ b/block/qcow2-bitmap.c +@@ -254,7 +254,6 @@ static int free_bitmap_clusters(BlockDriverState *bs, Qcow2BitmapTable *tb) + + ret = bitmap_table_load(bs, tb, &bitmap_table); + if (ret < 0) { +- assert(bitmap_table == NULL); + return ret; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-qcow2-improve-error-message-in-qcow2_inactivat.patch b/SOURCES/kvm-block-qcow2-improve-error-message-in-qcow2_inactivat.patch new file mode 100644 index 0000000..8a16a5f --- /dev/null +++ b/SOURCES/kvm-block-qcow2-improve-error-message-in-qcow2_inactivat.patch @@ -0,0 +1,48 @@ +From 821666d219f5caefa9b94c5fedeb850983616f24 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:24 +0000 +Subject: [PATCH 30/35] block/qcow2: improve error message in qcow2_inactivate + +RH-Author: John Snow +Message-id: <20181120181828.15132-21-jsnow@redhat.com> +Patchwork-id: 83070 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 20/24] block/qcow2: improve error message in qcow2_inactivate +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Signed-off-by: Vladimir Sementsov-Ogievskiy +[Maintainer edit -- touched up error message. --js] +Reviewed-by: John Snow +Signed-off-by: John Snow +(cherry picked from commit 132adb682098e9af40a2132ec4feec6850fce8cd) +Signed-off-by: John Snow + +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 1ea7203..5c5530d 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -2116,9 +2116,9 @@ static int qcow2_inactivate(BlockDriverState *bs) + qcow2_store_persistent_dirty_bitmaps(bs, &local_err); + if (local_err != NULL) { + result = -EINVAL; +- error_report_err(local_err); +- error_report("Persistent bitmaps are lost for node '%s'", +- bdrv_get_device_or_node_name(bs)); ++ error_reportf_err(local_err, "Lost persistent bitmaps during " ++ "inactivation of node '%s': ", ++ bdrv_get_device_or_node_name(bs)); + } + + ret = qcow2_cache_flush(bs, s->l2_table_cache); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-qdict-Clean-up-qdict_crumple-a-bit.patch b/SOURCES/kvm-block-qdict-Clean-up-qdict_crumple-a-bit.patch new file mode 100644 index 0000000..0566a2d --- /dev/null +++ b/SOURCES/kvm-block-qdict-Clean-up-qdict_crumple-a-bit.patch @@ -0,0 +1,91 @@ +From 5d1c329444f36191b0e868f2a43178d289ba1f59 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:24 +0200 +Subject: [PATCH 026/268] block-qdict: Clean up qdict_crumple() a bit + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-18-armbru@redhat.com> +Patchwork-id: 80729 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 17/23] block-qdict: Clean up qdict_crumple() a bit +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +When you mix scalar and non-scalar keys, whether you get an "already +set as scalar" or an "already set as dict" error depends on qdict +iteration order. Neither message makes much sense. Replace by +""Cannot mix scalar and non-scalar keys". This is similar to the +message we get for mixing list and non-list keys. + +I find qdict_crumple()'s first loop hard to understand. Rearrange it +and add a comment. + +Signed-off-by: Markus Armbruster +Signed-off-by: Kevin Wolf +(cherry picked from commit 3692b5d76819e573dedc9004c4b2b0e3dad83530) +Signed-off-by: Miroslav Rezanina +--- + qobject/block-qdict.c | 32 ++++++++++++++++---------------- + 1 file changed, 16 insertions(+), 16 deletions(-) + +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index a4e1c8d..36cf58a 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -403,7 +403,7 @@ static int qdict_is_list(QDict *maybe_list, Error **errp) + QObject *qdict_crumple(const QDict *src, Error **errp) + { + const QDictEntry *ent; +- QDict *two_level, *multi_level = NULL; ++ QDict *two_level, *multi_level = NULL, *child_dict; + QObject *dst = NULL, *child; + size_t i; + char *prefix = NULL; +@@ -422,28 +422,28 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + } + + qdict_split_flat_key(ent->key, &prefix, &suffix); +- + child = qdict_get(two_level, prefix); ++ child_dict = qobject_to(QDict, child); ++ ++ if (child) { ++ /* ++ * If @child_dict, then all previous keys with this prefix ++ * had a suffix. If @suffix, this one has one as well, ++ * and we're good, else there's a clash. ++ */ ++ if (!child_dict || !suffix) { ++ error_setg(errp, "Cannot mix scalar and non-scalar keys"); ++ goto error; ++ } ++ } ++ + if (suffix) { +- QDict *child_dict = qobject_to(QDict, child); + if (!child_dict) { +- if (child) { +- error_setg(errp, "Key %s prefix is already set as a scalar", +- prefix); +- goto error; +- } +- + child_dict = qdict_new(); +- qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); ++ qdict_put(two_level, prefix, child_dict); + } +- + qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); + } else { +- if (child) { +- error_setg(errp, "Key %s prefix is already set as a dict", +- prefix); +- goto error; +- } + qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-qdict-Simplify-qdict_flatten_qdict.patch b/SOURCES/kvm-block-qdict-Simplify-qdict_flatten_qdict.patch new file mode 100644 index 0000000..fc89bc0 --- /dev/null +++ b/SOURCES/kvm-block-qdict-Simplify-qdict_flatten_qdict.patch @@ -0,0 +1,82 @@ +From 26a9117f9c190c15102378ca14963c17f04c8e57 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:22 +0200 +Subject: [PATCH 024/268] block-qdict: Simplify qdict_flatten_qdict() + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-16-armbru@redhat.com> +Patchwork-id: 80723 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 15/23] block-qdict: Simplify qdict_flatten_qdict() +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +There's no need to restart the loop. We don't elsewhere, e.g. in +qdict_extract_subqdict(), qdict_join() and qemu_opts_absorb_qdict(). +Simplify accordingly. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit eb0e0f7d3d4a9c585421d05b19ca71df5d69fc47) +Signed-off-by: Miroslav Rezanina +--- + qobject/block-qdict.c | 18 +++--------------- + 1 file changed, 3 insertions(+), 15 deletions(-) + +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index 41f39ab..f32df34 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -89,16 +89,13 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + QObject *value; + const QDictEntry *entry, *next; + char *new_key; +- bool delete; + + entry = qdict_first(qdict); + + while (entry != NULL) { +- + next = qdict_next(qdict, entry); + value = qdict_entry_value(entry); + new_key = NULL; +- delete = false; + + if (prefix) { + new_key = g_strdup_printf("%s.%s", prefix, entry->key); +@@ -109,27 +106,18 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + * itself disappears. */ + qdict_flatten_qdict(qobject_to(QDict, value), target, + new_key ? new_key : entry->key); +- delete = true; ++ qdict_del(qdict, entry->key); + } else if (qobject_type(value) == QTYPE_QLIST) { + qdict_flatten_qlist(qobject_to(QList, value), target, + new_key ? new_key : entry->key); +- delete = true; ++ qdict_del(qdict, entry->key); + } else if (prefix) { + /* All other objects are moved to the target unchanged. */ + qdict_put_obj(target, new_key, qobject_ref(value)); +- delete = true; +- } +- +- g_free(new_key); +- +- if (delete) { + qdict_del(qdict, entry->key); +- +- /* Restart loop after modifying the iterated QDict */ +- entry = qdict_first(qdict); +- continue; + } + ++ g_free(new_key); + entry = next; + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-qdict-Simplify-qdict_is_list-some.patch b/SOURCES/kvm-block-qdict-Simplify-qdict_is_list-some.patch new file mode 100644 index 0000000..8e08741 --- /dev/null +++ b/SOURCES/kvm-block-qdict-Simplify-qdict_is_list-some.patch @@ -0,0 +1,69 @@ +From f4dbafc267d6ae5a1c359a026b87d4d423e5f607 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:25 +0200 +Subject: [PATCH 027/268] block-qdict: Simplify qdict_is_list() some + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-19-armbru@redhat.com> +Patchwork-id: 80741 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 18/23] block-qdict: Simplify qdict_is_list() some +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit c78b8cfbfd53737353dc94dfb99c57d72ce31ab5) +Signed-off-by: Miroslav Rezanina +--- + qobject/block-qdict.c | 27 +++++++++++---------------- + 1 file changed, 11 insertions(+), 16 deletions(-) + +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index 36cf58a..e51a3d2 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -317,27 +317,22 @@ static int qdict_is_list(QDict *maybe_list, Error **errp) + + for (ent = qdict_first(maybe_list); ent != NULL; + ent = qdict_next(maybe_list, ent)) { ++ int is_index = !qemu_strtoi64(ent->key, NULL, 10, &val); + +- if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { +- if (is_list == -1) { +- is_list = 1; +- } else if (!is_list) { +- error_setg(errp, +- "Cannot mix list and non-list keys"); +- return -1; +- } ++ if (is_list == -1) { ++ is_list = is_index; ++ } ++ ++ if (is_index != is_list) { ++ error_setg(errp, "Cannot mix list and non-list keys"); ++ return -1; ++ } ++ ++ if (is_index) { + len++; + if (val > max) { + max = val; + } +- } else { +- if (is_list == -1) { +- is_list = 0; +- } else if (is_list) { +- error_setg(errp, +- "Cannot mix list and non-list keys"); +- return -1; +- } + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-qdict-Tweak-qdict_flatten_qdict-qdict_flatten_.patch b/SOURCES/kvm-block-qdict-Tweak-qdict_flatten_qdict-qdict_flatten_.patch new file mode 100644 index 0000000..a525a84 --- /dev/null +++ b/SOURCES/kvm-block-qdict-Tweak-qdict_flatten_qdict-qdict_flatten_.patch @@ -0,0 +1,77 @@ +From c756588134a48f4d1e33c62117aadc749544c5e7 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:23 +0200 +Subject: [PATCH 025/268] block-qdict: Tweak qdict_flatten_qdict(), + qdict_flatten_qlist() + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-17-armbru@redhat.com> +Patchwork-id: 80725 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 16/23] block-qdict: Tweak qdict_flatten_qdict(), qdict_flatten_qlist() +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +qdict_flatten_qdict() skips copying scalars from @qdict to @target +when the two are the same. Fair enough, but it uses a non-obvious +test for "same". Replace it by the obvious one. While there, improve +comments. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit f1b34a248e9785e8cc0d28a1685d2cf4460bb256) +Signed-off-by: Miroslav Rezanina +--- + qobject/block-qdict.c | 14 +++++++++----- + 1 file changed, 9 insertions(+), 5 deletions(-) + +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +index f32df34..a4e1c8d 100644 +--- a/qobject/block-qdict.c ++++ b/qobject/block-qdict.c +@@ -71,12 +71,15 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) + value = qlist_entry_obj(entry); + new_key = g_strdup_printf("%s.%i", prefix, i); + ++ /* ++ * Flatten non-empty QDict and QList recursively into @target, ++ * copy other objects to @target ++ */ + if (qobject_type(value) == QTYPE_QDICT) { + qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); + } else if (qobject_type(value) == QTYPE_QLIST) { + qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + } else { +- /* All other types are moved to the target unchanged. */ + qdict_put_obj(target, new_key, qobject_ref(value)); + } + +@@ -101,9 +104,11 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + new_key = g_strdup_printf("%s.%s", prefix, entry->key); + } + ++ /* ++ * Flatten non-empty QDict and QList recursively into @target, ++ * copy other objects to @target ++ */ + if (qobject_type(value) == QTYPE_QDICT) { +- /* Entries of QDicts are processed recursively, the QDict object +- * itself disappears. */ + qdict_flatten_qdict(qobject_to(QDict, value), target, + new_key ? new_key : entry->key); + qdict_del(qdict, entry->key); +@@ -111,8 +116,7 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + qdict_flatten_qlist(qobject_to(QList, value), target, + new_key ? new_key : entry->key); + qdict_del(qdict, entry->key); +- } else if (prefix) { +- /* All other objects are moved to the target unchanged. */ ++ } else if (target != qdict) { + qdict_put_obj(target, new_key, qobject_ref(value)); + qdict_del(qdict, entry->key); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-quorum-Support-BDRV_REQ_WRITE_UNCHANGED.patch b/SOURCES/kvm-block-quorum-Support-BDRV_REQ_WRITE_UNCHANGED.patch new file mode 100644 index 0000000..ef6fd89 --- /dev/null +++ b/SOURCES/kvm-block-quorum-Support-BDRV_REQ_WRITE_UNCHANGED.patch @@ -0,0 +1,114 @@ +From a3f3c2cb19e33574f3566843cba7e3ad253dc9af Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:08 +0200 +Subject: [PATCH 041/268] block/quorum: Support BDRV_REQ_WRITE_UNCHANGED + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-7-mreitz@redhat.com> +Patchwork-id: 80770 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 06/10] block/quorum: Support BDRV_REQ_WRITE_UNCHANGED +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +We just need to forward it to quorum's children (except in case of a +rewrite because of corruption), but for that we first have to support +flags in child requests at all. + +Signed-off-by: Max Reitz +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Alberto Garcia +Message-id: 20180421132929.21610-6-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 1b1a920b713af6af795d49d0e3d2a8a65020bf82) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/quorum.c | 19 +++++++++++++------ + 1 file changed, 13 insertions(+), 6 deletions(-) + +diff --git a/block/quorum.c b/block/quorum.c +index f1f39ba..0d90a02 100644 +--- a/block/quorum.c ++++ b/block/quorum.c +@@ -116,6 +116,7 @@ struct QuorumAIOCB { + /* Request metadata */ + uint64_t offset; + uint64_t bytes; ++ int flags; + + QEMUIOVector *qiov; /* calling IOV */ + +@@ -158,7 +159,8 @@ static bool quorum_64bits_compare(QuorumVoteValue *a, QuorumVoteValue *b) + static QuorumAIOCB *quorum_aio_get(BlockDriverState *bs, + QEMUIOVector *qiov, + uint64_t offset, +- uint64_t bytes) ++ uint64_t bytes, ++ int flags) + { + BDRVQuorumState *s = bs->opaque; + QuorumAIOCB *acb = g_new(QuorumAIOCB, 1); +@@ -169,6 +171,7 @@ static QuorumAIOCB *quorum_aio_get(BlockDriverState *bs, + .bs = bs, + .offset = offset, + .bytes = bytes, ++ .flags = flags, + .qiov = qiov, + .votes.compare = quorum_sha256_compare, + .votes.vote_list = QLIST_HEAD_INITIALIZER(acb.votes.vote_list), +@@ -272,9 +275,11 @@ static void quorum_rewrite_entry(void *opaque) + BDRVQuorumState *s = acb->bs->opaque; + + /* Ignore any errors, it's just a correction attempt for already +- * corrupted data. */ ++ * corrupted data. ++ * Mask out BDRV_REQ_WRITE_UNCHANGED because this overwrites the ++ * area with different data from the other children. */ + bdrv_co_pwritev(s->children[co->idx], acb->offset, acb->bytes, +- acb->qiov, 0); ++ acb->qiov, acb->flags & ~BDRV_REQ_WRITE_UNCHANGED); + + /* Wake up the caller after the last rewrite */ + acb->rewrite_count--; +@@ -674,7 +679,7 @@ static int quorum_co_preadv(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, int flags) + { + BDRVQuorumState *s = bs->opaque; +- QuorumAIOCB *acb = quorum_aio_get(bs, qiov, offset, bytes); ++ QuorumAIOCB *acb = quorum_aio_get(bs, qiov, offset, bytes, flags); + int ret; + + acb->is_read = true; +@@ -700,7 +705,7 @@ static void write_quorum_entry(void *opaque) + + sacb->bs = s->children[i]->bs; + sacb->ret = bdrv_co_pwritev(s->children[i], acb->offset, acb->bytes, +- acb->qiov, 0); ++ acb->qiov, acb->flags); + if (sacb->ret == 0) { + acb->success_count++; + } else { +@@ -720,7 +725,7 @@ static int quorum_co_pwritev(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, int flags) + { + BDRVQuorumState *s = bs->opaque; +- QuorumAIOCB *acb = quorum_aio_get(bs, qiov, offset, bytes); ++ QuorumAIOCB *acb = quorum_aio_get(bs, qiov, offset, bytes, flags); + int i, ret; + + for (i = 0; i < s->num_children; i++) { +@@ -962,6 +967,8 @@ static int quorum_open(BlockDriverState *bs, QDict *options, int flags, + } + s->next_child_index = s->num_children; + ++ bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED; ++ + g_free(opened); + goto exit; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-rbd-Attempt-to-parse-legacy-filenames.patch b/SOURCES/kvm-block-rbd-Attempt-to-parse-legacy-filenames.patch new file mode 100644 index 0000000..85b38ef --- /dev/null +++ b/SOURCES/kvm-block-rbd-Attempt-to-parse-legacy-filenames.patch @@ -0,0 +1,120 @@ +From 0433f2d3bd818b83908636ae240ecbfd256e0a9c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:30:13 +0100 +Subject: [PATCH 2/4] block/rbd: Attempt to parse legacy filenames + +RH-Author: John Snow +Message-id: <20181010203015.11719-3-jsnow@redhat.com> +Patchwork-id: 82629 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/4] block/rbd: Attempt to parse legacy filenames +Bugzilla: 1635585 +RH-Acked-by: Markus Armbruster +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Thomas Huth + +From: Jeff Cody + +When we converted rbd to get rid of the older key/value-centric +encoding format, we broke compatibility with image files with backing +file strings encoded in the old format. + +This leaves a bit of an ugly conundrum, and a hacky solution. + +If the initial attempt to parse the "proper" options fails, it assumes +that we may have an older key/value encoded filename. Fall back to +attempting to parse the filename, and extract the required options from +it. If that fails, pass along the original error message. + +We do not support mixed modern usage alongside legacy keyvalue pair +usage. + +A deprecation warning has been added, although care should be taken +when actually deprecating since the impact is not limited to +commandline or qapi usage, but also opening existing images. + +Reviewed-by: Eric Blake +Signed-off-by: Jeff Cody +Message-id: 15b332e5432ad069441f7275a46080f465d789a0.1536704901.git.jcody@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit 084d1d13bdb753d558b991996e7686c077bd6d80) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/rbd.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- + 1 file changed, 52 insertions(+), 2 deletions(-) + +diff --git a/block/rbd.c b/block/rbd.c +index 1e4d339..ebe0701 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -671,6 +671,33 @@ static int qemu_rbd_convert_options(QDict *options, BlockdevOptionsRbd **opts, + return 0; + } + ++static int qemu_rbd_attempt_legacy_options(QDict *options, ++ BlockdevOptionsRbd **opts, ++ char **keypairs) ++{ ++ char *filename; ++ int r; ++ ++ filename = g_strdup(qdict_get_try_str(options, "filename")); ++ if (!filename) { ++ return -EINVAL; ++ } ++ qdict_del(options, "filename"); ++ ++ qemu_rbd_parse_filename(filename, options, NULL); ++ ++ /* keypairs freed by caller */ ++ *keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs")); ++ if (*keypairs) { ++ qdict_del(options, "=keyvalue-pairs"); ++ } ++ ++ r = qemu_rbd_convert_options(options, opts, NULL); ++ ++ g_free(filename); ++ return r; ++} ++ + static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) + { +@@ -693,8 +720,31 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + + r = qemu_rbd_convert_options(options, &opts, &local_err); + if (local_err) { +- error_propagate(errp, local_err); +- goto out; ++ /* If keypairs are present, that means some options are present in ++ * the modern option format. Don't attempt to parse legacy option ++ * formats, as we won't support mixed usage. */ ++ if (keypairs) { ++ error_propagate(errp, local_err); ++ goto out; ++ } ++ ++ /* If the initial attempt to convert and process the options failed, ++ * we may be attempting to open an image file that has the rbd options ++ * specified in the older format consisting of all key/value pairs ++ * encoded in the filename. Go ahead and attempt to parse the ++ * filename, and see if we can pull out the required options. */ ++ r = qemu_rbd_attempt_legacy_options(options, &opts, &keypairs); ++ if (r < 0) { ++ /* Propagate the original error, not the legacy parsing fallback ++ * error, as the latter was just a best-effort attempt. */ ++ error_propagate(errp, local_err); ++ goto out; ++ } ++ /* Take care whenever deciding to actually deprecate; once this ability ++ * is removed, we will not be able to open any images with legacy-styled ++ * backing image strings. */ ++ error_report("RBD options encoded in the filename as keyvalue pairs " ++ "is deprecated"); + } + + /* Remove the processed options from the QDict (the visitor processes +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-rbd-add-deprecation-documentation-for-filename.patch b/SOURCES/kvm-block-rbd-add-deprecation-documentation-for-filename.patch new file mode 100644 index 0000000..fcb78f3 --- /dev/null +++ b/SOURCES/kvm-block-rbd-add-deprecation-documentation-for-filename.patch @@ -0,0 +1,61 @@ +From 5c520d5572b0c7974bc71e89d5d5a77d9d0d9284 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:30:15 +0100 +Subject: [PATCH 3/4] block/rbd: add deprecation documentation for filename + keyvalue pairs + +RH-Author: John Snow +Message-id: <20181010203015.11719-5-jsnow@redhat.com> +Patchwork-id: 82625 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 4/4] block/rbd: add deprecation documentation for filename keyvalue pairs +Bugzilla: 1635585 +RH-Acked-by: Markus Armbruster +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Thomas Huth + +From: Jeff Cody + +Signed-off-by: Jeff Cody +Message-id: 647f5b5ab7efd8bf567a504c832b1d2d6f719b23.1536704901.git.jcody@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit 3bebd37e04f972775b1ece1bdda95451bc9fb14c) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + qemu-deprecated.texi: not yet factored out as of 2.12.0; + docs added to qemu-doc.texi instead. +Signed-off-by: John Snow +--- + qemu-doc.texi | 15 +++++++++++++++ + 1 file changed, 15 insertions(+) + +diff --git a/qemu-doc.texi b/qemu-doc.texi +index 985e0f2..0013170 100644 +--- a/qemu-doc.texi ++++ b/qemu-doc.texi +@@ -2982,6 +2982,21 @@ The ``xlnx-zcu102'' machine has the same features and capabilites in QEMU. + In order to prevent QEMU from automatically opening an image's backing + chain, use ``"backing": null'' instead. + ++@subsubsection rbd keyvalue pair encoded filenames: "" (since 3.1.0) ++ ++Options for ``rbd'' should be specified according to its runtime options, ++like other block drivers. Legacy parsing of keyvalue pair encoded ++filenames is useful to open images with the old format for backing files; ++These image files should be updated to use the current format. ++ ++Example of legacy encoding: ++ ++@code{json:@{"file.driver":"rbd", "file.filename":"rbd:rbd/name"@}} ++ ++The above, converted to the current supported format: ++ ++@code{json:@{"file.driver":"rbd", "file.pool":"rbd", "file.image":"name"@}} ++ + @node License + @appendix License + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-rbd-add-iotest-for-rbd-legacy-keyvalue-filenam.patch b/SOURCES/kvm-block-rbd-add-iotest-for-rbd-legacy-keyvalue-filenam.patch new file mode 100644 index 0000000..38a1ac8 --- /dev/null +++ b/SOURCES/kvm-block-rbd-add-iotest-for-rbd-legacy-keyvalue-filenam.patch @@ -0,0 +1,142 @@ +From dba144d5f04b39a0c95c74d6084ec1015f8d09e7 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:30:14 +0100 +Subject: [PATCH 4/4] block/rbd: add iotest for rbd legacy keyvalue filename + parsing + +RH-Author: John Snow +Message-id: <20181010203015.11719-4-jsnow@redhat.com> +Patchwork-id: 82628 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 3/4] block/rbd: add iotest for rbd legacy keyvalue filename parsing +Bugzilla: 1635585 +RH-Acked-by: Markus Armbruster +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Thomas Huth + +From: Jeff Cody + +This is a small test that will check for the ability to parse +both legacy and modern options for rbd. + +The way the test is set up is for failure to occur, but without +having to wait to timeout on a non-existent rbd server. The error +messages in the success path show that the arguments were parsed. + +The failure behavior prior to the patch series that has this test, is +qemu-img complaining about mandatory options (e.g. 'pool') not being +provided. + +Reviewed-by: Eric Blake +Signed-off-by: Jeff Cody +Message-id: f830580e339b974a83ed4870d11adcdc17f49a47.1536704901.git.jcody@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit 66e6a735e97450ac50fcaf40f78600c688534cae) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + tests/qemu-iotests/group: context (missing prior tests) +Signed-off-by: John Snow +--- + tests/qemu-iotests/231 | 62 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/231.out | 9 +++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 72 insertions(+) + create mode 100755 tests/qemu-iotests/231 + create mode 100644 tests/qemu-iotests/231.out + +diff --git a/tests/qemu-iotests/231 b/tests/qemu-iotests/231 +new file mode 100755 +index 0000000..3e28370 +--- /dev/null ++++ b/tests/qemu-iotests/231 +@@ -0,0 +1,62 @@ ++#!/bin/bash ++# ++# Test legacy and modern option parsing for rbd/ceph. This will not ++# actually connect to a ceph server, but rather looks for the appropriate ++# error message that indicates we parsed the options correctly. ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++# creator ++owner=jcody@redhat.com ++ ++seq=`basename $0` ++echo "QA output created by $seq" ++ ++here=`pwd` ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ rm "${BOGUS_CONF}" ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++ ++_supported_fmt generic ++_supported_proto rbd ++_supported_os Linux ++ ++BOGUS_CONF=${TEST_DIR}/ceph-$$.conf ++touch "${BOGUS_CONF}" ++ ++_filter_conf() ++{ ++ sed -e "s#$BOGUS_CONF#BOGUS_CONF#g" ++} ++ ++# We expect this to fail, with no monitor ip provided and a null conf file. Just want it ++# to fail in the right way. ++$QEMU_IMG info "json:{'file.driver':'rbd','file.filename':'rbd:rbd/bogus:conf=${BOGUS_CONF}'}" 2>&1 | _filter_conf ++$QEMU_IMG info "json:{'file.driver':'rbd','file.pool':'rbd','file.image':'bogus','file.conf':'${BOGUS_CONF}'}" 2>&1 | _filter_conf ++ ++# success, all done ++echo "*** done" ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/231.out b/tests/qemu-iotests/231.out +new file mode 100644 +index 0000000..579ba11 +--- /dev/null ++++ b/tests/qemu-iotests/231.out +@@ -0,0 +1,9 @@ ++QA output created by 231 ++qemu-img: RBD options encoded in the filename as keyvalue pairs is deprecated. Future versions may cease to parse these options in the future. ++unable to get monitor info from DNS SRV with service name: ceph-mon ++no monitors specified to connect to. ++qemu-img: Could not open 'json:{'file.driver':'rbd','file.filename':'rbd:rbd/bogus:conf=BOGUS_CONF'}': error connecting: No such file or directory ++unable to get monitor info from DNS SRV with service name: ceph-mon ++no monitors specified to connect to. ++qemu-img: Could not open 'json:{'file.driver':'rbd','file.pool':'rbd','file.image':'bogus','file.conf':'BOGUS_CONF'}': error connecting: No such file or directory ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 1cb2ccb..303daa5 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -223,3 +223,4 @@ + 223 rw auto quick + 226 auto quick + 229 auto quick ++231 auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-rbd-pull-out-qemu_rbd_convert_options.patch b/SOURCES/kvm-block-rbd-pull-out-qemu_rbd_convert_options.patch new file mode 100644 index 0000000..114619d --- /dev/null +++ b/SOURCES/kvm-block-rbd-pull-out-qemu_rbd_convert_options.patch @@ -0,0 +1,95 @@ +From 04d398c3738cfc37dba9dc5c1df4e9d41931cf63 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:30:12 +0100 +Subject: [PATCH 1/4] block/rbd: pull out qemu_rbd_convert_options + +RH-Author: John Snow +Message-id: <20181010203015.11719-2-jsnow@redhat.com> +Patchwork-id: 82627 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/4] block/rbd: pull out qemu_rbd_convert_options +Bugzilla: 1635585 +RH-Acked-by: Markus Armbruster +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Thomas Huth + +From: Jeff Cody + +Code movement to pull the conversion from Qdict to BlockdevOptionsRbd +into a helper function. + +Reviewed-by: Eric Blake +Reviewed-by: John Snow +Signed-off-by: Jeff Cody +Message-id: 5b49a980f2cde6610ab1df41bb0277d00b5db893.1536704901.git.jcody@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit f24b03b56cdb28d753b4ff9ae210d555f14cb0d8) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/rbd.c | 36 ++++++++++++++++++++++++------------ + 1 file changed, 24 insertions(+), 12 deletions(-) + +diff --git a/block/rbd.c b/block/rbd.c +index b93046b..1e4d339 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -648,12 +648,34 @@ failed_opts: + return r; + } + ++static int qemu_rbd_convert_options(QDict *options, BlockdevOptionsRbd **opts, ++ Error **errp) ++{ ++ Visitor *v; ++ Error *local_err = NULL; ++ ++ /* Convert the remaining options into a QAPI object */ ++ v = qobject_input_visitor_new_flat_confused(options, errp); ++ if (!v) { ++ return -EINVAL; ++ } ++ ++ visit_type_BlockdevOptionsRbd(v, NULL, opts, &local_err); ++ visit_free(v); ++ ++ if (local_err) { ++ error_propagate(errp, local_err); ++ return -EINVAL; ++ } ++ ++ return 0; ++} ++ + static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) + { + BDRVRBDState *s = bs->opaque; + BlockdevOptionsRbd *opts = NULL; +- Visitor *v; + const QDictEntry *e; + Error *local_err = NULL; + char *keypairs, *secretid; +@@ -669,19 +691,9 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + qdict_del(options, "password-secret"); + } + +- /* Convert the remaining options into a QAPI object */ +- v = qobject_input_visitor_new_flat_confused(options, errp); +- if (!v) { +- r = -EINVAL; +- goto out; +- } +- +- visit_type_BlockdevOptionsRbd(v, NULL, &opts, &local_err); +- visit_free(v); +- ++ r = qemu_rbd_convert_options(options, &opts, &local_err); + if (local_err) { + error_propagate(errp, local_err); +- r = -EINVAL; + goto out; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-remove-bdrv_dirty_bitmap_make_anon.patch b/SOURCES/kvm-block-remove-bdrv_dirty_bitmap_make_anon.patch new file mode 100644 index 0000000..337168f --- /dev/null +++ b/SOURCES/kvm-block-remove-bdrv_dirty_bitmap_make_anon.patch @@ -0,0 +1,82 @@ +From 5944f5aa58219777441158766624e571b53de954 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:05 +0000 +Subject: [PATCH 11/35] block: remove bdrv_dirty_bitmap_make_anon + +RH-Author: John Snow +Message-id: <20181120181828.15132-2-jsnow@redhat.com> +Patchwork-id: 83052 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 01/24] block: remove bdrv_dirty_bitmap_make_anon +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Paolo Bonzini + +All this function is doing will be repeated by +bdrv_do_release_matching_dirty_bitmap_locked, except +resetting bm->persistent. But even that does not matter +because the bitmap will be freed. + +Signed-off-by: Paolo Bonzini +Reviewed-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: Stefan Hajnoczi +Message-id: 20180323164254.26487-1-pbonzini@redhat.com +Signed-off-by: John Snow +(cherry picked from commit ab41fc4853cc0cf01ed4903ffe7c36e3768b538f) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 9 --------- + blockdev.c | 1 - + include/block/dirty-bitmap.h | 1 - + 3 files changed, 11 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index f580c1a..634f143 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -98,15 +98,6 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, const char *name) + } + + /* Called with BQL taken. */ +-void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap) +-{ +- assert(!bdrv_dirty_bitmap_frozen(bitmap)); +- g_free(bitmap->name); +- bitmap->name = NULL; +- bitmap->persistent = false; +-} +- +-/* Called with BQL taken. */ + BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs, + uint32_t granularity, + const char *name, +diff --git a/blockdev.c b/blockdev.c +index 70af034..69610e7 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2977,7 +2977,6 @@ void qmp_block_dirty_bitmap_remove(const char *node, const char *name, + } + } + +- bdrv_dirty_bitmap_make_anon(bitmap); + bdrv_release_dirty_bitmap(bs, bitmap); + } + +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index af9ba3c..bf68dd7 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -24,7 +24,6 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs, + void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap); + BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, + const char *name); +-void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap); + void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); + void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs); + void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-simplify-code-around-releasing-bitmaps.patch b/SOURCES/kvm-block-simplify-code-around-releasing-bitmaps.patch new file mode 100644 index 0000000..7f0cc91 --- /dev/null +++ b/SOURCES/kvm-block-simplify-code-around-releasing-bitmaps.patch @@ -0,0 +1,166 @@ +From 9d75983943f6510a804b2b9c48efa1eb3dcb41a2 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:06 +0000 +Subject: [PATCH 12/35] block: simplify code around releasing bitmaps + +RH-Author: John Snow +Message-id: <20181120181828.15132-3-jsnow@redhat.com> +Patchwork-id: 83053 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 02/24] block: simplify code around releasing bitmaps +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Paolo Bonzini + +QLIST_REMOVE does not require walking the list, and once the "bitmap" +argument is removed from bdrv_do_release_matching_dirty_bitmap_locked +the code simplifies a lot and it is worth inlining everything in the +callers of bdrv_do_release_matching_dirty_bitmap. + +Signed-off-by: Paolo Bonzini +Reviewed-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +Message-id: 20180326104037.6894-1-pbonzini@redhat.com +Signed-off-by: John Snow +(cherry picked from commit b133c27f5dc59969574b0715e5837d32c99caa86) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 84 ++++++++++++++++++++-------------------------------- + 1 file changed, 32 insertions(+), 52 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 634f143..dac8d74 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -256,49 +256,16 @@ void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap) + qemu_mutex_unlock(bitmap->mutex); + } + +-/* Called within bdrv_dirty_bitmap_lock..unlock */ +-static void bdrv_do_release_matching_dirty_bitmap_locked( +- BlockDriverState *bs, BdrvDirtyBitmap *bitmap, +- bool (*cond)(BdrvDirtyBitmap *bitmap)) ++/* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken. */ ++static void bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap) + { +- BdrvDirtyBitmap *bm, *next; +- +- QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) { +- if ((!bitmap || bm == bitmap) && (!cond || cond(bm))) { +- assert(!bm->active_iterators); +- assert(!bdrv_dirty_bitmap_frozen(bm)); +- assert(!bm->meta); +- QLIST_REMOVE(bm, list); +- hbitmap_free(bm->bitmap); +- g_free(bm->name); +- g_free(bm); +- +- if (bitmap) { +- return; +- } +- } +- } +- +- if (bitmap) { +- abort(); +- } +-} +- +-/* Called with BQL taken. */ +-static void bdrv_do_release_matching_dirty_bitmap( +- BlockDriverState *bs, BdrvDirtyBitmap *bitmap, +- bool (*cond)(BdrvDirtyBitmap *bitmap)) +-{ +- bdrv_dirty_bitmaps_lock(bs); +- bdrv_do_release_matching_dirty_bitmap_locked(bs, bitmap, cond); +- bdrv_dirty_bitmaps_unlock(bs); +-} +- +-/* Called within bdrv_dirty_bitmap_lock..unlock */ +-static void bdrv_release_dirty_bitmap_locked(BlockDriverState *bs, +- BdrvDirtyBitmap *bitmap) +-{ +- bdrv_do_release_matching_dirty_bitmap_locked(bs, bitmap, NULL); ++ assert(!bitmap->active_iterators); ++ assert(!bdrv_dirty_bitmap_frozen(bitmap)); ++ assert(!bitmap->meta); ++ QLIST_REMOVE(bitmap, list); ++ hbitmap_free(bitmap->bitmap); ++ g_free(bitmap->name); ++ g_free(bitmap); + } + + /** +@@ -351,7 +318,7 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs, + error_setg(errp, "Merging of parent and successor bitmap failed"); + return NULL; + } +- bdrv_release_dirty_bitmap_locked(bs, successor); ++ bdrv_release_dirty_bitmap_locked(successor); + parent->successor = NULL; + + return parent; +@@ -389,15 +356,12 @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes) + bdrv_dirty_bitmaps_unlock(bs); + } + +-static bool bdrv_dirty_bitmap_has_name(BdrvDirtyBitmap *bitmap) +-{ +- return !!bdrv_dirty_bitmap_name(bitmap); +-} +- + /* Called with BQL taken. */ + void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap) + { +- bdrv_do_release_matching_dirty_bitmap(bs, bitmap, NULL); ++ bdrv_dirty_bitmaps_lock(bs); ++ bdrv_release_dirty_bitmap_locked(bitmap); ++ bdrv_dirty_bitmaps_unlock(bs); + } + + /** +@@ -408,7 +372,15 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap) + */ + void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) + { +- bdrv_do_release_matching_dirty_bitmap(bs, NULL, bdrv_dirty_bitmap_has_name); ++ BdrvDirtyBitmap *bm, *next; ++ ++ bdrv_dirty_bitmaps_lock(bs); ++ QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) { ++ if (bdrv_dirty_bitmap_name(bm)) { ++ bdrv_release_dirty_bitmap_locked(bm); ++ } ++ } ++ bdrv_dirty_bitmaps_unlock(bs); + } + + /** +@@ -416,11 +388,19 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) + * bdrv_inactivate_recurse()). + * There must not be any frozen bitmaps attached. + * This function does not remove persistent bitmaps from the storage. ++ * Called with BQL taken. + */ + void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs) + { +- bdrv_do_release_matching_dirty_bitmap(bs, NULL, +- bdrv_dirty_bitmap_get_persistance); ++ BdrvDirtyBitmap *bm, *next; ++ ++ bdrv_dirty_bitmaps_lock(bs); ++ QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) { ++ if (bdrv_dirty_bitmap_get_persistance(bm)) { ++ bdrv_release_dirty_bitmap_locked(bm); ++ } ++ } ++ bdrv_dirty_bitmaps_unlock(bs); + } + + /** +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-split-flags-in-copy_range.patch b/SOURCES/kvm-block-split-flags-in-copy_range.patch new file mode 100644 index 0000000..75c5ec7 --- /dev/null +++ b/SOURCES/kvm-block-split-flags-in-copy_range.patch @@ -0,0 +1,466 @@ +From fa82fe1be0413e2dbc453aa5a2a930218e915907 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 24 Jul 2018 13:01:08 +0200 +Subject: [PATCH 238/268] block: split flags in copy_range + +RH-Author: John Snow +Message-id: <20180718225511.14878-21-jsnow@redhat.com> +Patchwork-id: 81415 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 20/35] block: split flags in copy_range +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Pass read flags and write flags separately. This is needed to handle +coming BDRV_REQ_NO_SERIALISING clearly in following patches. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit 67b51fb998c697afb5d744066fcbde53e04fe941) +Signed-off-by: John Snow +--- + block/backup.c | 2 +- + block/block-backend.c | 5 +++-- + block/file-posix.c | 21 +++++++++++-------- + block/io.c | 46 +++++++++++++++++++++++------------------- + block/iscsi.c | 9 ++++++--- + block/qcow2.c | 20 +++++++++--------- + block/raw-format.c | 24 ++++++++++++++-------- + include/block/block.h | 3 ++- + include/block/block_int.h | 14 +++++++++---- + include/sysemu/block-backend.h | 3 ++- + qemu-img.c | 2 +- + 11 files changed, 90 insertions(+), 59 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index adb3cbd..369155a 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -163,7 +163,7 @@ static int coroutine_fn backup_cow_with_offload(BackupBlockJob *job, + hbitmap_reset(job->copy_bitmap, start / job->cluster_size, + nr_clusters); + ret = blk_co_copy_range(blk, start, job->target, start, nbytes, +- is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0); ++ is_write_notifier ? BDRV_REQ_NO_SERIALISING : 0, 0); + if (ret < 0) { + trace_backup_do_cow_copy_range_fail(job, start, ret); + hbitmap_set(job->copy_bitmap, start / job->cluster_size, +diff --git a/block/block-backend.c b/block/block-backend.c +index f34e4c3..3554b7e 100644 +--- a/block/block-backend.c ++++ b/block/block-backend.c +@@ -2225,7 +2225,8 @@ void blk_unregister_buf(BlockBackend *blk, void *host) + + int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + BlockBackend *blk_out, int64_t off_out, +- int bytes, BdrvRequestFlags flags) ++ int bytes, BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + int r; + r = blk_check_byte_request(blk_in, off_in, bytes); +@@ -2238,5 +2239,5 @@ int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + } + return bdrv_co_copy_range(blk_in->root, off_in, + blk_out->root, off_out, +- bytes, flags); ++ bytes, read_flags, write_flags); + } +diff --git a/block/file-posix.c b/block/file-posix.c +index 06ec67d..c12cdb7 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -2476,18 +2476,23 @@ static void raw_abort_perm_update(BlockDriverState *bs) + raw_handle_perm_lock(bs, RAW_PL_ABORT, 0, 0, NULL); + } + +-static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, +- BdrvChild *src, uint64_t src_offset, +- BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++static int coroutine_fn raw_co_copy_range_from( ++ BlockDriverState *bs, BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, uint64_t bytes, ++ BdrvRequestFlags read_flags, BdrvRequestFlags write_flags) + { +- return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, flags); ++ return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, ++ read_flags, write_flags); + } + + static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, +- BdrvChild *src, uint64_t src_offset, +- BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + BDRVRawState *s = bs->opaque; + BDRVRawState *src_s; +diff --git a/block/io.c b/block/io.c +index f8de42f..2d04289 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -2841,13 +2841,11 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host) + } + } + +-static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, +- uint64_t src_offset, +- BdrvChild *dst, +- uint64_t dst_offset, +- uint64_t bytes, +- BdrvRequestFlags flags, +- bool recurse_src) ++static int coroutine_fn bdrv_co_copy_range_internal( ++ BdrvChild *src, uint64_t src_offset, BdrvChild *dst, ++ uint64_t dst_offset, uint64_t bytes, ++ BdrvRequestFlags read_flags, BdrvRequestFlags write_flags, ++ bool recurse_src) + { + BdrvTrackedRequest req; + int ret; +@@ -2860,8 +2858,8 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + if (ret) { + return ret; + } +- if (flags & BDRV_REQ_ZERO_WRITE) { +- return bdrv_co_pwrite_zeroes(dst, dst_offset, bytes, flags); ++ if (write_flags & BDRV_REQ_ZERO_WRITE) { ++ return bdrv_co_pwrite_zeroes(dst, dst_offset, bytes, write_flags); + } + + if (!src || !src->bs) { +@@ -2883,14 +2881,15 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + tracked_request_begin(&req, src->bs, src_offset, bytes, + BDRV_TRACKED_READ); + +- if (!(flags & BDRV_REQ_NO_SERIALISING)) { ++ if (!(read_flags & BDRV_REQ_NO_SERIALISING)) { + wait_serialising_requests(&req); + } + + ret = src->bs->drv->bdrv_co_copy_range_from(src->bs, + src, src_offset, + dst, dst_offset, +- bytes, flags); ++ bytes, ++ read_flags, write_flags); + + tracked_request_end(&req); + bdrv_dec_in_flight(src->bs); +@@ -2899,15 +2898,15 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + tracked_request_begin(&req, dst->bs, dst_offset, bytes, + BDRV_TRACKED_WRITE); + +- /* BDRV_REQ_NO_SERIALISING is only for read operation, +- * so we ignore it in flags. +- */ ++ /* BDRV_REQ_NO_SERIALISING is only for read operation */ ++ assert(!(write_flags & BDRV_REQ_NO_SERIALISING)); + wait_serialising_requests(&req); + + ret = dst->bs->drv->bdrv_co_copy_range_to(dst->bs, + src, src_offset, + dst, dst_offset, +- bytes, flags); ++ bytes, ++ read_flags, write_flags); + + tracked_request_end(&req); + bdrv_dec_in_flight(dst->bs); +@@ -2922,10 +2921,12 @@ static int coroutine_fn bdrv_co_copy_range_internal(BdrvChild *src, + * semantics. */ + int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset, +- bytes, flags, true); ++ bytes, read_flags, write_flags, true); + } + + /* Copy range from @src to @dst. +@@ -2934,19 +2935,22 @@ int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, + * semantics. */ + int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + return bdrv_co_copy_range_internal(src, src_offset, dst, dst_offset, +- bytes, flags, false); ++ bytes, read_flags, write_flags, false); + } + + int coroutine_fn bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ uint64_t bytes, BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + return bdrv_co_copy_range_from(src, src_offset, + dst, dst_offset, +- bytes, flags); ++ bytes, read_flags, write_flags); + } + + static void bdrv_parent_cb_resize(BlockDriverState *bs) +diff --git a/block/iscsi.c b/block/iscsi.c +index 5047e83..2b45458 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -2193,9 +2193,11 @@ static int coroutine_fn iscsi_co_copy_range_from(BlockDriverState *bs, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, +- BdrvRequestFlags flags) ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { +- return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, flags); ++ return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, ++ read_flags, write_flags); + } + + static struct scsi_task *iscsi_xcopy_task(int param_len) +@@ -2332,7 +2334,8 @@ static int coroutine_fn iscsi_co_copy_range_to(BlockDriverState *bs, + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, +- BdrvRequestFlags flags) ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + IscsiLun *dst_lun = dst->bs->opaque; + IscsiLun *src_lun; +diff --git a/block/qcow2.c b/block/qcow2.c +index e171a99..1ea7203 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3248,13 +3248,14 @@ static int coroutine_fn + qcow2_co_copy_range_from(BlockDriverState *bs, + BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ uint64_t bytes, BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + BDRVQcow2State *s = bs->opaque; + int ret; + unsigned int cur_bytes; /* number of bytes in current iteration */ + BdrvChild *child = NULL; +- BdrvRequestFlags cur_flags; ++ BdrvRequestFlags cur_write_flags; + + assert(!bs->encrypted); + qemu_co_mutex_lock(&s->lock); +@@ -3263,7 +3264,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, + uint64_t copy_offset = 0; + /* prepare next request */ + cur_bytes = MIN(bytes, INT_MAX); +- cur_flags = flags; ++ cur_write_flags = write_flags; + + ret = qcow2_get_cluster_offset(bs, src_offset, &cur_bytes, ©_offset); + if (ret < 0) { +@@ -3275,20 +3276,20 @@ qcow2_co_copy_range_from(BlockDriverState *bs, + if (bs->backing && bs->backing->bs) { + int64_t backing_length = bdrv_getlength(bs->backing->bs); + if (src_offset >= backing_length) { +- cur_flags |= BDRV_REQ_ZERO_WRITE; ++ cur_write_flags |= BDRV_REQ_ZERO_WRITE; + } else { + child = bs->backing; + cur_bytes = MIN(cur_bytes, backing_length - src_offset); + copy_offset = src_offset; + } + } else { +- cur_flags |= BDRV_REQ_ZERO_WRITE; ++ cur_write_flags |= BDRV_REQ_ZERO_WRITE; + } + break; + + case QCOW2_CLUSTER_ZERO_PLAIN: + case QCOW2_CLUSTER_ZERO_ALLOC: +- cur_flags |= BDRV_REQ_ZERO_WRITE; ++ cur_write_flags |= BDRV_REQ_ZERO_WRITE; + break; + + case QCOW2_CLUSTER_COMPRESSED: +@@ -3312,7 +3313,7 @@ qcow2_co_copy_range_from(BlockDriverState *bs, + ret = bdrv_co_copy_range_from(child, + copy_offset, + dst, dst_offset, +- cur_bytes, cur_flags); ++ cur_bytes, read_flags, cur_write_flags); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + goto out; +@@ -3333,7 +3334,8 @@ static int coroutine_fn + qcow2_co_copy_range_to(BlockDriverState *bs, + BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ uint64_t bytes, BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + BDRVQcow2State *s = bs->opaque; + int offset_in_cluster; +@@ -3377,7 +3379,7 @@ qcow2_co_copy_range_to(BlockDriverState *bs, + ret = bdrv_co_copy_range_to(src, src_offset, + bs->file, + cluster_offset + offset_in_cluster, +- cur_bytes, flags); ++ cur_bytes, read_flags, write_flags); + qemu_co_mutex_lock(&s->lock); + if (ret < 0) { + goto fail; +diff --git a/block/raw-format.c b/block/raw-format.c +index b78da56..a359198 100644 +--- a/block/raw-format.c ++++ b/block/raw-format.c +@@ -498,9 +498,13 @@ static int raw_probe_geometry(BlockDriverState *bs, HDGeometry *geo) + } + + static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, +- BdrvChild *src, uint64_t src_offset, +- BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + int ret; + +@@ -509,13 +513,17 @@ static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, + return ret; + } + return bdrv_co_copy_range_from(bs->file, src_offset, dst, dst_offset, +- bytes, flags); ++ bytes, read_flags, write_flags); + } + + static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, +- BdrvChild *src, uint64_t src_offset, +- BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags) ++ BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags) + { + int ret; + +@@ -524,7 +532,7 @@ static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, + return ret; + } + return bdrv_co_copy_range_to(src, src_offset, bs->file, dst_offset, bytes, +- flags); ++ read_flags, write_flags); + } + + BlockDriver bdrv_raw = { +diff --git a/include/block/block.h b/include/block/block.h +index 716fb5b..409db21 100644 +--- a/include/block/block.h ++++ b/include/block/block.h +@@ -661,5 +661,6 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host); + **/ + int coroutine_fn bdrv_co_copy_range(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags); ++ uint64_t bytes, BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags); + #endif +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 27e168f..b05cf11 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -220,7 +220,8 @@ struct BlockDriver { + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, +- BdrvRequestFlags flags); ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags); + + /* Map [offset, offset + nbytes) range onto a child of bs to copy data to, + * and invoke bdrv_co_copy_range_to(child, src, ...), or perform the copy +@@ -236,7 +237,8 @@ struct BlockDriver { + BdrvChild *dst, + uint64_t dst_offset, + uint64_t bytes, +- BdrvRequestFlags flags); ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags); + + /* + * Building block for bdrv_block_status[_above] and +@@ -1125,10 +1127,14 @@ void blockdev_close_all_bdrv_states(void); + + int coroutine_fn bdrv_co_copy_range_from(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags); ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags); + int coroutine_fn bdrv_co_copy_range_to(BdrvChild *src, uint64_t src_offset, + BdrvChild *dst, uint64_t dst_offset, +- uint64_t bytes, BdrvRequestFlags flags); ++ uint64_t bytes, ++ BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags); + + int refresh_total_sectors(BlockDriverState *bs, int64_t hint); + +diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-backend.h +index 8d03d49..830d873 100644 +--- a/include/sysemu/block-backend.h ++++ b/include/sysemu/block-backend.h +@@ -234,6 +234,7 @@ void blk_unregister_buf(BlockBackend *blk, void *host); + + int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, + BlockBackend *blk_out, int64_t off_out, +- int bytes, BdrvRequestFlags flags); ++ int bytes, BdrvRequestFlags read_flags, ++ BdrvRequestFlags write_flags); + + #endif +diff --git a/qemu-img.c b/qemu-img.c +index b2ef54e..eaee6d6 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -1786,7 +1786,7 @@ static int coroutine_fn convert_co_copy_range(ImgConvertState *s, int64_t sector + + ret = blk_co_copy_range(blk, offset, s->target, + sector_num << BDRV_SECTOR_BITS, +- n << BDRV_SECTOR_BITS, 0); ++ n << BDRV_SECTOR_BITS, 0, 0); + if (ret < 0) { + return ret; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-stream-add-block-job-creation-flags.patch b/SOURCES/kvm-block-stream-add-block-job-creation-flags.patch new file mode 100644 index 0000000..0f723de --- /dev/null +++ b/SOURCES/kvm-block-stream-add-block-job-creation-flags.patch @@ -0,0 +1,100 @@ +From c9f61825124efe29eb8d77e72a6c93961cb4c389 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:18 +0100 +Subject: [PATCH 15/28] block/stream: add block job creation flags + +RH-Author: John Snow +Message-id: <20180925223431.24791-13-jsnow@redhat.com> +Patchwork-id: 82263 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 12/25] block/stream: add block job creation flags +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Add support for taking and passing forward job creation flags. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +Message-id: 20180906130225.5118-4-jsnow@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit cf6320df581e6cbde6a95075266859a8f9ba9d55) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/stream.c | 5 +++-- + blockdev.c | 3 ++- + include/block/block_int.h | 5 ++++- + 3 files changed, 9 insertions(+), 4 deletions(-) + +diff --git a/block/stream.c b/block/stream.c +index 67e1e72..700eb23 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -214,7 +214,8 @@ static const BlockJobDriver stream_job_driver = { + + void stream_start(const char *job_id, BlockDriverState *bs, + BlockDriverState *base, const char *backing_file_str, +- int64_t speed, BlockdevOnError on_error, Error **errp) ++ int creation_flags, int64_t speed, ++ BlockdevOnError on_error, Error **errp) + { + StreamBlockJob *s; + BlockDriverState *iter; +@@ -236,7 +237,7 @@ void stream_start(const char *job_id, BlockDriverState *bs, + BLK_PERM_GRAPH_MOD, + BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED | + BLK_PERM_WRITE, +- speed, JOB_DEFAULT, NULL, NULL, errp); ++ speed, creation_flags, NULL, NULL, errp); + if (!s) { + goto fail; + } +diff --git a/blockdev.c b/blockdev.c +index a265dc7..90a50d0 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3234,6 +3234,7 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device, + AioContext *aio_context; + Error *local_err = NULL; + const char *base_name = NULL; ++ int job_flags = JOB_DEFAULT; + + if (!has_on_error) { + on_error = BLOCKDEV_ON_ERROR_REPORT; +@@ -3296,7 +3297,7 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device, + base_name = has_backing_file ? backing_file : base_name; + + stream_start(has_job_id ? job_id : NULL, bs, base_bs, base_name, +- has_speed ? speed : 0, on_error, &local_err); ++ job_flags, has_speed ? speed : 0, on_error, &local_err); + if (local_err) { + error_propagate(errp, local_err); + goto out; +diff --git a/include/block/block_int.h b/include/block/block_int.h +index 07517cf..341cbe8 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -929,6 +929,8 @@ int is_windows_drive(const char *filename); + * flatten the whole backing file chain onto @bs. + * @backing_file_str: The file name that will be written to @bs as the + * the new backing file if the job completes. Ignored if @base is %NULL. ++ * @creation_flags: Flags that control the behavior of the Job lifetime. ++ * See @BlockJobCreateFlags + * @speed: The maximum speed, in bytes per second, or 0 for unlimited. + * @on_error: The action to take upon error. + * @errp: Error object. +@@ -942,7 +944,8 @@ int is_windows_drive(const char *filename); + */ + void stream_start(const char *job_id, BlockDriverState *bs, + BlockDriverState *base, const char *backing_file_str, +- int64_t speed, BlockdevOnError on_error, Error **errp); ++ int creation_flags, int64_t speed, ++ BlockdevOnError on_error, Error **errp); + + /** + * commit_start: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-block-stream-refactor-stream-to-use-job-callbacks.patch b/SOURCES/kvm-block-stream-refactor-stream-to-use-job-callbacks.patch new file mode 100644 index 0000000..bfc40d1 --- /dev/null +++ b/SOURCES/kvm-block-stream-refactor-stream-to-use-job-callbacks.patch @@ -0,0 +1,94 @@ +From a71d2a15a8f4b4e637786345a4fcfc736a20dc4b Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:22 +0100 +Subject: [PATCH 19/28] block/stream: refactor stream to use job callbacks + +RH-Author: John Snow +Message-id: <20180925223431.24791-17-jsnow@redhat.com> +Patchwork-id: 82280 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 16/25] block/stream: refactor stream to use job callbacks +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-8-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 1b57488acf1beba157bcd8c926e596342bcb5c60) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/stream.c | 23 +++++++++++++++-------- + 1 file changed, 15 insertions(+), 8 deletions(-) + +diff --git a/block/stream.c b/block/stream.c +index 700eb23..81a7ec8 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -54,16 +54,16 @@ static int coroutine_fn stream_populate(BlockBackend *blk, + return blk_co_preadv(blk, offset, qiov.size, &qiov, BDRV_REQ_COPY_ON_READ); + } + +-static void stream_exit(Job *job) ++static int stream_prepare(Job *job) + { + StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); + BlockJob *bjob = &s->common; + BlockDriverState *bs = blk_bs(bjob->blk); + BlockDriverState *base = s->base; + Error *local_err = NULL; +- int ret = job->ret; ++ int ret = 0; + +- if (!job_is_cancelled(job) && bs->backing && ret == 0) { ++ if (bs->backing) { + const char *base_id = NULL, *base_fmt = NULL; + if (base) { + base_id = s->backing_file_str; +@@ -75,12 +75,19 @@ static void stream_exit(Job *job) + bdrv_set_backing_hd(bs, base, &local_err); + if (local_err) { + error_report_err(local_err); +- ret = -EPERM; +- goto out; ++ return -EPERM; + } + } + +-out: ++ return ret; ++} ++ ++static void stream_clean(Job *job) ++{ ++ StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); ++ BlockJob *bjob = &s->common; ++ BlockDriverState *bs = blk_bs(bjob->blk); ++ + /* Reopen the image back in read-only mode if necessary */ + if (s->bs_flags != bdrv_get_flags(bs)) { + /* Give up write permissions before making it read-only */ +@@ -89,7 +96,6 @@ out: + } + + g_free(s->backing_file_str); +- job->ret = ret; + } + + static int coroutine_fn stream_run(Job *job, Error **errp) +@@ -206,7 +212,8 @@ static const BlockJobDriver stream_job_driver = { + .job_type = JOB_TYPE_STREAM, + .free = block_job_free, + .run = stream_run, +- .exit = stream_exit, ++ .prepare = stream_prepare, ++ .clean = stream_clean, + .user_resume = block_job_user_resume, + .drain = block_job_drain, + }, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockdev-backup-add-bitmap-argument.patch b/SOURCES/kvm-blockdev-backup-add-bitmap-argument.patch new file mode 100644 index 0000000..b91bdfb --- /dev/null +++ b/SOURCES/kvm-blockdev-backup-add-bitmap-argument.patch @@ -0,0 +1,99 @@ +From 5d85a570191c28cc5c1f894c7fcfd1d14bf80033 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:10 +0000 +Subject: [PATCH 16/35] blockdev-backup: add bitmap argument + +RH-Author: John Snow +Message-id: <20181120181828.15132-7-jsnow@redhat.com> +Patchwork-id: 83060 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 06/24] blockdev-backup: add bitmap argument +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +It is only an oversight that we don't allow incremental backup with +blockdev-backup. Add the bitmap argument which enables this. + +Signed-off-by: John Snow +Message-id: 20180830211605.13683-2-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 945c1ee0cb7d29f2fd0fece2cd2b5329802de5e9) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 18 +++++++++++++++++- + qapi/block-core.json | 7 ++++++- + 2 files changed, 23 insertions(+), 2 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index 69610e7..a722188 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3654,6 +3654,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + BlockDriverState *bs; + BlockDriverState *target_bs; + Error *local_err = NULL; ++ BdrvDirtyBitmap *bmap = NULL; + AioContext *aio_context; + BlockJob *job = NULL; + int job_flags = JOB_DEFAULT; +@@ -3704,6 +3705,21 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + goto out; + } + } ++ ++ if (backup->has_bitmap) { ++ bmap = bdrv_find_dirty_bitmap(bs, backup->bitmap); ++ if (!bmap) { ++ error_setg(errp, "Bitmap '%s' could not be found", backup->bitmap); ++ goto out; ++ } ++ if (bdrv_dirty_bitmap_qmp_locked(bmap)) { ++ error_setg(errp, ++ "Bitmap '%s' is currently locked and cannot be used for " ++ "backup", backup->bitmap); ++ goto out; ++ } ++ } ++ + if (!backup->auto_finalize) { + job_flags |= JOB_MANUAL_FINALIZE; + } +@@ -3711,7 +3727,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + job_flags |= JOB_MANUAL_DISMISS; + } + job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, +- backup->sync, NULL, backup->compress, ++ backup->sync, bmap, backup->compress, + backup->on_source_error, backup->on_target_error, + job_flags, NULL, NULL, txn, &local_err); + if (local_err != NULL) { +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 5fb7983..a6c3977 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1293,6 +1293,10 @@ + # @speed: the maximum speed, in bytes per second. The default is 0, + # for unlimited. + # ++# @bitmap: the name of dirty bitmap if sync is "incremental". ++# Must be present if sync is "incremental", must NOT be present ++# otherwise. (Since 3.1) ++# + # @compress: true to compress data, if the target format supports it. + # (default: false) (since 2.8) + # +@@ -1325,7 +1329,8 @@ + ## + { 'struct': 'BlockdevBackup', + 'data': { '*job-id': 'str', 'device': 'str', 'target': 'str', +- 'sync': 'MirrorSyncMode', '*speed': 'int', '*compress': 'bool', ++ 'sync': 'MirrorSyncMode', '*speed': 'int', ++ '*bitmap': 'str', '*compress': 'bool', + '*on-source-error': 'BlockdevOnError', + '*on-target-error': 'BlockdevOnError', + '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockdev-document-transactional-shortcomings.patch b/SOURCES/kvm-blockdev-document-transactional-shortcomings.patch new file mode 100644 index 0000000..43b7d16 --- /dev/null +++ b/SOURCES/kvm-blockdev-document-transactional-shortcomings.patch @@ -0,0 +1,53 @@ +From 329a0c2d4967b6dac1530c010856119447385457 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:31 +0100 +Subject: [PATCH 28/28] blockdev: document transactional shortcomings + +RH-Author: John Snow +Message-id: <20180925223431.24791-26-jsnow@redhat.com> +Patchwork-id: 82286 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 25/25] blockdev: document transactional shortcomings +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Presently only the backup job really guarantees what one would consider +transactional semantics. To guard against someone helpfully adding them +in the future, document that there are shortcomings in the model that +would need to be audited at that time. + +Signed-off-by: John Snow +Message-id: 20180906130225.5118-17-jsnow@redhat.com +Reviewed-by: Jeff Cody +Reviewed-by: Max Reitz +Signed-off-by: Max Reitz +(cherry picked from commit 66da04ddd3dcb8c61ee664b6faced132da002006) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 8 +++++++- + 1 file changed, 7 insertions(+), 1 deletion(-) + +diff --git a/blockdev.c b/blockdev.c +index bf026d2..b8e4b0d 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2292,7 +2292,13 @@ static const BlkActionOps actions[] = { + .instance_size = sizeof(BlockDirtyBitmapState), + .prepare = block_dirty_bitmap_disable_prepare, + .abort = block_dirty_bitmap_disable_abort, +- } ++ }, ++ /* Where are transactions for MIRROR, COMMIT and STREAM? ++ * Although these blockjobs use transaction callbacks like the backup job, ++ * these jobs do not necessarily adhere to transaction semantics. ++ * These jobs may not fully undo all of their actions on abort, nor do they ++ * necessarily work in transactions with more than one job in them. ++ */ + }; + + /** +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockdev-enable-non-root-nodes-for-backup-source.patch b/SOURCES/kvm-blockdev-enable-non-root-nodes-for-backup-source.patch new file mode 100644 index 0000000..6c97a12 --- /dev/null +++ b/SOURCES/kvm-blockdev-enable-non-root-nodes-for-backup-source.patch @@ -0,0 +1,61 @@ +From a1618346e6f1b3eff463b57cc10380b880cd91ff Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:46 +0200 +Subject: [PATCH 228/268] blockdev: enable non-root nodes for backup source + +RH-Author: John Snow +Message-id: <20180718225511.14878-11-jsnow@redhat.com> +Patchwork-id: 81395 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 10/35] blockdev: enable non-root nodes for backup source +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +This is needed to implement the image-fleecing workflow where we +create a temporary node backed by an active node, then start +backupdev-backup sync=none from the active node to the temp node. + +In this case, the active node is now a root node AND a backing node, +so it no longer qualifies as a root node, so we loosen the restriction +on which nodes can be considered as the source for a backup. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: Eric Blake +Signed-off-by: John Snow +Message-Id: <20180702194630.9360-2-jsnow@redhat.com> +Signed-off-by: Eric Blake +(cherry picked from commit 930fe17f9900e9c879834f2d2e5c301992623332) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + blockdev.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index d425746..0bdd3b5 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -1969,7 +1969,7 @@ static void blockdev_backup_prepare(BlkActionState *common, Error **errp) + assert(common->action->type == TRANSACTION_ACTION_KIND_BLOCKDEV_BACKUP); + backup = common->action->u.blockdev_backup.data; + +- bs = qmp_get_root_bs(backup->device, errp); ++ bs = bdrv_lookup_bs(backup->device, backup->device, errp); + if (!bs) { + return; + } +@@ -3628,7 +3628,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + backup->compress = false; + } + +- bs = qmp_get_root_bs(backup->device, errp); ++ bs = bdrv_lookup_bs(backup->device, backup->device, errp); + if (!bs) { + return NULL; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockdev-rename-block-dirty-bitmap-clear-transaction.patch b/SOURCES/kvm-blockdev-rename-block-dirty-bitmap-clear-transaction.patch new file mode 100644 index 0000000..480b9df --- /dev/null +++ b/SOURCES/kvm-blockdev-rename-block-dirty-bitmap-clear-transaction.patch @@ -0,0 +1,65 @@ +From d0276297465c2de2c0a47e6f10f2749f17a26343 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:14 +0000 +Subject: [PATCH 20/35] blockdev: rename block-dirty-bitmap-clear transaction + handlers + +RH-Author: John Snow +Message-id: <20181120181828.15132-11-jsnow@redhat.com> +Patchwork-id: 83065 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 10/24] blockdev: rename block-dirty-bitmap-clear transaction handlers +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Rename block-dirty-bitmap-clear transaction handlers to reuse them for +x-block-dirty-bitmap-merge transaction in the following patch. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +(cherry picked from commit 5c4cf8b294ee65c049d6c40f5f6ff7c1befdb3d9) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 8 ++++---- + 1 file changed, 4 insertions(+), 4 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index a10fbbd..c4b9ddd 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2136,7 +2136,7 @@ static void block_dirty_bitmap_clear_prepare(BlkActionState *common, + bdrv_clear_dirty_bitmap(state->bitmap, &state->backup); + } + +-static void block_dirty_bitmap_clear_abort(BlkActionState *common) ++static void block_dirty_bitmap_restore(BlkActionState *common) + { + BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, + common, common); +@@ -2146,7 +2146,7 @@ static void block_dirty_bitmap_clear_abort(BlkActionState *common) + } + } + +-static void block_dirty_bitmap_clear_commit(BlkActionState *common) ++static void block_dirty_bitmap_free_backup(BlkActionState *common) + { + BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, + common, common); +@@ -2280,8 +2280,8 @@ static const BlkActionOps actions[] = { + [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_CLEAR] = { + .instance_size = sizeof(BlockDirtyBitmapState), + .prepare = block_dirty_bitmap_clear_prepare, +- .commit = block_dirty_bitmap_clear_commit, +- .abort = block_dirty_bitmap_clear_abort, ++ .commit = block_dirty_bitmap_free_backup, ++ .abort = block_dirty_bitmap_restore, + }, + [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_ENABLE] = { + .instance_size = sizeof(BlockDirtyBitmapState), +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Add-block_job_driver.patch b/SOURCES/kvm-blockjob-Add-block_job_driver.patch new file mode 100644 index 0000000..3475ed7 --- /dev/null +++ b/SOURCES/kvm-blockjob-Add-block_job_driver.patch @@ -0,0 +1,105 @@ +From b0753bf51b9b5dcd6645e3321aef01286a3584a9 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:56 +0200 +Subject: [PATCH 088/268] blockjob: Add block_job_driver() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-14-kwolf@redhat.com> +Patchwork-id: 81064 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 13/73] blockjob: Add block_job_driver() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +The backup block job directly accesses the driver field in BlockJob. Add +a wrapper for getting it. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit bd21935b50d100d8da8c05cd3c2009f0f3432cb4) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 8 +++++--- + blockjob.c | 5 +++++ + include/block/blockjob.h | 7 +++++++ + 3 files changed, 17 insertions(+), 3 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index cfdb6ec..e14d995 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -47,6 +47,8 @@ typedef struct BackupBlockJob { + HBitmap *copy_bitmap; + } BackupBlockJob; + ++static const BlockJobDriver backup_job_driver; ++ + /* See if in-flight requests overlap and wait for them to complete */ + static void coroutine_fn wait_for_overlapping_requests(BackupBlockJob *job, + int64_t start, +@@ -241,7 +243,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) + BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common); + int64_t len; + +- assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP); ++ assert(block_job_driver(job) == &backup_job_driver); + + if (backup_job->sync_mode != MIRROR_SYNC_MODE_NONE) { + error_setg(errp, "The backup job only supports block checkpoint in" +@@ -259,7 +261,7 @@ void backup_wait_for_overlapping_requests(BlockJob *job, int64_t offset, + BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common); + int64_t start, end; + +- assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP); ++ assert(block_job_driver(job) == &backup_job_driver); + + start = QEMU_ALIGN_DOWN(offset, backup_job->cluster_size); + end = QEMU_ALIGN_UP(offset + bytes, backup_job->cluster_size); +@@ -272,7 +274,7 @@ void backup_cow_request_begin(CowRequest *req, BlockJob *job, + BackupBlockJob *backup_job = container_of(job, BackupBlockJob, common); + int64_t start, end; + +- assert(job->driver->job_type == BLOCK_JOB_TYPE_BACKUP); ++ assert(block_job_driver(job) == &backup_job_driver); + + start = QEMU_ALIGN_DOWN(offset, backup_job->cluster_size); + end = QEMU_ALIGN_UP(offset + bytes, backup_job->cluster_size); +diff --git a/blockjob.c b/blockjob.c +index e30f5ec..112672a 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -373,6 +373,11 @@ static bool block_job_started(BlockJob *job) + return job->co; + } + ++const BlockJobDriver *block_job_driver(BlockJob *job) ++{ ++ return job->driver; ++} ++ + /** + * All jobs must allow a pause point before entering their job proper. This + * ensures that jobs can be paused prior to being started, then resumed later. +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 82f52f4..0f56f72 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -452,4 +452,11 @@ void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job); + */ + bool block_job_is_internal(BlockJob *job); + ++/** ++ * block_job_driver: ++ * ++ * Returns the driver associated with a block job. ++ */ ++const BlockJobDriver *block_job_driver(BlockJob *job); ++ + #endif +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Fix-assertion-in-block_job_finalize.patch b/SOURCES/kvm-blockjob-Fix-assertion-in-block_job_finalize.patch new file mode 100644 index 0000000..c2bab93 --- /dev/null +++ b/SOURCES/kvm-blockjob-Fix-assertion-in-block_job_finalize.patch @@ -0,0 +1,52 @@ +From eb103056587a7b8109fb720fd29afabbaad454ef Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:51 +0200 +Subject: [PATCH 083/268] blockjob: Fix assertion in block_job_finalize() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-9-kwolf@redhat.com> +Patchwork-id: 81060 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 08/73] blockjob: Fix assertion in block_job_finalize() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Every job gets a non-NULL job->txn on creation, but it doesn't +necessarily keep it until it is decommissioned: Finalising a job removes +it from its transaction. Therefore, calling 'blockdev-job-finalize' a +second time on an already concluded job causes an assertion failure. + +Remove job->txn from the assertion in block_job_finalize() to fix this. +block_job_do_finalize() still has the same assertion, but if a job is +already removed from its transaction, block_job_apply_verb() will +already error out before we run into that assertion. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 37aa19b63c46d933f1e4ea944cfccee54e2caf4a) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/blockjob.c b/blockjob.c +index 6746cad..0033b96 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -710,7 +710,7 @@ void block_job_complete(BlockJob *job, Error **errp) + + void block_job_finalize(BlockJob *job, Error **errp) + { +- assert(job && job->id && job->txn); ++ assert(job && job->id); + if (block_job_apply_verb(job, BLOCK_JOB_VERB_FINALIZE, errp)) { + return; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Implement-block_job_set_speed-centrally.patch b/SOURCES/kvm-blockjob-Implement-block_job_set_speed-centrally.patch new file mode 100644 index 0000000..9b3f7e7 --- /dev/null +++ b/SOURCES/kvm-blockjob-Implement-block_job_set_speed-centrally.patch @@ -0,0 +1,284 @@ +From bd3d631a2e5d982559f99bac55718813dde1b783 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:54 +0200 +Subject: [PATCH 086/268] blockjob: Implement block_job_set_speed() centrally + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-12-kwolf@redhat.com> +Patchwork-id: 81102 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 11/73] blockjob: Implement block_job_set_speed() centrally +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +All block job drivers support .set_speed and all of them duplicate the +same code to implement it. Move that code to blockjob.c and remove the +now useless callback. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 18bb69287ea522ab696e1bea818b93e5eaa85745) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 13 ------------- + block/commit.c | 14 -------------- + block/mirror.c | 26 ++++++-------------------- + block/stream.c | 14 -------------- + blockjob.c | 12 ++++-------- + include/block/blockjob.h | 2 ++ + include/block/blockjob_int.h | 3 --- + 7 files changed, 12 insertions(+), 72 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 7585c43..8468fd9 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -27,7 +27,6 @@ + #include "qemu/error-report.h" + + #define BACKUP_CLUSTER_SIZE_DEFAULT (1 << 16) +-#define SLICE_TIME 100000000ULL /* ns */ + + typedef struct BackupBlockJob { + BlockJob common; +@@ -190,17 +189,6 @@ static int coroutine_fn backup_before_write_notify( + return backup_do_cow(job, req->offset, req->bytes, NULL, true); + } + +-static void backup_set_speed(BlockJob *job, int64_t speed, Error **errp) +-{ +- BackupBlockJob *s = container_of(job, BackupBlockJob, common); +- +- if (speed < 0) { +- error_setg(errp, QERR_INVALID_PARAMETER, "speed"); +- return; +- } +- ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); +-} +- + static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) + { + BdrvDirtyBitmap *bm; +@@ -540,7 +528,6 @@ static const BlockJobDriver backup_job_driver = { + .instance_size = sizeof(BackupBlockJob), + .job_type = BLOCK_JOB_TYPE_BACKUP, + .start = backup_run, +- .set_speed = backup_set_speed, + .commit = backup_commit, + .abort = backup_abort, + .clean = backup_clean, +diff --git a/block/commit.c b/block/commit.c +index beec5d0..46cbeae 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -31,8 +31,6 @@ enum { + COMMIT_BUFFER_SIZE = 512 * 1024, /* in bytes */ + }; + +-#define SLICE_TIME 100000000ULL /* ns */ +- + typedef struct CommitBlockJob { + BlockJob common; + BlockDriverState *commit_top_bs; +@@ -216,21 +214,9 @@ out: + block_job_defer_to_main_loop(&s->common, commit_complete, data); + } + +-static void commit_set_speed(BlockJob *job, int64_t speed, Error **errp) +-{ +- CommitBlockJob *s = container_of(job, CommitBlockJob, common); +- +- if (speed < 0) { +- error_setg(errp, QERR_INVALID_PARAMETER, "speed"); +- return; +- } +- ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); +-} +- + static const BlockJobDriver commit_job_driver = { + .instance_size = sizeof(CommitBlockJob), + .job_type = BLOCK_JOB_TYPE_COMMIT, +- .set_speed = commit_set_speed, + .start = commit_run, + }; + +diff --git a/block/mirror.c b/block/mirror.c +index a515ec1..d5e0ff2 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -22,7 +22,6 @@ + #include "qemu/ratelimit.h" + #include "qemu/bitmap.h" + +-#define SLICE_TIME 100000000ULL /* ns */ + #define MAX_IN_FLIGHT 16 + #define MAX_IO_BYTES (1 << 20) /* 1 Mb */ + #define DEFAULT_MIRROR_BUF_SIZE (MAX_IN_FLIGHT * MAX_IO_BYTES) +@@ -596,7 +595,7 @@ static void mirror_throttle(MirrorBlockJob *s) + { + int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + +- if (now - s->last_pause_ns > SLICE_TIME) { ++ if (now - s->last_pause_ns > BLOCK_JOB_SLICE_TIME) { + s->last_pause_ns = now; + block_job_sleep_ns(&s->common, 0); + } else { +@@ -799,11 +798,10 @@ static void coroutine_fn mirror_run(void *opaque) + + /* Note that even when no rate limit is applied we need to yield + * periodically with no pending I/O so that bdrv_drain_all() returns. +- * We do so every SLICE_TIME nanoseconds, or when there is an error, +- * or when the source is clean, whichever comes first. +- */ ++ * We do so every BLKOCK_JOB_SLICE_TIME nanoseconds, or when there is ++ * an error, or when the source is clean, whichever comes first. */ + delta = qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - s->last_pause_ns; +- if (delta < SLICE_TIME && ++ if (delta < BLOCK_JOB_SLICE_TIME && + s->common.iostatus == BLOCK_DEVICE_IO_STATUS_OK) { + if (s->in_flight >= MAX_IN_FLIGHT || s->buf_free_count == 0 || + (cnt == 0 && s->in_flight > 0)) { +@@ -869,7 +867,8 @@ static void coroutine_fn mirror_run(void *opaque) + ret = 0; + + if (s->synced && !should_complete) { +- delay_ns = (s->in_flight == 0 && cnt == 0 ? SLICE_TIME : 0); ++ delay_ns = (s->in_flight == 0 && ++ cnt == 0 ? BLOCK_JOB_SLICE_TIME : 0); + } + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); + block_job_sleep_ns(&s->common, delay_ns); +@@ -908,17 +907,6 @@ immediate_exit: + block_job_defer_to_main_loop(&s->common, mirror_exit, data); + } + +-static void mirror_set_speed(BlockJob *job, int64_t speed, Error **errp) +-{ +- MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); +- +- if (speed < 0) { +- error_setg(errp, QERR_INVALID_PARAMETER, "speed"); +- return; +- } +- ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); +-} +- + static void mirror_complete(BlockJob *job, Error **errp) + { + MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); +@@ -1003,7 +991,6 @@ static void mirror_drain(BlockJob *job) + static const BlockJobDriver mirror_job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = BLOCK_JOB_TYPE_MIRROR, +- .set_speed = mirror_set_speed, + .start = mirror_run, + .complete = mirror_complete, + .pause = mirror_pause, +@@ -1014,7 +1001,6 @@ static const BlockJobDriver mirror_job_driver = { + static const BlockJobDriver commit_active_job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = BLOCK_JOB_TYPE_COMMIT, +- .set_speed = mirror_set_speed, + .start = mirror_run, + .complete = mirror_complete, + .pause = mirror_pause, +diff --git a/block/stream.c b/block/stream.c +index a1d4768..797d7c4 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -29,8 +29,6 @@ enum { + STREAM_BUFFER_SIZE = 512 * 1024, /* in bytes */ + }; + +-#define SLICE_TIME 100000000ULL /* ns */ +- + typedef struct StreamBlockJob { + BlockJob common; + BlockDriverState *base; +@@ -210,21 +208,9 @@ out: + block_job_defer_to_main_loop(&s->common, stream_complete, data); + } + +-static void stream_set_speed(BlockJob *job, int64_t speed, Error **errp) +-{ +- StreamBlockJob *s = container_of(job, StreamBlockJob, common); +- +- if (speed < 0) { +- error_setg(errp, QERR_INVALID_PARAMETER, "speed"); +- return; +- } +- ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); +-} +- + static const BlockJobDriver stream_job_driver = { + .instance_size = sizeof(StreamBlockJob), + .job_type = BLOCK_JOB_TYPE_STREAM, +- .set_speed = stream_set_speed, + .start = stream_run, + }; + +diff --git a/blockjob.c b/blockjob.c +index d0a2ac5..0f7214c 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -667,22 +667,18 @@ static void block_job_completed_txn_success(BlockJob *job) + + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) + { +- Error *local_err = NULL; + int64_t old_speed = job->speed; + +- if (!job->driver->set_speed) { +- error_setg(errp, QERR_UNSUPPORTED); +- return; +- } + if (block_job_apply_verb(job, BLOCK_JOB_VERB_SET_SPEED, errp)) { + return; + } +- job->driver->set_speed(job, speed, &local_err); +- if (local_err) { +- error_propagate(errp, local_err); ++ if (speed < 0) { ++ error_setg(errp, QERR_INVALID_PARAMETER, "speed"); + return; + } + ++ ratelimit_set_speed(&job->limit, speed, BLOCK_JOB_SLICE_TIME); ++ + job->speed = speed; + if (speed && speed <= old_speed) { + return; +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 22bf418..82f52f4 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -29,6 +29,8 @@ + #include "block/block.h" + #include "qemu/ratelimit.h" + ++#define BLOCK_JOB_SLICE_TIME 100000000ULL /* ns */ ++ + typedef struct BlockJobDriver BlockJobDriver; + typedef struct BlockJobTxn BlockJobTxn; + +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index d5a515d..ad510d5 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -41,9 +41,6 @@ struct BlockJobDriver { + /** String describing the operation, part of query-block-jobs QMP API */ + BlockJobType job_type; + +- /** Optional callback for job types that support setting a speed limit */ +- void (*set_speed)(BlockJob *job, int64_t speed, Error **errp); +- + /** Mandatory: Entrypoint for the Coroutine. */ + CoroutineEntry *start; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Improve-BlockJobInfo.offset-len-documentati.patch b/SOURCES/kvm-blockjob-Improve-BlockJobInfo.offset-len-documentati.patch new file mode 100644 index 0000000..4098b72 --- /dev/null +++ b/SOURCES/kvm-blockjob-Improve-BlockJobInfo.offset-len-documentati.patch @@ -0,0 +1,61 @@ +From 82c4f0c7c192af2c7db4b7d4ca6ab3454e4b25f5 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:58 +0200 +Subject: [PATCH 090/268] blockjob: Improve BlockJobInfo.offset/len + documentation + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-16-kwolf@redhat.com> +Patchwork-id: 81087 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 15/73] blockjob: Improve BlockJobInfo.offset/len documentation +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Clarify that len is just an estimation of the end value of offset, and +that offset increases monotonically while len can change arbitrarily. + +While touching the documentation of offset, move it directly after len +to match the order of the declaration below. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: John Snow +(cherry picked from commit a81e0a825e3b89039a427bca037112f461b95fec) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + qapi/block-core.json | 9 ++++++--- + 1 file changed, 6 insertions(+), 3 deletions(-) + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 31bf64a..2b4566f 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1148,7 +1148,12 @@ + # @device: The job identifier. Originally the device name but other + # values are allowed since QEMU 2.7 + # +-# @len: the maximum progress value ++# @len: Estimated @offset value at the completion of the job. This value can ++# arbitrarily change while the job is running, in both directions. ++# ++# @offset: Progress made until now. The unit is arbitrary and the value can ++# only meaningfully be used for the ratio of @offset to @len. The ++# value is monotonically increasing. + # + # @busy: false if the job is known to be in a quiescent state, with + # no pending I/O. Since 1.3. +@@ -1156,8 +1161,6 @@ + # @paused: whether the job is paused or, if @busy is true, will + # pause itself as soon as possible. Since 1.3. + # +-# @offset: the current progress value +-# + # @speed: the rate limit, bytes per second + # + # @io-status: the status of the job (since 1.3) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Introduce-block_job_ratelimit_get_delay.patch b/SOURCES/kvm-blockjob-Introduce-block_job_ratelimit_get_delay.patch new file mode 100644 index 0000000..de4bbae --- /dev/null +++ b/SOURCES/kvm-blockjob-Introduce-block_job_ratelimit_get_delay.patch @@ -0,0 +1,154 @@ +From a266d0b6b07ddf185c5f2eaa05bda38e279ac36f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:55 +0200 +Subject: [PATCH 087/268] blockjob: Introduce block_job_ratelimit_get_delay() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-13-kwolf@redhat.com> +Patchwork-id: 81079 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 12/73] blockjob: Introduce block_job_ratelimit_get_delay() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This gets us rid of more direct accesses to BlockJob fields from the +job drivers. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit dee81d5111ff0e24ac63ab0dbbd19e84c2f87904) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 18 +++++++----------- + block/commit.c | 4 ++-- + block/mirror.c | 5 +---- + block/stream.c | 4 ++-- + blockjob.c | 9 +++++++++ + include/block/blockjob_int.h | 8 ++++++++ + 6 files changed, 29 insertions(+), 19 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 8468fd9..cfdb6ec 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -325,21 +325,17 @@ static void backup_complete(BlockJob *job, void *opaque) + + static bool coroutine_fn yield_and_check(BackupBlockJob *job) + { ++ uint64_t delay_ns; ++ + if (block_job_is_cancelled(&job->common)) { + return true; + } + +- /* we need to yield so that bdrv_drain_all() returns. +- * (without, VM does not reboot) +- */ +- if (job->common.speed) { +- uint64_t delay_ns = ratelimit_calculate_delay(&job->common.limit, +- job->bytes_read); +- job->bytes_read = 0; +- block_job_sleep_ns(&job->common, delay_ns); +- } else { +- block_job_sleep_ns(&job->common, 0); +- } ++ /* We need to yield even for delay_ns = 0 so that bdrv_drain_all() can ++ * return. Without a yield, the VM would not reboot. */ ++ delay_ns = block_job_ratelimit_get_delay(&job->common, job->bytes_read); ++ job->bytes_read = 0; ++ block_job_sleep_ns(&job->common, delay_ns); + + if (block_job_is_cancelled(&job->common)) { + return true; +diff --git a/block/commit.c b/block/commit.c +index 46cbeae..ba5df6a 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -197,8 +197,8 @@ static void coroutine_fn commit_run(void *opaque) + /* Publish progress */ + block_job_progress_update(&s->common, n); + +- if (copy && s->common.speed) { +- delay_ns = ratelimit_calculate_delay(&s->common.limit, n); ++ if (copy) { ++ delay_ns = block_job_ratelimit_get_delay(&s->common, n); + } else { + delay_ns = 0; + } +diff --git a/block/mirror.c b/block/mirror.c +index d5e0ff2..a4197bb 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -447,10 +447,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) + assert(io_bytes); + offset += io_bytes; + nb_chunks -= DIV_ROUND_UP(io_bytes, s->granularity); +- if (s->common.speed) { +- delay_ns = ratelimit_calculate_delay(&s->common.limit, +- io_bytes_acct); +- } ++ delay_ns = block_job_ratelimit_get_delay(&s->common, io_bytes_acct); + } + return delay_ns; + } +diff --git a/block/stream.c b/block/stream.c +index 797d7c4..df9660d 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -185,8 +185,8 @@ static void coroutine_fn stream_run(void *opaque) + + /* Publish progress */ + block_job_progress_update(&s->common, n); +- if (copy && s->common.speed) { +- delay_ns = ratelimit_calculate_delay(&s->common.limit, n); ++ if (copy) { ++ delay_ns = block_job_ratelimit_get_delay(&s->common, n); + } else { + delay_ns = 0; + } +diff --git a/blockjob.c b/blockjob.c +index 0f7214c..e30f5ec 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -688,6 +688,15 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) + block_job_enter_cond(job, block_job_timer_pending); + } + ++int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) ++{ ++ if (!job->speed) { ++ return 0; ++ } ++ ++ return ratelimit_calculate_delay(&job->limit, n); ++} ++ + void block_job_complete(BlockJob *job, Error **errp) + { + /* Should not be reachable via external interface for internal jobs */ +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index ad510d5..62ec964 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -166,6 +166,14 @@ void block_job_sleep_ns(BlockJob *job, int64_t ns); + void block_job_yield(BlockJob *job); + + /** ++ * block_job_ratelimit_get_delay: ++ * ++ * Calculate and return delay for the next request in ns. See the documentation ++ * of ratelimit_calculate_delay() for details. ++ */ ++int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n); ++ ++/** + * block_job_early_fail: + * @bs: The block device. + * +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Lie-better-in-child_job_drained_poll.patch b/SOURCES/kvm-blockjob-Lie-better-in-child_job_drained_poll.patch new file mode 100644 index 0000000..08dd702 --- /dev/null +++ b/SOURCES/kvm-blockjob-Lie-better-in-child_job_drained_poll.patch @@ -0,0 +1,104 @@ +From 7c16384f8ce4d46d6baa11db376c488ed8478744 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:06 +0100 +Subject: [PATCH 40/49] blockjob: Lie better in child_job_drained_poll() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-28-kwolf@redhat.com> +Patchwork-id: 82616 +O-Subject: [RHEL-8 qemu-kvm PATCH 37/44] blockjob: Lie better in child_job_drained_poll() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Block jobs claim in .drained_poll() that they are in a quiescent state +as soon as job->deferred_to_main_loop is true. This is obviously wrong, +they still have a completion BH to run. We only get away with this +because commit 91af091f923 added an unconditional aio_poll(false) to the +drain functions, but this is bypassing the regular drain mechanisms. + +However, just removing this and telling that the job is still active +doesn't work either: The completion callbacks themselves call drain +functions (directly, or indirectly with bdrv_reopen), so they would +deadlock then. + +As a better lie, tell that the job is active as long as the BH is +pending, but falsely call it quiescent from the point in the BH when the +completion callback is called. At this point, nested drain calls won't +deadlock because they ignore the job, and outer drains will wait for the +job to really reach a quiescent state because the callback is already +running. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit b5a7a0573530698ee448b063ac01d485e30446bd) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + blockjob.c | 2 +- + include/qemu/job.h | 3 +++ + job.c | 11 ++++++++++- + 3 files changed, 14 insertions(+), 2 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 8d27e8e..617d86f 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -164,7 +164,7 @@ static bool child_job_drained_poll(BdrvChild *c) + /* An inactive or completed job doesn't have any pending requests. Jobs + * with !job->busy are either already paused or have a pause point after + * being reentered, so no job driver code will run before they pause. */ +- if (!job->busy || job_is_completed(job) || job->deferred_to_main_loop) { ++ if (!job->busy || job_is_completed(job)) { + return false; + } + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 35ac7a9..d1710f3 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -76,6 +76,9 @@ typedef struct Job { + * Set to false by the job while the coroutine has yielded and may be + * re-entered by job_enter(). There may still be I/O or event loop activity + * pending. Accessed under block_job_mutex (in blockjob.c). ++ * ++ * When the job is deferred to the main loop, busy is true as long as the ++ * bottom half is still pending. + */ + bool busy; + +diff --git a/job.c b/job.c +index 47b5a11..42af9e2 100644 +--- a/job.c ++++ b/job.c +@@ -852,7 +852,16 @@ static void job_exit(void *opaque) + AioContext *ctx = job->aio_context; + + aio_context_acquire(ctx); ++ ++ /* This is a lie, we're not quiescent, but still doing the completion ++ * callbacks. However, completion callbacks tend to involve operations that ++ * drain block nodes, and if .drained_poll still returned true, we would ++ * deadlock. */ ++ job->busy = false; ++ job_event_idle(job); ++ + job_completed(job); ++ + aio_context_release(ctx); + } + +@@ -867,8 +876,8 @@ static void coroutine_fn job_co_entry(void *opaque) + assert(job && job->driver && job->driver->run); + job_pause_point(job); + job->ret = job->driver->run(job, &job->err); +- job_event_idle(job); + job->deferred_to_main_loop = true; ++ job->busy = true; + aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Move-RateLimit-to-BlockJob.patch b/SOURCES/kvm-blockjob-Move-RateLimit-to-BlockJob.patch new file mode 100644 index 0000000..987a381 --- /dev/null +++ b/SOURCES/kvm-blockjob-Move-RateLimit-to-BlockJob.patch @@ -0,0 +1,179 @@ +From 2544eb52961510e97b6bda33f9d9e71e01e5143d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:53 +0200 +Subject: [PATCH 085/268] blockjob: Move RateLimit to BlockJob + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-11-kwolf@redhat.com> +Patchwork-id: 81088 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 10/73] blockjob: Move RateLimit to BlockJob +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Every block job has a RateLimit, and they all do the exact same thing +with it, so it should be common infrastructure. Move the struct field +for a start. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit f05fee508f538ca262d2ab19bcd8772196efe848) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 5 ++--- + block/commit.c | 5 ++--- + block/mirror.c | 6 +++--- + block/stream.c | 5 ++--- + include/block/blockjob.h | 4 ++++ + 5 files changed, 13 insertions(+), 12 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 5d95805..7585c43 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -35,7 +35,6 @@ typedef struct BackupBlockJob { + /* bitmap for sync=incremental */ + BdrvDirtyBitmap *sync_bitmap; + MirrorSyncMode sync_mode; +- RateLimit limit; + BlockdevOnError on_source_error; + BlockdevOnError on_target_error; + CoRwlock flush_rwlock; +@@ -199,7 +198,7 @@ static void backup_set_speed(BlockJob *job, int64_t speed, Error **errp) + error_setg(errp, QERR_INVALID_PARAMETER, "speed"); + return; + } +- ratelimit_set_speed(&s->limit, speed, SLICE_TIME); ++ ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); + } + + static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) +@@ -346,7 +345,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) + * (without, VM does not reboot) + */ + if (job->common.speed) { +- uint64_t delay_ns = ratelimit_calculate_delay(&job->limit, ++ uint64_t delay_ns = ratelimit_calculate_delay(&job->common.limit, + job->bytes_read); + job->bytes_read = 0; + block_job_sleep_ns(&job->common, delay_ns); +diff --git a/block/commit.c b/block/commit.c +index 50b191c..beec5d0 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -35,7 +35,6 @@ enum { + + typedef struct CommitBlockJob { + BlockJob common; +- RateLimit limit; + BlockDriverState *commit_top_bs; + BlockBackend *top; + BlockBackend *base; +@@ -201,7 +200,7 @@ static void coroutine_fn commit_run(void *opaque) + block_job_progress_update(&s->common, n); + + if (copy && s->common.speed) { +- delay_ns = ratelimit_calculate_delay(&s->limit, n); ++ delay_ns = ratelimit_calculate_delay(&s->common.limit, n); + } else { + delay_ns = 0; + } +@@ -225,7 +224,7 @@ static void commit_set_speed(BlockJob *job, int64_t speed, Error **errp) + error_setg(errp, QERR_INVALID_PARAMETER, "speed"); + return; + } +- ratelimit_set_speed(&s->limit, speed, SLICE_TIME); ++ ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); + } + + static const BlockJobDriver commit_job_driver = { +diff --git a/block/mirror.c b/block/mirror.c +index ed711b5..a515ec1 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -36,7 +36,6 @@ typedef struct MirrorBuffer { + + typedef struct MirrorBlockJob { + BlockJob common; +- RateLimit limit; + BlockBackend *target; + BlockDriverState *mirror_top_bs; + BlockDriverState *source; +@@ -450,7 +449,8 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) + offset += io_bytes; + nb_chunks -= DIV_ROUND_UP(io_bytes, s->granularity); + if (s->common.speed) { +- delay_ns = ratelimit_calculate_delay(&s->limit, io_bytes_acct); ++ delay_ns = ratelimit_calculate_delay(&s->common.limit, ++ io_bytes_acct); + } + } + return delay_ns; +@@ -916,7 +916,7 @@ static void mirror_set_speed(BlockJob *job, int64_t speed, Error **errp) + error_setg(errp, QERR_INVALID_PARAMETER, "speed"); + return; + } +- ratelimit_set_speed(&s->limit, speed, SLICE_TIME); ++ ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); + } + + static void mirror_complete(BlockJob *job, Error **errp) +diff --git a/block/stream.c b/block/stream.c +index 8369852..a1d4768 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -33,7 +33,6 @@ enum { + + typedef struct StreamBlockJob { + BlockJob common; +- RateLimit limit; + BlockDriverState *base; + BlockdevOnError on_error; + char *backing_file_str; +@@ -189,7 +188,7 @@ static void coroutine_fn stream_run(void *opaque) + /* Publish progress */ + block_job_progress_update(&s->common, n); + if (copy && s->common.speed) { +- delay_ns = ratelimit_calculate_delay(&s->limit, n); ++ delay_ns = ratelimit_calculate_delay(&s->common.limit, n); + } else { + delay_ns = 0; + } +@@ -219,7 +218,7 @@ static void stream_set_speed(BlockJob *job, int64_t speed, Error **errp) + error_setg(errp, QERR_INVALID_PARAMETER, "speed"); + return; + } +- ratelimit_set_speed(&s->limit, speed, SLICE_TIME); ++ ratelimit_set_speed(&s->common.limit, speed, SLICE_TIME); + } + + static const BlockJobDriver stream_job_driver = { +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index a2cc522..22bf418 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -27,6 +27,7 @@ + #define BLOCKJOB_H + + #include "block/block.h" ++#include "qemu/ratelimit.h" + + typedef struct BlockJobDriver BlockJobDriver; + typedef struct BlockJobTxn BlockJobTxn; +@@ -118,6 +119,9 @@ typedef struct BlockJob { + /** Speed that was set with @block_job_set_speed. */ + int64_t speed; + ++ /** Rate limiting data structure for implementing @speed. */ ++ RateLimit limit; ++ + /** The completion function that will be called when the job completes. */ + BlockCompletionFunc *cb; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Remove-BlockJob.driver.patch b/SOURCES/kvm-blockjob-Remove-BlockJob.driver.patch new file mode 100644 index 0000000..b711b34 --- /dev/null +++ b/SOURCES/kvm-blockjob-Remove-BlockJob.driver.patch @@ -0,0 +1,96 @@ +From 2278ae1bb1dbbf0038e4bcc96ecdd875498c5eec Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:34 +0200 +Subject: [PATCH 126/268] blockjob: Remove BlockJob.driver + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-52-kwolf@redhat.com> +Patchwork-id: 81123 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 51/73] blockjob: Remove BlockJob.driver +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +BlockJob.driver is redundant with Job.driver and only used in very few +places any more. Remove it. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 9f6bb4c004a6458227b9eec6aff3f79afe159699) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 17 ++++++++++------- + include/block/blockjob.h | 3 --- + 2 files changed, 10 insertions(+), 10 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 5c8ff6f..0306533 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -103,10 +103,12 @@ static void block_job_attached_aio_context(AioContext *new_context, + void *opaque) + { + BlockJob *job = opaque; ++ const JobDriver *drv = job->job.driver; ++ BlockJobDriver *bjdrv = container_of(drv, BlockJobDriver, job_driver); + + job->job.aio_context = new_context; +- if (job->driver->attached_aio_context) { +- job->driver->attached_aio_context(job, new_context); ++ if (bjdrv->attached_aio_context) { ++ bjdrv->attached_aio_context(job, new_context); + } + + job_resume(&job->job); +@@ -115,10 +117,12 @@ static void block_job_attached_aio_context(AioContext *new_context, + void block_job_drain(Job *job) + { + BlockJob *bjob = container_of(job, BlockJob, job); ++ const JobDriver *drv = job->driver; ++ BlockJobDriver *bjdrv = container_of(drv, BlockJobDriver, job_driver); + + blk_drain(bjob->blk); +- if (bjob->driver->drain) { +- bjob->driver->drain(bjob); ++ if (bjdrv->drain) { ++ bjdrv->drain(bjob); + } + } + +@@ -201,7 +205,7 @@ bool block_job_is_internal(BlockJob *job) + + const BlockJobDriver *block_job_driver(BlockJob *job) + { +- return job->driver; ++ return container_of(job->job.driver, BlockJobDriver, job_driver); + } + + /* Assumes the job_mutex is held */ +@@ -386,8 +390,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + assert(job->job.driver->user_resume == &block_job_user_resume); + assert(job->job.driver->drain == &block_job_drain); + +- job->driver = driver; +- job->blk = blk; ++ job->blk = blk; + + job->finalize_cancelled_notifier.notify = block_job_event_cancelled; + job->finalize_completed_notifier.notify = block_job_event_completed; +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 3021d11..32c00b7 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -43,9 +43,6 @@ typedef struct BlockJob { + /** Data belonging to the generic Job infrastructure */ + Job job; + +- /** The job type, including the job vtable. */ +- const BlockJobDriver *driver; +- + /** The block device on which the job is operating. */ + BlockBackend *blk; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Split-block_job_event_pending.patch b/SOURCES/kvm-blockjob-Split-block_job_event_pending.patch new file mode 100644 index 0000000..1a6b578 --- /dev/null +++ b/SOURCES/kvm-blockjob-Split-block_job_event_pending.patch @@ -0,0 +1,92 @@ +From 6d0b0e3738595df279028c62682f033d941cc97c Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:14 +0200 +Subject: [PATCH 106/268] blockjob: Split block_job_event_pending() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-32-kwolf@redhat.com> +Patchwork-id: 81071 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 31/73] blockjob: Split block_job_event_pending() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +block_job_event_pending() doesn't only send a QMP event, but it also +transitions to the PENDING state. Split the function so that we get one +part only sending the event (like other block_job_event_* functions) and +another part that does the state transition. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 5d4f376998bc6b01402b90634385b082b2eb5c5b) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 27 ++++++++++++++++++--------- + 1 file changed, 18 insertions(+), 9 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index d9d8ff7..b4334fb 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -38,7 +38,7 @@ + + static void block_job_event_cancelled(BlockJob *job); + static void block_job_event_completed(BlockJob *job, const char *msg); +-static int block_job_event_pending(BlockJob *job); ++static void block_job_event_pending(BlockJob *job); + + /* Transactional group of block jobs */ + struct BlockJobTxn { +@@ -500,6 +500,15 @@ static void block_job_do_finalize(BlockJob *job) + } + } + ++static int block_job_transition_to_pending(BlockJob *job) ++{ ++ job_state_transition(&job->job, JOB_STATUS_PENDING); ++ if (!job->job.auto_finalize) { ++ block_job_event_pending(job); ++ } ++ return 0; ++} ++ + static void block_job_completed_txn_success(BlockJob *job) + { + BlockJobTxn *txn = job->txn; +@@ -518,7 +527,7 @@ static void block_job_completed_txn_success(BlockJob *job) + assert(other_job->ret == 0); + } + +- block_job_txn_apply(txn, block_job_event_pending, false); ++ block_job_txn_apply(txn, block_job_transition_to_pending, false); + + /* If no jobs need manual finalization, automatically do so */ + if (block_job_txn_apply(txn, block_job_needs_finalize, false) == 0) { +@@ -733,15 +742,15 @@ static void block_job_event_completed(BlockJob *job, const char *msg) + &error_abort); + } + +-static int block_job_event_pending(BlockJob *job) ++static void block_job_event_pending(BlockJob *job) + { +- job_state_transition(&job->job, JOB_STATUS_PENDING); +- if (!job->job.auto_finalize && !block_job_is_internal(job)) { +- qapi_event_send_block_job_pending(job_type(&job->job), +- job->job.id, +- &error_abort); ++ if (block_job_is_internal(job)) { ++ return; + } +- return 0; ++ ++ qapi_event_send_block_job_pending(job_type(&job->job), ++ job->job.id, ++ &error_abort); + } + + /* +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Update-block-job-pause-resume-documentation.patch b/SOURCES/kvm-blockjob-Update-block-job-pause-resume-documentation.patch new file mode 100644 index 0000000..88bb27a --- /dev/null +++ b/SOURCES/kvm-blockjob-Update-block-job-pause-resume-documentation.patch @@ -0,0 +1,54 @@ +From ec5e9da1b57dddafbc4b2d643f8e93d485d362c8 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:57 +0200 +Subject: [PATCH 089/268] blockjob: Update block-job-pause/resume documentation + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-15-kwolf@redhat.com> +Patchwork-id: 81069 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 14/73] blockjob: Update block-job-pause/resume documentation +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Commit 0ec4dfb8d changed block-job_pause/resume so that they return an +error if they don't do anything because the job is already +paused/running. It forgot to update the documentation, so do that now. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: John Snow +(cherry picked from commit cd44d96be90e7767c6fb8f33b90939eb58814956) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + qapi/block-core.json | 5 ++--- + 1 file changed, 2 insertions(+), 3 deletions(-) + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 28b4964..31bf64a 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -2338,8 +2338,7 @@ + # + # This command returns immediately after marking the active background block + # operation for pausing. It is an error to call this command if no +-# operation is in progress. Pausing an already paused job has no cumulative +-# effect; a single block-job-resume command will resume the job. ++# operation is in progress or if the job is already paused. + # + # The operation will pause as soon as possible. No event is emitted when + # the operation is actually paused. Cancelling a paused job automatically +@@ -2363,7 +2362,7 @@ + # + # This command returns immediately after resuming a paused background block + # operation. It is an error to call this command if no operation is in +-# progress. Resuming an already running job is not an error. ++# progress or if the job is not paused. + # + # This command also clears the error status of the job. + # +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Wake-up-BDS-when-job-becomes-idle.patch b/SOURCES/kvm-blockjob-Wake-up-BDS-when-job-becomes-idle.patch new file mode 100644 index 0000000..eabd73e --- /dev/null +++ b/SOURCES/kvm-blockjob-Wake-up-BDS-when-job-becomes-idle.patch @@ -0,0 +1,161 @@ +From c48802abf2f0912ce3c34775587f674b037939ac Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:56 +0100 +Subject: [PATCH 30/49] blockjob: Wake up BDS when job becomes idle + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-18-kwolf@redhat.com> +Patchwork-id: 82610 +O-Subject: [RHEL-8 qemu-kvm PATCH 27/44] blockjob: Wake up BDS when job becomes idle +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +In the context of draining a BDS, the .drained_poll callback of block +jobs is called. If this returns true (i.e. there is still some activity +pending), the drain operation may call aio_poll() with blocking=true to +wait for completion. + +As soon as the pending activity is completed and the job finally arrives +in a quiescent state (i.e. its coroutine either yields with busy=false +or terminates), the block job must notify the aio_poll() loop to wake +up, otherwise we get a deadlock if both are running in different +threads. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +Reviewed-by: Max Reitz +(cherry picked from commit 34dc97b9a0e592bc466bdb0bbfe45d77304a72b6) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + blockjob.c | 18 ++++++++++++++++++ + include/block/blockjob.h | 13 +++++++++++++ + include/qemu/job.h | 3 +++ + job.c | 7 +++++++ + 4 files changed, 41 insertions(+) + +diff --git a/blockjob.c b/blockjob.c +index be5903a..8d27e8e 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -221,6 +221,22 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, + return 0; + } + ++void block_job_wakeup_all_bdrv(BlockJob *job) ++{ ++ GSList *l; ++ ++ for (l = job->nodes; l; l = l->next) { ++ BdrvChild *c = l->data; ++ bdrv_wakeup(c->bs); ++ } ++} ++ ++static void block_job_on_idle(Notifier *n, void *opaque) ++{ ++ BlockJob *job = opaque; ++ block_job_wakeup_all_bdrv(job); ++} ++ + bool block_job_is_internal(BlockJob *job) + { + return (job->job.id == NULL); +@@ -419,6 +435,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->finalize_completed_notifier.notify = block_job_event_completed; + job->pending_notifier.notify = block_job_event_pending; + job->ready_notifier.notify = block_job_event_ready; ++ job->idle_notifier.notify = block_job_on_idle; + + notifier_list_add(&job->job.on_finalize_cancelled, + &job->finalize_cancelled_notifier); +@@ -426,6 +443,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + &job->finalize_completed_notifier); + notifier_list_add(&job->job.on_pending, &job->pending_notifier); + notifier_list_add(&job->job.on_ready, &job->ready_notifier); ++ notifier_list_add(&job->job.on_idle, &job->idle_notifier); + + error_setg(&job->blocker, "block device is in use by block job: %s", + job_type_str(&job->job)); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 32c00b7..2290bbb 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -70,6 +70,9 @@ typedef struct BlockJob { + /** Called when the job transitions to READY */ + Notifier ready_notifier; + ++ /** Called when the job coroutine yields or terminates */ ++ Notifier idle_notifier; ++ + /** BlockDriverStates that are involved in this block job */ + GSList *nodes; + } BlockJob; +@@ -119,6 +122,16 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, + void block_job_remove_all_bdrv(BlockJob *job); + + /** ++ * block_job_wakeup_all_bdrv: ++ * @job: The block job ++ * ++ * Calls bdrv_wakeup() for all BlockDriverStates that have been added to the ++ * job. This function is to be called whenever child_job_drained_poll() would ++ * go from true to false to notify waiting drain requests. ++ */ ++void block_job_wakeup_all_bdrv(BlockJob *job); ++ ++/** + * block_job_set_speed: + * @job: The job to set the speed for. + * @speed: The new value +diff --git a/include/qemu/job.h b/include/qemu/job.h +index fdaa06f..407d549 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -156,6 +156,9 @@ typedef struct Job { + /** Notifiers called when the job transitions to READY */ + NotifierList on_ready; + ++ /** Notifiers called when the job coroutine yields or terminates */ ++ NotifierList on_idle; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + +diff --git a/job.c b/job.c +index db53163..5a0ccc7 100644 +--- a/job.c ++++ b/job.c +@@ -397,6 +397,11 @@ static void job_event_ready(Job *job) + notifier_list_notify(&job->on_ready, job); + } + ++static void job_event_idle(Job *job) ++{ ++ notifier_list_notify(&job->on_idle, job); ++} ++ + void job_enter_cond(Job *job, bool(*fn)(Job *job)) + { + if (!job_started(job)) { +@@ -442,6 +447,7 @@ static void coroutine_fn job_do_yield(Job *job, uint64_t ns) + timer_mod(&job->sleep_timer, ns); + } + job->busy = false; ++ job_event_idle(job); + job_unlock(); + qemu_coroutine_yield(); + +@@ -860,6 +866,7 @@ static void coroutine_fn job_co_entry(void *opaque) + assert(job && job->driver && job->driver->run); + job_pause_point(job); + job->ret = job->driver->run(job, &job->err); ++ job_event_idle(job); + job->deferred_to_main_loop = true; + aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-Wrappers-for-progress-counter-access.patch b/SOURCES/kvm-blockjob-Wrappers-for-progress-counter-access.patch new file mode 100644 index 0000000..e473e93 --- /dev/null +++ b/SOURCES/kvm-blockjob-Wrappers-for-progress-counter-access.patch @@ -0,0 +1,311 @@ +From cc0cf0f2e402c4885d37683a9c79b7dbe85ad378 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:52 +0200 +Subject: [PATCH 084/268] blockjob: Wrappers for progress counter access + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-10-kwolf@redhat.com> +Patchwork-id: 81078 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 09/73] blockjob: Wrappers for progress counter access +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Block job drivers are not expected to mess with the internals of the +BlockJob object, so provide wrapper functions for one of the cases where +they still do it: Updating the progress counter. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 05df8a6a2b4e36e8d69de2130e616d5ac28e8837) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 22 +++++++++++++--------- + block/commit.c | 16 ++++++++-------- + block/mirror.c | 11 +++++------ + block/stream.c | 14 ++++++++------ + blockjob.c | 10 ++++++++++ + include/block/blockjob.h | 19 +++++++++++++++++++ + 6 files changed, 63 insertions(+), 29 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 453cd62..5d95805 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -39,6 +39,7 @@ typedef struct BackupBlockJob { + BlockdevOnError on_source_error; + BlockdevOnError on_target_error; + CoRwlock flush_rwlock; ++ uint64_t len; + uint64_t bytes_read; + int64_t cluster_size; + bool compress; +@@ -118,7 +119,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, + + trace_backup_do_cow_process(job, start); + +- n = MIN(job->cluster_size, job->common.len - start); ++ n = MIN(job->cluster_size, job->len - start); + + if (!bounce_buffer) { + bounce_buffer = blk_blockalign(blk, job->cluster_size); +@@ -159,7 +160,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, + * offset field is an opaque progress value, it is not a disk offset. + */ + job->bytes_read += n; +- job->common.offset += n; ++ block_job_progress_update(&job->common, n); + } + + out: +@@ -261,7 +262,7 @@ void backup_do_checkpoint(BlockJob *job, Error **errp) + return; + } + +- len = DIV_ROUND_UP(backup_job->common.len, backup_job->cluster_size); ++ len = DIV_ROUND_UP(backup_job->len, backup_job->cluster_size); + hbitmap_set(backup_job->copy_bitmap, 0, len); + } + +@@ -420,8 +421,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) + bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size); + } + +- job->common.offset = job->common.len - +- hbitmap_count(job->copy_bitmap) * job->cluster_size; ++ /* TODO block_job_progress_set_remaining() would make more sense */ ++ block_job_progress_update(&job->common, ++ job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); + + bdrv_dirty_iter_free(dbi); + } +@@ -437,7 +439,9 @@ static void coroutine_fn backup_run(void *opaque) + QLIST_INIT(&job->inflight_reqs); + qemu_co_rwlock_init(&job->flush_rwlock); + +- nb_clusters = DIV_ROUND_UP(job->common.len, job->cluster_size); ++ nb_clusters = DIV_ROUND_UP(job->len, job->cluster_size); ++ block_job_progress_set_remaining(&job->common, job->len); ++ + job->copy_bitmap = hbitmap_alloc(nb_clusters, 0); + if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { + backup_incremental_init_copy_bitmap(job); +@@ -461,7 +465,7 @@ static void coroutine_fn backup_run(void *opaque) + ret = backup_run_incremental(job); + } else { + /* Both FULL and TOP SYNC_MODE's require copying.. */ +- for (offset = 0; offset < job->common.len; ++ for (offset = 0; offset < job->len; + offset += job->cluster_size) { + bool error_is_read; + int alloced = 0; +@@ -620,7 +624,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + goto error; + } + +- /* job->common.len is fixed, so we can't allow resize */ ++ /* job->len is fixed, so we can't allow resize */ + job = block_job_create(job_id, &backup_job_driver, txn, bs, + BLK_PERM_CONSISTENT_READ, + BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE | +@@ -676,7 +680,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + /* Required permissions are already taken with target's blk_new() */ + block_job_add_bdrv(&job->common, "target", target, 0, BLK_PERM_ALL, + &error_abort); +- job->common.len = len; ++ job->len = len; + + return &job->common; + +diff --git a/block/commit.c b/block/commit.c +index 1432bae..50b191c 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -146,21 +146,21 @@ static void coroutine_fn commit_run(void *opaque) + int64_t n = 0; /* bytes */ + void *buf = NULL; + int bytes_written = 0; +- int64_t base_len; ++ int64_t len, base_len; + +- ret = s->common.len = blk_getlength(s->top); +- +- if (s->common.len < 0) { ++ ret = len = blk_getlength(s->top); ++ if (len < 0) { + goto out; + } ++ block_job_progress_set_remaining(&s->common, len); + + ret = base_len = blk_getlength(s->base); + if (base_len < 0) { + goto out; + } + +- if (base_len < s->common.len) { +- ret = blk_truncate(s->base, s->common.len, PREALLOC_MODE_OFF, NULL); ++ if (base_len < len) { ++ ret = blk_truncate(s->base, len, PREALLOC_MODE_OFF, NULL); + if (ret) { + goto out; + } +@@ -168,7 +168,7 @@ static void coroutine_fn commit_run(void *opaque) + + buf = blk_blockalign(s->top, COMMIT_BUFFER_SIZE); + +- for (offset = 0; offset < s->common.len; offset += n) { ++ for (offset = 0; offset < len; offset += n) { + bool copy; + + /* Note that even when no rate limit is applied we need to yield +@@ -198,7 +198,7 @@ static void coroutine_fn commit_run(void *opaque) + } + } + /* Publish progress */ +- s->common.offset += n; ++ block_job_progress_update(&s->common, n); + + if (copy && s->common.speed) { + delay_ns = ratelimit_calculate_delay(&s->limit, n); +diff --git a/block/mirror.c b/block/mirror.c +index 003f957..ed711b5 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -121,7 +121,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret) + bitmap_set(s->cow_bitmap, chunk_num, nb_chunks); + } + if (!s->initial_zeroing_ongoing) { +- s->common.offset += op->bytes; ++ block_job_progress_update(&s->common, op->bytes); + } + } + qemu_iovec_destroy(&op->qiov); +@@ -792,11 +792,10 @@ static void coroutine_fn mirror_run(void *opaque) + block_job_pause_point(&s->common); + + cnt = bdrv_get_dirty_count(s->dirty_bitmap); +- /* s->common.offset contains the number of bytes already processed so +- * far, cnt is the number of dirty bytes remaining and +- * s->bytes_in_flight is the number of bytes currently being +- * processed; together those are the current total operation length */ +- s->common.len = s->common.offset + s->bytes_in_flight + cnt; ++ /* cnt is the number of dirty bytes remaining and s->bytes_in_flight is ++ * the number of bytes currently being processed; together those are ++ * the current remaining operation length */ ++ block_job_progress_set_remaining(&s->common, s->bytes_in_flight + cnt); + + /* Note that even when no rate limit is applied we need to yield + * periodically with no pending I/O so that bdrv_drain_all() returns. +diff --git a/block/stream.c b/block/stream.c +index 1a85708..8369852 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -107,6 +107,7 @@ static void coroutine_fn stream_run(void *opaque) + BlockBackend *blk = s->common.blk; + BlockDriverState *bs = blk_bs(blk); + BlockDriverState *base = s->base; ++ int64_t len; + int64_t offset = 0; + uint64_t delay_ns = 0; + int error = 0; +@@ -118,11 +119,12 @@ static void coroutine_fn stream_run(void *opaque) + goto out; + } + +- s->common.len = bdrv_getlength(bs); +- if (s->common.len < 0) { +- ret = s->common.len; ++ len = bdrv_getlength(bs); ++ if (len < 0) { ++ ret = len; + goto out; + } ++ block_job_progress_set_remaining(&s->common, len); + + buf = qemu_blockalign(bs, STREAM_BUFFER_SIZE); + +@@ -135,7 +137,7 @@ static void coroutine_fn stream_run(void *opaque) + bdrv_enable_copy_on_read(bs); + } + +- for ( ; offset < s->common.len; offset += n) { ++ for ( ; offset < len; offset += n) { + bool copy; + + /* Note that even when no rate limit is applied we need to yield +@@ -159,7 +161,7 @@ static void coroutine_fn stream_run(void *opaque) + + /* Finish early if end of backing file has been reached */ + if (ret == 0 && n == 0) { +- n = s->common.len - offset; ++ n = len - offset; + } + + copy = (ret == 1); +@@ -185,7 +187,7 @@ static void coroutine_fn stream_run(void *opaque) + ret = 0; + + /* Publish progress */ +- s->common.offset += n; ++ block_job_progress_update(&s->common, n); + if (copy && s->common.speed) { + delay_ns = ratelimit_calculate_delay(&s->limit, n); + } else { +diff --git a/blockjob.c b/blockjob.c +index 0033b96..d0a2ac5 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -818,6 +818,16 @@ int block_job_complete_sync(BlockJob *job, Error **errp) + return block_job_finish_sync(job, &block_job_complete, errp); + } + ++void block_job_progress_update(BlockJob *job, uint64_t done) ++{ ++ job->offset += done; ++} ++ ++void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining) ++{ ++ job->len = job->offset + remaining; ++} ++ + BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + { + BlockJobInfo *info; +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index fc645da..a2cc522 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -278,6 +278,25 @@ void block_job_finalize(BlockJob *job, Error **errp); + void block_job_dismiss(BlockJob **job, Error **errp); + + /** ++ * block_job_progress_update: ++ * @job: The job that has made progress ++ * @done: How much progress the job made ++ * ++ * Updates the progress counter of the job. ++ */ ++void block_job_progress_update(BlockJob *job, uint64_t done); ++ ++/** ++ * block_job_progress_set_remaining: ++ * @job: The job whose expected progress end value is set ++ * @remaining: Expected end value of the progress counter of the job ++ * ++ * Sets the expected end value of the progress counter of a job so that a ++ * completion percentage can be calculated when the progress is updated. ++ */ ++void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining); ++ ++/** + * block_job_query: + * @job: The job to get information about. + * +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-do-not-cancel-timer-in-resume.patch b/SOURCES/kvm-blockjob-do-not-cancel-timer-in-resume.patch new file mode 100644 index 0000000..03bac55 --- /dev/null +++ b/SOURCES/kvm-blockjob-do-not-cancel-timer-in-resume.patch @@ -0,0 +1,165 @@ +From 84a7e1d25f5611e43e3781dda588b4306606c809 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:46 +0200 +Subject: [PATCH 078/268] blockjob: do not cancel timer in resume + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-4-kwolf@redhat.com> +Patchwork-id: 81089 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 03/73] blockjob: do not cancel timer in resume +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: Stefan Hajnoczi + +Currently the timer is cancelled and the block job is entered by +block_job_resume(). This behavior causes drain to run extra blockjob +iterations when the job was sleeping due to the ratelimit. + +This patch leaves the job asleep when block_job_resume() is called. +Jobs can still be forcibly woken up using block_job_enter(), which is +used to cancel jobs. + +After this patch drain no longer runs extra blockjob iterations. This +is the expected behavior that qemu-iotests 185 used to rely on. We +temporarily changed the 185 test output to make it pass for the QEMU +2.12 release but now it's time to address this issue. + +Cc: QingFeng Hao +Signed-off-by: Stefan Hajnoczi +Reviewed-by: Eric Blake +Reviewed-by: QingFeng Hao +Message-id: 20180508135436.30140-3-stefanha@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit 4c7e813ce964e230bb55cf4afc862ccb091ca3a3) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 22 +++++++++++++++------- + tests/qemu-iotests/185 | 5 +---- + tests/qemu-iotests/185.out | 12 +++++------- + 3 files changed, 21 insertions(+), 18 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 27f957e..7064389 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -209,6 +209,18 @@ static void block_job_txn_del_job(BlockJob *job) + } + } + ++/* Assumes the block_job_mutex is held */ ++static bool block_job_timer_pending(BlockJob *job) ++{ ++ return timer_pending(&job->sleep_timer); ++} ++ ++/* Assumes the block_job_mutex is held */ ++static bool block_job_timer_not_pending(BlockJob *job) ++{ ++ return !block_job_timer_pending(job); ++} ++ + static void block_job_pause(BlockJob *job) + { + job->pause_count++; +@@ -221,7 +233,9 @@ static void block_job_resume(BlockJob *job) + if (job->pause_count) { + return; + } +- block_job_enter(job); ++ ++ /* kick only if no timer is pending */ ++ block_job_enter_cond(job, block_job_timer_not_pending); + } + + void block_job_ref(BlockJob *job) +@@ -651,12 +665,6 @@ static void block_job_completed_txn_success(BlockJob *job) + } + } + +-/* Assumes the block_job_mutex is held */ +-static bool block_job_timer_pending(BlockJob *job) +-{ +- return timer_pending(&job->sleep_timer); +-} +- + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) + { + Error *local_err = NULL; +diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 +index 975404c..9a2d317 100755 +--- a/tests/qemu-iotests/185 ++++ b/tests/qemu-iotests/185 +@@ -101,14 +101,11 @@ echo + # command to be received (after receiving the command, the rest runs + # synchronously, so jobs can arbitrarily continue or complete). + # +-# Jobs present while QEMU is terminating iterate once more due to +-# bdrv_drain_all(). +-# + # The buffer size for commit and streaming is 512k (waiting for 8 seconds after + # the first request), for active commit and mirror it's large enough to cover + # the full 4M, and for backup it's the qcow2 cluster size, which we know is + # 64k. As all of these are at least as large as the speed, we are sure that the +-# offset advances exactly twice before qemu exits. ++# offset advances exactly once before qemu exits. + + _send_qemu_cmd $h \ + "{ 'execute': 'block-commit', +diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out +index 992162f..57eaf8d 100644 +--- a/tests/qemu-iotests/185.out ++++ b/tests/qemu-iotests/185.out +@@ -20,7 +20,7 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 1048576, "speed": 65536, "type": "commit"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 524288, "speed": 65536, "type": "commit"}} + + === Start active commit job and exit qemu === + +@@ -28,8 +28,7 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "commit"}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "commit"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "commit"}} + + === Start mirror job and exit qemu === + +@@ -38,8 +37,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 4194304, "offset": 4194304, "speed": 65536, "type": "mirror"}} + + === Start backup job and exit qemu === + +@@ -48,7 +46,7 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 131072, "speed": 65536, "type": "backup"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 65536, "speed": 65536, "type": "backup"}} + + === Start streaming job and exit qemu === + +@@ -56,6 +54,6 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 1048576, "speed": 65536, "type": "stream"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "disk", "len": 67108864, "offset": 524288, "speed": 65536, "type": "stream"}} + No errors were found on the image. + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-drop-block_job_pause-resume_all.patch b/SOURCES/kvm-blockjob-drop-block_job_pause-resume_all.patch new file mode 100644 index 0000000..dd81302 --- /dev/null +++ b/SOURCES/kvm-blockjob-drop-block_job_pause-resume_all.patch @@ -0,0 +1,109 @@ +From 322f7ce7a3bdad03b0185cbf6d16f29bb8685029 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:49 +0200 +Subject: [PATCH 081/268] blockjob: drop block_job_pause/resume_all() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-7-kwolf@redhat.com> +Patchwork-id: 81068 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 06/73] blockjob: drop block_job_pause/resume_all() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: Stefan Hajnoczi + +Commit 8119334918e86f45877cfc139192d54f2449a239 ("block: Don't +block_job_pause_all() in bdrv_drain_all()") removed the only callers of +block_job_pause/resume_all(). + +Pausing and resuming now happens in child_job_drained_begin/end() so +it's no longer necessary to globally pause/resume jobs. + +Signed-off-by: Stefan Hajnoczi +Reviewed-by: John Snow +Reviewed-by: Alberto Garcia +Message-id: 20180424085240.5798-1-stefanha@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 23d702d898bdd8e6772d83ea9789767ed589e17e) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 27 --------------------------- + include/block/blockjob_int.h | 14 -------------- + 2 files changed, 41 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 7064389..b39d0f8 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -996,19 +996,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return job; + } + +-void block_job_pause_all(void) +-{ +- BlockJob *job = NULL; +- while ((job = block_job_next(job))) { +- AioContext *aio_context = blk_get_aio_context(job->blk); +- +- aio_context_acquire(aio_context); +- block_job_ref(job); +- block_job_pause(job); +- aio_context_release(aio_context); +- } +-} +- + void block_job_early_fail(BlockJob *job) + { + assert(job->status == BLOCK_JOB_STATUS_CREATED); +@@ -1086,20 +1073,6 @@ void coroutine_fn block_job_pause_point(BlockJob *job) + } + } + +-void block_job_resume_all(void) +-{ +- BlockJob *job, *next; +- +- QLIST_FOREACH_SAFE(job, &block_jobs, job_list, next) { +- AioContext *aio_context = blk_get_aio_context(job->blk); +- +- aio_context_acquire(aio_context); +- block_job_resume(job); +- block_job_unref(job); +- aio_context_release(aio_context); +- } +-} +- + /* + * Conditionally enter a block_job pending a call to fn() while + * under the block_job_lock critical section. +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 642adce..d5a515d 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -169,20 +169,6 @@ void block_job_sleep_ns(BlockJob *job, int64_t ns); + void block_job_yield(BlockJob *job); + + /** +- * block_job_pause_all: +- * +- * Asynchronously pause all jobs. +- */ +-void block_job_pause_all(void); +- +-/** +- * block_job_resume_all: +- * +- * Resume all block jobs. Must be paired with a preceding block_job_pause_all. +- */ +-void block_job_resume_all(void); +- +-/** + * block_job_early_fail: + * @bs: The block device. + * +-- +1.8.3.1 + diff --git a/SOURCES/kvm-blockjob-expose-error-string-via-query.patch b/SOURCES/kvm-blockjob-expose-error-string-via-query.patch new file mode 100644 index 0000000..e928c7a --- /dev/null +++ b/SOURCES/kvm-blockjob-expose-error-string-via-query.patch @@ -0,0 +1,80 @@ +From bc04db3da4dea7e587bb9d0c587b8a4295f90a1f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:50 +0200 +Subject: [PATCH 082/268] blockjob: expose error string via query + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-8-kwolf@redhat.com> +Patchwork-id: 81059 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 07/73] blockjob: expose error string via query +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: John Snow + +When we've reached the concluded state, we need to expose the error +state if applicable. Add the new field. + +This should be sufficient for determining if a job completed +successfully or not after concluding; if we want to discriminate +based on how it failed more mechanically, we can always add an +explicit return code enumeration later. + +I didn't bother to make it only show up if we are in the concluded +state; I don't think it's necessary. + +Cc: qemu-stable@nongnu.org +Signed-off-by: John Snow +Reviewed-by: Eric Blake +Reviewed-by: Alberto Garcia +Signed-off-by: Kevin Wolf +(cherry picked from commit ab9ba614556ac5b0f8d96b99e0dba19f1e28d6c2) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 2 ++ + qapi/block-core.json | 6 +++++- + 2 files changed, 7 insertions(+), 1 deletion(-) + +diff --git a/blockjob.c b/blockjob.c +index b39d0f8..6746cad 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -839,6 +839,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info->status = job->status; + info->auto_finalize = job->auto_finalize; + info->auto_dismiss = job->auto_dismiss; ++ info->has_error = job->ret != 0; ++ info->error = job->ret ? g_strdup(strerror(-job->ret)) : NULL; + return info; + } + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 9e4f1ac..28b4964 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1172,6 +1172,9 @@ + # @auto-dismiss: Job will dismiss itself when CONCLUDED, moving to the NULL + # state and disappearing from the query list. (since 2.12) + # ++# @error: Error information if the job did not complete successfully. ++# Not set if the job completed successfully. (since 2.12.1) ++# + # Since: 1.1 + ## + { 'struct': 'BlockJobInfo', +@@ -1179,7 +1182,8 @@ + 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int', + 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', + 'status': 'BlockJobStatus', +- 'auto-finalize': 'bool', 'auto-dismiss': 'bool' } } ++ 'auto-finalize': 'bool', 'auto-dismiss': 'bool', ++ '*error': 'str' } } + + ## + # @query-block-jobs: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ccid-card-passthru-fix-regression-in-realize.patch b/SOURCES/kvm-ccid-card-passthru-fix-regression-in-realize.patch new file mode 100644 index 0000000..b6323ae --- /dev/null +++ b/SOURCES/kvm-ccid-card-passthru-fix-regression-in-realize.patch @@ -0,0 +1,51 @@ +From 5e6d388c89b6437aeb9f54607ed09fe75bb9c2cd Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Mon, 11 Jun 2018 17:13:26 +0200 +Subject: [PATCH 006/268] ccid-card-passthru: fix regression in realize() +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20180611171326.5043-1-marcandre.lureau@redhat.com> +Patchwork-id: 80632 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH] ccid-card-passthru: fix regression in realize() +Bugzilla: 1584984 +RH-Acked-by: Gerd Hoffmann +RH-Acked-by: Markus Armbruster +RH-Acked-by: Thomas Huth + +Since cc847bfd16d894fd8c1a2ce25f31772f6cdbbc74, CCID card-passthru +fails to intialize, because it changed a debug line to an error, +probably by mistake. Change it back to a DPRINTF debug. + +(solves Boxes creating VM with smartcard passthru failing to start) + +Signed-off-by: Marc-André Lureau +Reviewed-by: Philippe Mathieu-Daudé +Message-id: 20180515153039.27514-1-marcandre.lureau@redhat.com +Signed-off-by: Gerd Hoffmann + +(cherry picked from commit e58d64a16abc2304c4dcb644411eb9580bf63b1e) +Signed-off-by: Marc-André Lureau +Signed-off-by: Miroslav Rezanina +--- + hw/usb/ccid-card-passthru.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/usb/ccid-card-passthru.c b/hw/usb/ccid-card-passthru.c +index b7dd360..668a22d 100644 +--- a/hw/usb/ccid-card-passthru.c ++++ b/hw/usb/ccid-card-passthru.c +@@ -345,7 +345,7 @@ static void passthru_realize(CCIDCardState *base, Error **errp) + card->vscard_in_pos = 0; + card->vscard_in_hdr = 0; + if (qemu_chr_fe_backend_connected(&card->cs)) { +- error_setg(errp, "ccid-card-passthru: initing chardev"); ++ DPRINTF(card, D_INFO, "ccid-card-passthru: initing chardev"); + qemu_chr_fe_set_handlers(&card->cs, + ccid_card_vscard_can_read, + ccid_card_vscard_read, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-check-Only-test-ivshm-when-it-is-compiled-in.patch b/SOURCES/kvm-check-Only-test-ivshm-when-it-is-compiled-in.patch new file mode 100644 index 0000000..64fa802 --- /dev/null +++ b/SOURCES/kvm-check-Only-test-ivshm-when-it-is-compiled-in.patch @@ -0,0 +1,58 @@ +From 6fb5e1331742aeb2780cd5dc0ae53716666b703c Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Wed, 10 Oct 2018 04:58:18 +0100 +Subject: [PATCH 1/5] check: Only test ivshm when it is compiled in + +RH-Author: Markus Armbruster +Message-id: <20181010045819.32729-2-armbru@redhat.com> +Patchwork-id: 82527 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/2] check: Only test ivshm when it is compiled in +Bugzilla: 1621817 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Thomas Huth +RH-Acked-by: Stefan Hajnoczi + +From: Juan Quintela + +Signed-off-by: Juan Quintela +Reviewed-by: Thomas Huth +Signed-off-by: Juan Quintela +(cherry picked from commit 1336e6085b5b01d274d1cd668897f13a40817fe3) + +Conflicts contextually because we have downstream hacks instead of the +full upstream series containing this patch. + +Signed-off-by: Markus Armbruster +Signed-off-by: Danilo C. L. de Paula +--- + tests/Makefile.include | 7 ++++--- + 1 file changed, 4 insertions(+), 3 deletions(-) + +diff --git a/tests/Makefile.include b/tests/Makefile.include +index cb12ee6..3ed8531 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -244,8 +244,8 @@ gcov-files-pci-y += hw/display/virtio-gpu-pci.c + gcov-files-pci-$(CONFIG_VIRTIO_VGA) += hw/display/virtio-vga.c + check-qtest-pci-y += tests/intel-hda-test$(EXESUF) + gcov-files-pci-y += hw/audio/intel-hda.c hw/audio/hda-codec.c +-check-qtest-pci-$(CONFIG_IVSHMEM) += tests/ivshmem-test$(EXESUF) +-gcov-files-pci-y += hw/misc/ivshmem.c ++check-qtest-pci-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) ++gcov-files-pci-$(CONFIG_IVSHMEM_DEVICE) += hw/misc/ivshmem.c + #check-qtest-pci-y += tests/megasas-test$(EXESUF) + #gcov-files-pci-y += hw/scsi/megasas.c + +@@ -356,7 +356,8 @@ check-qtest-ppc64-y += $(check-qtest-virtio-y) + #check-qtest-ppc64-$(CONFIG_POSIX) += tests/test-filter-redirector$(EXESUF) + check-qtest-ppc64-y += tests/display-vga-test$(EXESUF) + check-qtest-ppc64-y += tests/numa-test$(EXESUF) +-#check-qtest-ppc64-$(CONFIG_IVSHMEM) += tests/ivshmem-test$(EXESUF) ++check-qtest-ppc64-$(CONFIG_IVSHMEM_DEVICE) += tests/ivshmem-test$(EXESUF) ++gcov-files-ppc64-$(CONFIG_IVSHMEM_DEVICE) += hw/misc/ivshmem.c + check-qtest-ppc64-y += tests/cpu-plug-test$(EXESUF) + + check-qtest-sh4-y = tests/endianness-test$(EXESUF) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-check-block-qdict-Cover-flattening-of-empty-lists-an.patch b/SOURCES/kvm-check-block-qdict-Cover-flattening-of-empty-lists-an.patch new file mode 100644 index 0000000..4b68442 --- /dev/null +++ b/SOURCES/kvm-check-block-qdict-Cover-flattening-of-empty-lists-an.patch @@ -0,0 +1,76 @@ +From bd71f94946365ba320066bf166f7ce1789191521 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:27 +0200 +Subject: [PATCH 029/268] check-block-qdict: Cover flattening of empty lists + and dictionaries + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-21-armbru@redhat.com> +Patchwork-id: 80734 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 20/23] check-block-qdict: Cover flattening of empty lists and dictionaries +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit bef96b1549907b005ce1fa1456d2a0910d2a1aa5) +Signed-off-by: Miroslav Rezanina +--- + tests/check-block-qdict.c | 14 +++++++++++++- + 1 file changed, 13 insertions(+), 1 deletion(-) + +diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c +index 29f58a2..2da16f0 100644 +--- a/tests/check-block-qdict.c ++++ b/tests/check-block-qdict.c +@@ -41,6 +41,8 @@ static void qdict_flatten_test(void) + QList *e = qlist_new(); + QDict *e_1_2 = qdict_new(); + QDict *f = qdict_new(); ++ QList *y = qlist_new(); ++ QDict *z = qdict_new(); + QDict *root = qdict_new(); + + /* +@@ -62,7 +64,9 @@ static void qdict_flatten_test(void) + * "c": 2, + * "d": 3, + * }, +- * "g": 4 ++ * "g": 4, ++ * "y": [{}], ++ * "z": {"a": []} + * } + * + * to +@@ -77,6 +81,8 @@ static void qdict_flatten_test(void) + * "f.d": 3, + * "g": 4 + * } ++ * ++ * Note that "y" and "z" get eaten. + */ + + qdict_put_int(e_1_2, "a", 0); +@@ -91,9 +97,15 @@ static void qdict_flatten_test(void) + qdict_put_int(f, "c", 2); + qdict_put_int(f, "d", 3); + ++ qlist_append(y, qdict_new()); ++ ++ qdict_put(z, "a", qlist_new()); ++ + qdict_put(root, "e", e); + qdict_put(root, "f", f); + qdict_put_int(root, "g", 4); ++ qdict_put(root, "y", y); ++ qdict_put(root, "z", z); + + qdict_flatten(root); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-check-block-qdict-Rename-qdict_flatten-s-variables-f.patch b/SOURCES/kvm-check-block-qdict-Rename-qdict_flatten-s-variables-f.patch new file mode 100644 index 0000000..fa58f73 --- /dev/null +++ b/SOURCES/kvm-check-block-qdict-Rename-qdict_flatten-s-variables-f.patch @@ -0,0 +1,108 @@ +From 239d3cf97ced74198ce100212e9ca2c39835b07a Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:26 +0200 +Subject: [PATCH 028/268] check-block-qdict: Rename qdict_flatten()'s variables + for clarity + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-20-armbru@redhat.com> +Patchwork-id: 80721 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 19/23] check-block-qdict: Rename qdict_flatten()'s variables for clarity +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit cddec036830ada5d5d45023bcfba09015b8ab394) +Signed-off-by: Miroslav Rezanina +--- + tests/check-block-qdict.c | 57 ++++++++++++++++++++++++----------------------- + 1 file changed, 29 insertions(+), 28 deletions(-) + +diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c +index 5b9f4d5..29f58a2 100644 +--- a/tests/check-block-qdict.c ++++ b/tests/check-block-qdict.c +@@ -37,11 +37,11 @@ static void qdict_defaults_test(void) + + static void qdict_flatten_test(void) + { +- QList *list1 = qlist_new(); +- QList *list2 = qlist_new(); +- QDict *dict1 = qdict_new(); +- QDict *dict2 = qdict_new(); +- QDict *dict3 = qdict_new(); ++ QList *e_1 = qlist_new(); ++ QList *e = qlist_new(); ++ QDict *e_1_2 = qdict_new(); ++ QDict *f = qdict_new(); ++ QDict *root = qdict_new(); + + /* + * Test the flattening of +@@ -79,35 +79,36 @@ static void qdict_flatten_test(void) + * } + */ + +- qdict_put_int(dict1, "a", 0); +- qdict_put_int(dict1, "b", 1); ++ qdict_put_int(e_1_2, "a", 0); ++ qdict_put_int(e_1_2, "b", 1); + +- qlist_append_int(list1, 23); +- qlist_append_int(list1, 66); +- qlist_append(list1, dict1); +- qlist_append_int(list2, 42); +- qlist_append(list2, list1); ++ qlist_append_int(e_1, 23); ++ qlist_append_int(e_1, 66); ++ qlist_append(e_1, e_1_2); ++ qlist_append_int(e, 42); ++ qlist_append(e, e_1); + +- qdict_put_int(dict2, "c", 2); +- qdict_put_int(dict2, "d", 3); +- qdict_put(dict3, "e", list2); +- qdict_put(dict3, "f", dict2); +- qdict_put_int(dict3, "g", 4); ++ qdict_put_int(f, "c", 2); ++ qdict_put_int(f, "d", 3); + +- qdict_flatten(dict3); ++ qdict_put(root, "e", e); ++ qdict_put(root, "f", f); ++ qdict_put_int(root, "g", 4); + +- g_assert(qdict_get_int(dict3, "e.0") == 42); +- g_assert(qdict_get_int(dict3, "e.1.0") == 23); +- g_assert(qdict_get_int(dict3, "e.1.1") == 66); +- g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); +- g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); +- g_assert(qdict_get_int(dict3, "f.c") == 2); +- g_assert(qdict_get_int(dict3, "f.d") == 3); +- g_assert(qdict_get_int(dict3, "g") == 4); ++ qdict_flatten(root); + +- g_assert(qdict_size(dict3) == 8); ++ g_assert(qdict_get_int(root, "e.0") == 42); ++ g_assert(qdict_get_int(root, "e.1.0") == 23); ++ g_assert(qdict_get_int(root, "e.1.1") == 66); ++ g_assert(qdict_get_int(root, "e.1.2.a") == 0); ++ g_assert(qdict_get_int(root, "e.1.2.b") == 1); ++ g_assert(qdict_get_int(root, "f.c") == 2); ++ g_assert(qdict_get_int(root, "f.d") == 3); ++ g_assert(qdict_get_int(root, "g") == 4); + +- qobject_unref(dict3); ++ g_assert(qdict_size(root) == 8); ++ ++ qobject_unref(root); + } + + static void qdict_array_split_test(void) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-commit-Add-top-node-base-node-options.patch b/SOURCES/kvm-commit-Add-top-node-base-node-options.patch new file mode 100644 index 0000000..0617c46 --- /dev/null +++ b/SOURCES/kvm-commit-Add-top-node-base-node-options.patch @@ -0,0 +1,141 @@ +From 736bcac80123ee6415953277449359722ae93d37 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 13:50:54 +0100 +Subject: [PATCH 4/5] commit: Add top-node/base-node options + +RH-Author: Kevin Wolf +Message-id: <20181010135055.3874-2-kwolf@redhat.com> +Patchwork-id: 82569 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/2] commit: Add top-node/base-node options +Bugzilla: 1637970 +RH-Acked-by: John Snow +RH-Acked-by: Fam Zheng +RH-Acked-by: Stefan Hajnoczi + +The block-commit QMP command required specifying the top and base nodes +of the commit jobs using the file name of that node. While this works +in simple cases (local files with absolute paths), the file names +generated for more complicated setups can be hard to predict. + +The block-commit command has more problems than just this, so we want to +replace it altogether in the long run, but libvirt needs a reliable way +to address nodes now. So we don't want to wait for a new, cleaner +command, but just add the minimal thing needed right now. + +This adds two new options top-node and base-node to the command, which +allow specifying node names instead. They are mutually exclusive with +the old options. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 3c605f4074ebeb97970eb660fb56a9cb06525923) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 32 ++++++++++++++++++++++++++++++-- + qapi/block-core.json | 24 ++++++++++++++++++------ + 2 files changed, 48 insertions(+), 8 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index b8e4b0d..70af034 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3325,7 +3325,9 @@ out: + } + + void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, ++ bool has_base_node, const char *base_node, + bool has_base, const char *base, ++ bool has_top_node, const char *top_node, + bool has_top, const char *top, + bool has_backing_file, const char *backing_file, + bool has_speed, int64_t speed, +@@ -3386,7 +3388,20 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + /* default top_bs is the active layer */ + top_bs = bs; + +- if (has_top && top) { ++ if (has_top_node && has_top) { ++ error_setg(errp, "'top-node' and 'top' are mutually exclusive"); ++ goto out; ++ } else if (has_top_node) { ++ top_bs = bdrv_lookup_bs(NULL, top_node, errp); ++ if (top_bs == NULL) { ++ goto out; ++ } ++ if (!bdrv_chain_contains(bs, top_bs)) { ++ error_setg(errp, "'%s' is not in this backing file chain", ++ top_node); ++ goto out; ++ } ++ } else if (has_top && top) { + if (strcmp(bs->filename, top) != 0) { + top_bs = bdrv_find_backing_image(bs, top); + } +@@ -3399,7 +3414,20 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + + assert(bdrv_get_aio_context(top_bs) == aio_context); + +- if (has_base && base) { ++ if (has_base_node && has_base) { ++ error_setg(errp, "'base-node' and 'base' are mutually exclusive"); ++ goto out; ++ } else if (has_base_node) { ++ base_bs = bdrv_lookup_bs(NULL, base_node, errp); ++ if (base_bs == NULL) { ++ goto out; ++ } ++ if (!bdrv_chain_contains(top_bs, base_bs)) { ++ error_setg(errp, "'%s' is not in this backing file chain", ++ base_node); ++ goto out; ++ } ++ } else if (has_base && base) { + base_bs = bdrv_find_backing_image(top_bs, base); + } else { + base_bs = bdrv_find_base(top_bs); +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 602c028..5fb7983 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1434,12 +1434,23 @@ + # + # @device: the device name or node-name of a root node + # +-# @base: The file name of the backing image to write data into. +-# If not specified, this is the deepest backing image. ++# @base-node: The node name of the backing image to write data into. ++# If not specified, this is the deepest backing image. ++# (since: 3.1) + # +-# @top: The file name of the backing image within the image chain, +-# which contains the topmost data to be committed down. If +-# not specified, this is the active layer. ++# @base: Same as @base-node, except that it is a file name rather than a node ++# name. This must be the exact filename string that was used to open the ++# node; other strings, even if addressing the same file, are not ++# accepted (deprecated, use @base-node instead) ++# ++# @top-node: The node name of the backing image within the image chain ++# which contains the topmost data to be committed down. If ++# not specified, this is the active layer. (since: 3.1) ++# ++# @top: Same as @top-node, except that it is a file name rather than a node ++# name. This must be the exact filename string that was used to open the ++# node; other strings, even if addressing the same file, are not ++# accepted (deprecated, use @base-node instead) + # + # @backing-file: The backing file string to write into the overlay + # image of 'top'. If 'top' is the active layer, +@@ -1508,7 +1519,8 @@ + # + ## + { 'command': 'block-commit', +- 'data': { '*job-id': 'str', 'device': 'str', '*base': 'str', '*top': 'str', ++ 'data': { '*job-id': 'str', 'device': 'str', '*base-node': 'str', ++ '*base': 'str', '*top-node': 'str', '*top': 'str', + '*backing-file': 'str', '*speed': 'int', + '*filter-node-name': 'str', + '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-configure-add-libpmem-support.patch b/SOURCES/kvm-configure-add-libpmem-support.patch new file mode 100644 index 0000000..7079389 --- /dev/null +++ b/SOURCES/kvm-configure-add-libpmem-support.patch @@ -0,0 +1,130 @@ +From 3ac4a8eb45ad35ff759a76233e9437566041dba0 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:18 +0000 +Subject: [PATCH 17/22] configure: add libpmem support + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-6-git-send-email-plai@redhat.com> +Patchwork-id: 83890 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 05/10] configure: add libpmem support +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +Add a pair of configure options --{enable,disable}-libpmem to control +whether QEMU is compiled with PMDK libpmem [1]. + +QEMU may write to the host persistent memory (e.g. in vNVDIMM label +emulation and live migration), so it must take the proper operations +to ensure the persistence of its own writes. Depending on the CPU +models and available instructions, the optimal operation can vary [2]. +PMDK libpmem have already implemented those operations on multiple CPU +models (x86 and ARM) and the logic to select the optimal ones, so QEMU +can just use libpmem rather than re-implement them. + +Libpem is a part of PMDK project(formerly known as NMVL). +The project's home page is: http://pmem.io/pmdk/ +And the project's repository is: https://github.com/pmem/pmdk/ + +For more information about libpmem APIs, you can refer to the comments +in source code of: pmdk/src/libpmem/pmem.c, begin at line 33. + +Signed-off-by: Junyan He +Signed-off-by: Haozhong Zhang +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Richard Henderson +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 17824406fa55b303379f2e4af715c1e876c3535f) +Signed-off-by: Paul Lai + +Resolved Conflicts: + configure + +Signed-off-by: Danilo C. L. de Paula +--- + configure | 29 +++++++++++++++++++++++++++++ + 1 file changed, 29 insertions(+) + +diff --git a/configure b/configure +index 139f3c8..858b456 100755 +--- a/configure ++++ b/configure +@@ -461,6 +461,7 @@ parallels="yes" + sheepdog="yes" + libxml2="" + libudev="no" ++libpmem="" + + supported_cpu="no" + supported_os="no" +@@ -1421,6 +1422,10 @@ for opt do + ;; + --disable-git-update) git_update=no + ;; ++ --enable-libpmem) libpmem=yes ++ ;; ++ --disable-libpmem) libpmem=no ++ ;; + *) + echo "ERROR: unknown option $opt" + echo "Try '$0 --help' for more information" +@@ -1687,6 +1692,7 @@ disabled with --disable-FEATURE, default is enabled if available: + crypto-afalg Linux AF_ALG crypto backend driver + vhost-user vhost-user support + capstone capstone disassembler support ++ libpmem libpmem support + + NOTE: The object files are built at the place where configure is launched + EOF +@@ -5458,6 +5464,24 @@ EOF + fi + + ########################################## ++# check for libpmem ++ ++if test "$libpmem" != "no"; then ++ if $pkg_config --exists "libpmem"; then ++ libpmem="yes" ++ libpmem_libs=$($pkg_config --libs libpmem) ++ libpmem_cflags=$($pkg_config --cflags libpmem) ++ libs_softmmu="$libs_softmmu $libpmem_libs" ++ QEMU_CFLAGS="$QEMU_CFLAGS $libpmem_cflags" ++ else ++ if test "$libpmem" = "yes" ; then ++ feature_not_found "libpmem" "Install nvml or pmdk" ++ fi ++ libpmem="no" ++ fi ++fi ++ ++########################################## + # End of CC checks + # After here, no more $cc or $ld runs + +@@ -5939,6 +5963,7 @@ echo "parallels support $parallels" + echo "sheepdog support $sheepdog" + echo "capstone $capstone" + echo "libudev $libudev" ++echo "libpmem support $libpmem" + + if test "$sdl_too_old" = "yes"; then + echo "-> Your SDL version is too old - please upgrade to have SDL support" +@@ -6714,6 +6739,10 @@ if test "$sheepdog" = "yes" ; then + echo "CONFIG_SHEEPDOG=y" >> $config_host_mak + fi + ++if test "$libpmem" = "yes" ; then ++ echo "CONFIG_LIBPMEM=y" >> $config_host_mak ++fi ++ + if test "$tcg_interpreter" = "yes"; then + QEMU_INCLUDES="-iquote \$(SRC_PATH)/tcg/tci $QEMU_INCLUDES" + elif test "$ARCH" = "sparc64" ; then +-- +1.8.3.1 + diff --git a/SOURCES/kvm-configure-add-test-for-libudev.patch b/SOURCES/kvm-configure-add-test-for-libudev.patch new file mode 100644 index 0000000..4f5168b --- /dev/null +++ b/SOURCES/kvm-configure-add-test-for-libudev.patch @@ -0,0 +1,98 @@ +From 1adffaf59f8b1a62353a009ac62aef8172514e2c Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Sun, 4 Nov 2018 15:45:26 +0000 +Subject: [PATCH 01/35] configure: add test for libudev +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20181104154528.19241-2-marcandre.lureau@redhat.com> +Patchwork-id: 82925 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/3] configure: add test for libudev +Bugzilla: 1636185 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Markus Armbruster + +From: Tomáš Golembiovský + +Signed-off-by: Tomáš Golembiovský +Reviewed-by: Marc-André Lureau +*make libudev optional to avoid breaking existing build/test environments +*disable libudev for --static builds +Signed-off-by: Michael Roth + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1636185 + +(cherry picked from commit 3efac6ebb88e4d099f07fef65178ebaa595ae770) + +[ fix conflicts due to extra configure checks for pmem & cross upstream ] +Signed-off-by: Marc-André Lureau + +Signed-off-by: Danilo C. L. de Paula +--- + configure | 19 +++++++++++++++++++ + 1 file changed, 19 insertions(+) + +diff --git a/configure b/configure +index efd9eb6..0cb2b79 100755 +--- a/configure ++++ b/configure +@@ -451,6 +451,7 @@ jemalloc="no" + replication="yes" + vxhs="" + libxml2="" ++libudev="no" + + supported_cpu="no" + supported_os="no" +@@ -819,6 +820,7 @@ Linux) + vhost_vsock="yes" + QEMU_INCLUDES="-I\$(SRC_PATH)/linux-headers -I$(pwd)/linux-headers $QEMU_INCLUDES" + supported_os="yes" ++ libudev="yes" + ;; + esac + +@@ -5440,6 +5442,17 @@ if test "$libnfs" != "no" ; then + fi + fi + ++########################################## ++# Do we have libudev ++if test "$libudev" != "no" ; then ++ if $pkg_config libudev && test "$static" != "yes"; then ++ libudev="yes" ++ libudev_libs=$($pkg_config --libs libudev) ++ else ++ libudev="no" ++ fi ++fi ++ + # Now we've finished running tests it's OK to add -Werror to the compiler flags + if test "$werror" = "yes"; then + QEMU_CFLAGS="-Werror $QEMU_CFLAGS" +@@ -5858,6 +5871,7 @@ echo "avx2 optimization $avx2_opt" + echo "replication support $replication" + echo "VxHS block device $vxhs" + echo "capstone $capstone" ++echo "libudev $libudev" + + if test "$sdl_too_old" = "yes"; then + echo "-> Your SDL version is too old - please upgrade to have SDL support" +@@ -6684,6 +6698,11 @@ if test "$gcov" = "yes" ; then + echo "GCOV=$gcov_tool" >> $config_host_mak + fi + ++if test "$libudev" != "no"; then ++ echo "CONFIG_LIBUDEV=y" >> $config_host_mak ++ echo "LIBUDEV_LIBS=$libudev_libs" >> $config_host_mak ++fi ++ + # use included Linux headers + if test "$linux" = "yes" ; then + mkdir -p linux-headers +-- +1.8.3.1 + diff --git a/SOURCES/kvm-configure-require-libseccomp-2.2.0.patch b/SOURCES/kvm-configure-require-libseccomp-2.2.0.patch new file mode 100644 index 0000000..c319994 --- /dev/null +++ b/SOURCES/kvm-configure-require-libseccomp-2.2.0.patch @@ -0,0 +1,73 @@ +From f98862c05573261a3fd0cd30ed27992290183835 Mon Sep 17 00:00:00 2001 +From: Eduardo Otubo +Date: Fri, 28 Sep 2018 07:56:38 +0100 +Subject: [PATCH 4/6] configure: require libseccomp 2.2.0 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Otubo +Message-id: <20180928075639.16746-5-otubo@redhat.com> +Patchwork-id: 82312 +O-Subject: [RHEL-8 qemu-kvm PATCH 4/5] configure: require libseccomp 2.2.0 +Bugzilla: 1618356 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Thomas Huth + +From: Marc-André Lureau + +commit d0699bd37c48067cffbd80383172efc29da6d2f9 +Author: Marc-André Lureau +Date: Wed Aug 22 19:02:49 2018 +0200 + + configure: require libseccomp 2.2.0 + + The following patch is going to require TSYNC, which is only available + since libseccomp 2.2.0. + + libseccomp 2.2.0 was released February 12, 2015. + + According to repology, libseccomp version in different distros: + + RHEL-7: 2.3.1 + Debian (Stretch): 2.3.1 + OpenSUSE Leap 15: 2.3.2 + Ubuntu (Xenial): 2.3.1 + + This will drop support for -sandbox on: + + Debian (Jessie): 2.1.1 (but 2.2.3 in backports) + + Signed-off-by: Marc-André Lureau + Acked-by: Eduardo Otubo + +Signed-off-by: Eduardo Otubo +Signed-off-by: Danilo C. L. de Paula +--- + configure | 7 ++----- + 1 file changed, 2 insertions(+), 5 deletions(-) + +diff --git a/configure b/configure +index 23d8d18..efd9eb6 100755 +--- a/configure ++++ b/configure +@@ -2137,13 +2137,10 @@ fi + ########################################## + # libseccomp check + ++libseccomp_minver="2.2.0" + if test "$seccomp" != "no" ; then + case "$cpu" in +- i386|x86_64) +- libseccomp_minver="2.1.0" +- ;; +- mips) +- libseccomp_minver="2.2.0" ++ i386|x86_64|mips) + ;; + arm|aarch64) + libseccomp_minver="2.2.3" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-cpus-Fix-event-order-on-resume-of-stopped-guest.patch b/SOURCES/kvm-cpus-Fix-event-order-on-resume-of-stopped-guest.patch new file mode 100644 index 0000000..47762c4 --- /dev/null +++ b/SOURCES/kvm-cpus-Fix-event-order-on-resume-of-stopped-guest.patch @@ -0,0 +1,149 @@ +From b324993816789b4cb0c36fb8b3d8f97191b86d78 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Wed, 9 May 2018 14:42:21 +0200 +Subject: [PATCH 002/268] cpus: Fix event order on resume of stopped guest + +RH-Author: Markus Armbruster +Message-id: <20180509144221.14799-2-armbru@redhat.com> +Patchwork-id: 80191 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] cpus: Fix event order on resume of stopped guest +Bugzilla: 1566153 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Igor Mammedov + +When resume of a stopped guest immediately runs into block device +errors, the BLOCK_IO_ERROR event is sent before the RESUME event. + +Reproducer: + +1. Create a scratch image + $ dd if=/dev/zero of=scratch.img bs=1M count=100 + + Size doesn't actually matter. + +2. Prepare blkdebug configuration: + + $ cat >blkdebug.conf < ' + + Issue QMP command 'qmp_capabilities': + QMP> { "execute": "qmp_capabilities" } + +5. Boot the guest. + +6. In the guest, write to the scratch disk, e.g. like this: + + # dd if=/dev/zero of=/dev/vdb count=1 + + Do double-check the device specified with of= is actually the + scratch device! + +7. Issue QMP command 'cont': + QMP> { "execute": "cont" } + +After step 6, I get a BLOCK_IO_ERROR event followed by a STOP event. Good. + +After step 7, I get BLOCK_IO_ERROR, then RESUME, then STOP. Not so +good; I'd expect RESUME, then BLOCK_IO_ERROR, then STOP. + +The funny event order confuses libvirt: virsh -r domstate DOMAIN +--reason reports "paused (unknown)" rather than "paused (I/O error)". + +The culprit is vm_prepare_start(). + + /* Ensure that a STOP/RESUME pair of events is emitted if a + * vmstop request was pending. The BLOCK_IO_ERROR event, for + * example, according to documentation is always followed by + * the STOP event. + */ + if (runstate_is_running()) { + qapi_event_send_stop(&error_abort); + res = -1; + } else { + replay_enable_events(); + cpu_enable_ticks(); + runstate_set(RUN_STATE_RUNNING); + vm_state_notify(1, RUN_STATE_RUNNING); + } + + /* We are sending this now, but the CPUs will be resumed shortly later */ + qapi_event_send_resume(&error_abort); + return res; + +When resuming a stopped guest, we take the else branch before we get +to sending RESUME. vm_state_notify() runs virtio_vmstate_change(), +among other things. This restarts I/O, triggering the BLOCK_IO_ERROR +event. + +Reshuffle vm_prepare_start() to send the RESUME event earlier. + +Fixes RHBZ 1566153. + +Cc: Paolo Bonzini +Signed-off-by: Markus Armbruster +Message-Id: <20180423084518.2426-1-armbru@redhat.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit f056158d694d2adc63ff120ca71c73ae8b14426c) +Signed-off-by: Miroslav Rezanina +--- + cpus.c | 16 ++++++++-------- + 1 file changed, 8 insertions(+), 8 deletions(-) + +diff --git a/cpus.c b/cpus.c +index 38eba8b..398392b 100644 +--- a/cpus.c ++++ b/cpus.c +@@ -2043,7 +2043,6 @@ int vm_stop(RunState state) + int vm_prepare_start(void) + { + RunState requested; +- int res = 0; + + qemu_vmstop_requested(&requested); + if (runstate_is_running() && requested == RUN_STATE__MAX) { +@@ -2057,17 +2056,18 @@ int vm_prepare_start(void) + */ + if (runstate_is_running()) { + qapi_event_send_stop(&error_abort); +- res = -1; +- } else { +- replay_enable_events(); +- cpu_enable_ticks(); +- runstate_set(RUN_STATE_RUNNING); +- vm_state_notify(1, RUN_STATE_RUNNING); ++ qapi_event_send_resume(&error_abort); ++ return -1; + } + + /* We are sending this now, but the CPUs will be resumed shortly later */ + qapi_event_send_resume(&error_abort); +- return res; ++ ++ replay_enable_events(); ++ cpu_enable_ticks(); ++ runstate_set(RUN_STATE_RUNNING); ++ vm_state_notify(1, RUN_STATE_RUNNING); ++ return 0; + } + + void vm_start(void) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-cpus-ignore-ESRCH-in-qemu_cpu_kick_thread.patch b/SOURCES/kvm-cpus-ignore-ESRCH-in-qemu_cpu_kick_thread.patch new file mode 100644 index 0000000..9353b39 --- /dev/null +++ b/SOURCES/kvm-cpus-ignore-ESRCH-in-qemu_cpu_kick_thread.patch @@ -0,0 +1,58 @@ +From 657543de9cde26be992ca594b0717136709a4ae6 Mon Sep 17 00:00:00 2001 +From: Laurent Vivier +Date: Wed, 23 Jan 2019 08:45:16 +0000 +Subject: [PATCH 11/11] cpus: ignore ESRCH in qemu_cpu_kick_thread() + +RH-Author: Laurent Vivier +Message-id: <20190123084516.16337-1-lvivier@redhat.com> +Patchwork-id: 84091 +O-Subject: [RHEL8/rhel qemu-kvm PATCH] cpus: ignore ESRCH in qemu_cpu_kick_thread() +Bugzilla: 1665844 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Serhii Popovych +RH-Acked-by: Thomas Huth +RH-Acked-by: Laszlo Ersek + +We can have a race condition between qemu_cpu_kick_thread() and +qemu_kvm_cpu_thread_fn() when we hotunplug a CPU. In this case, +qemu_cpu_kick_thread() can try to kick a thread that is exiting. +pthread_kill() returns an error and qemu is stopped by an exit(1). + + qemu:qemu_cpu_kick_thread: No such process + +We can ignore safely this error. + +Signed-off-by: Laurent Vivier +Signed-off-by: Paolo Bonzini +(cherry picked from commit e9979ef245549b8e1fd240ec9937271c7fda0b57) +Signed-off-by: Laurent Vivier + +BRANCH: rhel8/master-2.12.0 +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1665844 +BREW: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=19905508 +UPSTREAM: In maintainer pull request + git://github.com/bonzini/qemu.git tags/for-upstream + 3cf01054d896fa88ea0dd31c5abb605c2e68bb29 +TEST: Upstream version tested by QE + +Signed-off-by: Danilo C. L. de Paula +--- + cpus.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/cpus.c b/cpus.c +index 398392b..6100089 100644 +--- a/cpus.c ++++ b/cpus.c +@@ -1700,7 +1700,7 @@ static void qemu_cpu_kick_thread(CPUState *cpu) + } + cpu->thread_kicked = true; + err = pthread_kill(cpu->thread->thread, SIG_IPI); +- if (err) { ++ if (err && err != ESRCH) { + fprintf(stderr, "qemu:%s: %s", __func__, strerror(err)); + exit(1); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-curl-Make-sslverify-off-disable-host-as-well-as-peer.patch b/SOURCES/kvm-curl-Make-sslverify-off-disable-host-as-well-as-peer.patch new file mode 100644 index 0000000..2db8769 --- /dev/null +++ b/SOURCES/kvm-curl-Make-sslverify-off-disable-host-as-well-as-peer.patch @@ -0,0 +1,78 @@ +From b914607db0576e1e0a4f49c58b12058f713b5b75 Mon Sep 17 00:00:00 2001 +From: Jeffrey Cody +Date: Wed, 26 Sep 2018 04:08:14 +0100 +Subject: [PATCH 4/4] curl: Make sslverify=off disable host as well as peer + verification. + +RH-Author: Jeffrey Cody +Message-id: <543d2f667af465dd809329fcba5175bc974d58d4.1537933576.git.jcody@redhat.com> +Patchwork-id: 82293 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] curl: Make sslverify=off disable host as well as peer verification. +Bugzilla: 1575925 +RH-Acked-by: Richard Jones +RH-Acked-by: John Snow +RH-Acked-by: Max Reitz + +From: "Richard W.M. Jones" + +The sslverify setting is supposed to turn off all TLS certificate +checks in libcurl. However because of the way we use it, it only +turns off peer certificate authenticity checks +(CURLOPT_SSL_VERIFYPEER). This patch makes it also turn off the check +that the server name in the certificate is the same as the server +you're connecting to (CURLOPT_SSL_VERIFYHOST). + +We can use Google's server at 8.8.8.8 which happens to have a bad TLS +certificate to demonstrate this: + +$ ./qemu-img create -q -f qcow2 -b 'json: { "file.sslverify": "off", "file.driver": "https", "file.url": "https://8.8.8.8/foo" }' /var/tmp/file.qcow2 +qemu-img: /var/tmp/file.qcow2: CURL: Error opening file: SSL: no alternative certificate subject name matches target host name '8.8.8.8' +Could not open backing image to determine size. + +With this patch applied, qemu-img connects to the server regardless of +the bad certificate: + +$ ./qemu-img create -q -f qcow2 -b 'json: { "file.sslverify": "off", "file.driver": "https", "file.url": "https://8.8.8.8/foo" }' /var/tmp/file.qcow2 +qemu-img: /var/tmp/file.qcow2: CURL: Error opening file: The requested URL returned error: 404 Not Found + +(The 404 error is expected because 8.8.8.8 is not actually serving a +file called "/foo".) + +Of course the default (without sslverify=off) remains to always check +the certificate: + +$ ./qemu-img create -q -f qcow2 -b 'json: { "file.driver": "https", "file.url": "https://8.8.8.8/foo" }' /var/tmp/file.qcow2 +qemu-img: /var/tmp/file.qcow2: CURL: Error opening file: SSL: no alternative certificate subject name matches target host name '8.8.8.8' +Could not open backing image to determine size. + +Further information about the two settings is available here: + +https://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYPEER.html +https://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html + +Signed-off-by: Richard W.M. Jones +Message-id: 20180914095622.19698-1-rjones@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit 637fa44ab80c6b317adf1d117494325a95daad60) +Signed-off-by: Jeff Cody +Signed-off-by: Danilo C. L. de Paula +--- + block/curl.c | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/block/curl.c b/block/curl.c +index aa42535..4d28f77 100644 +--- a/block/curl.c ++++ b/block/curl.c +@@ -483,6 +483,8 @@ static int curl_init_state(BDRVCURLState *s, CURLState *state) + curl_easy_setopt(state->curl, CURLOPT_URL, s->url); + curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYPEER, + (long) s->sslverify); ++ curl_easy_setopt(state->curl, CURLOPT_SSL_VERIFYHOST, ++ s->sslverify ? 2L : 0L); + if (s->cookie) { + curl_easy_setopt(state->curl, CURLOPT_COOKIE, s->cookie); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-curl-Support-auto-read-only-option.patch b/SOURCES/kvm-curl-Support-auto-read-only-option.patch new file mode 100644 index 0000000..9db8fc9 --- /dev/null +++ b/SOURCES/kvm-curl-Support-auto-read-only-option.patch @@ -0,0 +1,48 @@ +From 94b5efa066c1d0a05e62d9435e203afa431f8909 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:37 +0000 +Subject: [PATCH 07/14] curl: Support auto-read-only option + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-8-kwolf@redhat.com> +Patchwork-id: 83956 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 07/12] curl: Support auto-read-only option +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +If read-only=off, but auto-read-only=on is given, just degrade to +read-only. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 6ceef36acb11819b255732b1de0ca62885da04bd) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/curl.c | 8 ++++---- + 1 file changed, 4 insertions(+), 4 deletions(-) + +diff --git a/block/curl.c b/block/curl.c +index 4d28f77..f0df33a 100644 +--- a/block/curl.c ++++ b/block/curl.c +@@ -684,10 +684,10 @@ static int curl_open(BlockDriverState *bs, QDict *options, int flags, + const char *protocol_delimiter; + int ret; + +- +- if (flags & BDRV_O_RDWR) { +- error_setg(errp, "curl block device does not support writes"); +- return -EROFS; ++ ret = bdrv_apply_auto_read_only(bs, "curl driver does not support writes", ++ errp); ++ if (ret < 0) { ++ return ret; + } + + if (!libcurl_initialized) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-dirty-bitmap-fix-double-lock-on-bitmap-enabling.patch b/SOURCES/kvm-dirty-bitmap-fix-double-lock-on-bitmap-enabling.patch new file mode 100644 index 0000000..d3aba02 --- /dev/null +++ b/SOURCES/kvm-dirty-bitmap-fix-double-lock-on-bitmap-enabling.patch @@ -0,0 +1,74 @@ +From c2d0ba193251782ce258dff254b00848bfc3de5a Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:43 +0200 +Subject: [PATCH 225/268] dirty-bitmap: fix double lock on bitmap enabling + +RH-Author: John Snow +Message-id: <20180718225511.14878-8-jsnow@redhat.com> +Patchwork-id: 81403 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 07/35] dirty-bitmap: fix double lock on bitmap enabling +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Bitmap lock/unlock were added to bdrv_enable_dirty_bitmap in +8b1402ce80d, but some places were not updated correspondingly, which +leads to trying to take this lock twice, which is dead-lock. Fix this. + +Actually, iotest 199 (about dirty bitmap postcopy migration) is broken +now, and this fixes it. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-id: 20180625165745.25259-3-vsementsov@virtuozzo.com +Signed-off-by: John Snow +(cherry picked from commit 58f72b965e9e1820d246329461216c4d13140122) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/dirty-bitmap.c | 3 ++- + migration/block-dirty-bitmap.c | 4 ++-- + 2 files changed, 4 insertions(+), 3 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 4d6ae8b..f580c1a 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -259,8 +259,9 @@ void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap) + /* Called with BQL taken. */ + void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap) + { ++ assert(bitmap->mutex == bitmap->successor->mutex); + qemu_mutex_lock(bitmap->mutex); +- bdrv_enable_dirty_bitmap(bitmap->successor); ++ bdrv_enable_dirty_bitmap_locked(bitmap->successor); + qemu_mutex_unlock(bitmap->mutex); + } + +diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c +index dd04f10..abba0b6 100644 +--- a/migration/block-dirty-bitmap.c ++++ b/migration/block-dirty-bitmap.c +@@ -511,7 +511,7 @@ void dirty_bitmap_mig_before_vm_start(void) + DirtyBitmapLoadBitmapState *b = item->data; + + if (b->migrated) { +- bdrv_enable_dirty_bitmap(b->bitmap); ++ bdrv_enable_dirty_bitmap_locked(b->bitmap); + } else { + bdrv_dirty_bitmap_enable_successor(b->bitmap); + } +@@ -547,7 +547,7 @@ static void dirty_bitmap_load_complete(QEMUFile *f, DirtyBitmapLoadState *s) + if (enabled_bitmaps == NULL) { + /* in postcopy */ + bdrv_reclaim_dirty_bitmap_locked(s->bs, s->bitmap, &error_abort); +- bdrv_enable_dirty_bitmap(s->bitmap); ++ bdrv_enable_dirty_bitmap_locked(s->bitmap); + } else { + /* target not started, successor must be empty */ + int64_t count = bdrv_get_dirty_count(s->bitmap); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-dirty-bitmap-make-it-possible-to-restore-bitmap-afte.patch b/SOURCES/kvm-dirty-bitmap-make-it-possible-to-restore-bitmap-afte.patch new file mode 100644 index 0000000..fa0744e --- /dev/null +++ b/SOURCES/kvm-dirty-bitmap-make-it-possible-to-restore-bitmap-afte.patch @@ -0,0 +1,192 @@ +From 4122c0993f5b6de8db0227f5751e1f49e0b4354c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:13 +0000 +Subject: [PATCH 19/35] dirty-bitmap: make it possible to restore bitmap after + merge + +RH-Author: John Snow +Message-id: <20181120181828.15132-10-jsnow@redhat.com> +Patchwork-id: 83057 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 09/24] dirty-bitmap: make it possible to restore bitmap after merge +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Add backup parameter to bdrv_merge_dirty_bitmap() to be used then with +bdrv_restore_dirty_bitmap() if it needed to restore the bitmap after +merge operation. + +This is needed to implement bitmap merge transaction action in further +commit. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +(cherry picked from commit fa000f2f9fd96a75a0a33d50ead247fce11da92a) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 17 ++++++++++++++--- + blockdev.c | 2 +- + include/block/dirty-bitmap.h | 2 +- + include/qemu/hbitmap.h | 25 ++++++++++++++++--------- + util/hbitmap.c | 11 ++++++++--- + 5 files changed, 40 insertions(+), 17 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 017ee9d..8ac933c 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -314,7 +314,7 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs, + return NULL; + } + +- if (!hbitmap_merge(parent->bitmap, successor->bitmap)) { ++ if (!hbitmap_merge(parent->bitmap, successor->bitmap, parent->bitmap)) { + error_setg(errp, "Merging of parent and successor bitmap failed"); + return NULL; + } +@@ -791,8 +791,10 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset) + } + + void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, +- Error **errp) ++ HBitmap **backup, Error **errp) + { ++ bool ret; ++ + /* only bitmaps from one bds are supported */ + assert(dest->mutex == src->mutex); + +@@ -810,11 +812,20 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, + goto out; + } + +- if (!hbitmap_merge(dest->bitmap, src->bitmap)) { ++ if (!hbitmap_can_merge(dest->bitmap, src->bitmap)) { + error_setg(errp, "Bitmaps are incompatible and can't be merged"); + goto out; + } + ++ if (backup) { ++ *backup = dest->bitmap; ++ dest->bitmap = hbitmap_alloc(dest->size, hbitmap_granularity(*backup)); ++ ret = hbitmap_merge(*backup, src->bitmap, dest->bitmap); ++ } else { ++ ret = hbitmap_merge(dest->bitmap, src->bitmap, dest->bitmap); ++ } ++ assert(ret); ++ + out: + qemu_mutex_unlock(dest->mutex); + } +diff --git a/blockdev.c b/blockdev.c +index 2d86465..a10fbbd 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3077,7 +3077,7 @@ void qmp_x_block_dirty_bitmap_merge(const char *node, const char *dst_name, + return; + } + +- bdrv_merge_dirty_bitmap(dst, src, errp); ++ bdrv_merge_dirty_bitmap(dst, src, NULL, errp); + } + + BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node, +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index 259bd27..201ff7f 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -71,7 +71,7 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, + bool persistent); + void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked); + void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, +- Error **errp); ++ HBitmap **backup, Error **errp); + + /* Functions that require manual locking. */ + void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); +diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h +index ddca52c..a7cb780 100644 +--- a/include/qemu/hbitmap.h ++++ b/include/qemu/hbitmap.h +@@ -73,16 +73,23 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size); + + /** + * hbitmap_merge: +- * @a: The bitmap to store the result in. +- * @b: The bitmap to merge into @a. +- * @return true if the merge was successful, +- * false if it was not attempted. +- * +- * Merge two bitmaps together. +- * A := A (BITOR) B. +- * B is left unmodified. ++ * ++ * Store result of merging @a and @b into @result. ++ * @result is allowed to be equal to @a or @b. ++ * ++ * Return true if the merge was successful, ++ * false if it was not attempted. ++ */ ++bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result); ++ ++/** ++ * hbitmap_can_merge: ++ * ++ * hbitmap_can_merge(a, b) && hbitmap_can_merge(a, result) is sufficient and ++ * necessary for hbitmap_merge will not fail. ++ * + */ +-bool hbitmap_merge(HBitmap *a, const HBitmap *b); ++bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b); + + /** + * hbitmap_empty: +diff --git a/util/hbitmap.c b/util/hbitmap.c +index bcd3040..d5aca51 100644 +--- a/util/hbitmap.c ++++ b/util/hbitmap.c +@@ -723,6 +723,10 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) + } + } + ++bool hbitmap_can_merge(const HBitmap *a, const HBitmap *b) ++{ ++ return (a->size == b->size) && (a->granularity == b->granularity); ++} + + /** + * Given HBitmaps A and B, let A := A (BITOR) B. +@@ -731,14 +735,15 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) + * @return true if the merge was successful, + * false if it was not attempted. + */ +-bool hbitmap_merge(HBitmap *a, const HBitmap *b) ++bool hbitmap_merge(const HBitmap *a, const HBitmap *b, HBitmap *result) + { + int i; + uint64_t j; + +- if ((a->size != b->size) || (a->granularity != b->granularity)) { ++ if (!hbitmap_can_merge(a, b) || !hbitmap_can_merge(a, result)) { + return false; + } ++ assert(hbitmap_can_merge(b, result)); + + if (hbitmap_count(b) == 0) { + return true; +@@ -750,7 +755,7 @@ bool hbitmap_merge(HBitmap *a, const HBitmap *b) + */ + for (i = HBITMAP_LEVELS - 1; i >= 0; i--) { + for (j = 0; j < a->sizes[i]; j++) { +- a->levels[i][j] |= b->levels[i][j]; ++ result->levels[i][j] = a->levels[i][j] | b->levels[i][j]; + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-dirty-bitmap-rename-bdrv_undo_clear_dirty_bitmap.patch b/SOURCES/kvm-dirty-bitmap-rename-bdrv_undo_clear_dirty_bitmap.patch new file mode 100644 index 0000000..f324e5f --- /dev/null +++ b/SOURCES/kvm-dirty-bitmap-rename-bdrv_undo_clear_dirty_bitmap.patch @@ -0,0 +1,78 @@ +From 765f5eca1f61622a160e3e8fdf8337f00f4b7558 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:12 +0000 +Subject: [PATCH 18/35] dirty-bitmap: rename bdrv_undo_clear_dirty_bitmap + +RH-Author: John Snow +Message-id: <20181120181828.15132-9-jsnow@redhat.com> +Patchwork-id: 83061 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 08/24] dirty-bitmap: rename bdrv_undo_clear_dirty_bitmap +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Use more generic names to reuse the function for bitmap merge in the +following commit. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +(cherry picked from commit 56bd662497259400b7c9f155aaebaddde4450028) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 4 ++-- + blockdev.c | 2 +- + include/block/block_int.h | 2 +- + 3 files changed, 4 insertions(+), 4 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 6c8761e..017ee9d 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -633,12 +633,12 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out) + bdrv_dirty_bitmap_unlock(bitmap); + } + +-void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *in) ++void bdrv_restore_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *backup) + { + HBitmap *tmp = bitmap->bitmap; + assert(bdrv_dirty_bitmap_enabled(bitmap)); + assert(!bdrv_dirty_bitmap_readonly(bitmap)); +- bitmap->bitmap = in; ++ bitmap->bitmap = backup; + hbitmap_free(tmp); + } + +diff --git a/blockdev.c b/blockdev.c +index b3d265b..2d86465 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2142,7 +2142,7 @@ static void block_dirty_bitmap_clear_abort(BlkActionState *common) + common, common); + + if (state->backup) { +- bdrv_undo_clear_dirty_bitmap(state->bitmap, state->backup); ++ bdrv_restore_dirty_bitmap(state->bitmap, state->backup); + } + } + +diff --git a/include/block/block_int.h b/include/block/block_int.h +index ff923b7..f457acb 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -1141,7 +1141,7 @@ bool blk_dev_is_medium_locked(BlockBackend *blk); + void bdrv_set_dirty(BlockDriverState *bs, int64_t offset, int64_t bytes); + + void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap **out); +-void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *in); ++void bdrv_restore_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *backup); + + void bdrv_inc_in_flight(BlockDriverState *bs); + void bdrv_dec_in_flight(BlockDriverState *bs); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-dirty-bitmap-switch-assert-fails-to-errors-in-bdrv_m.patch b/SOURCES/kvm-dirty-bitmap-switch-assert-fails-to-errors-in-bdrv_m.patch new file mode 100644 index 0000000..c360f34 --- /dev/null +++ b/SOURCES/kvm-dirty-bitmap-switch-assert-fails-to-errors-in-bdrv_m.patch @@ -0,0 +1,88 @@ +From dd277015d70cd5521398d298dbc236b9aeb296f5 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:11 +0000 +Subject: [PATCH 17/35] dirty-bitmap: switch assert-fails to errors in + bdrv_merge_dirty_bitmap + +RH-Author: John Snow +Message-id: <20181120181828.15132-8-jsnow@redhat.com> +Patchwork-id: 83055 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 07/24] dirty-bitmap: switch assert-fails to errors in bdrv_merge_dirty_bitmap +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Move checks from qmp_x_block_dirty_bitmap_merge() to +bdrv_merge_dirty_bitmap(), to share them with dirty bitmap merge +transaction action in future commit. + +Note: for now, only qmp_x_block_dirty_bitmap_merge() calls +bdrv_merge_dirty_bitmap(). + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +(cherry picked from commit 06bf50068a7e952afff8c4f6470ec54a712570f7) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/dirty-bitmap.c | 15 +++++++++++++-- + blockdev.c | 10 ---------- + 2 files changed, 13 insertions(+), 12 deletions(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index c9b8a6f..6c8761e 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -798,12 +798,23 @@ void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, + + qemu_mutex_lock(dest->mutex); + +- assert(bdrv_dirty_bitmap_enabled(dest)); +- assert(!bdrv_dirty_bitmap_readonly(dest)); ++ if (bdrv_dirty_bitmap_frozen(dest)) { ++ error_setg(errp, "Bitmap '%s' is frozen and cannot be modified", ++ dest->name); ++ goto out; ++ } ++ ++ if (bdrv_dirty_bitmap_readonly(dest)) { ++ error_setg(errp, "Bitmap '%s' is readonly and cannot be modified", ++ dest->name); ++ goto out; ++ } + + if (!hbitmap_merge(dest->bitmap, src->bitmap)) { + error_setg(errp, "Bitmaps are incompatible and can't be merged"); ++ goto out; + } + ++out: + qemu_mutex_unlock(dest->mutex); + } +diff --git a/blockdev.c b/blockdev.c +index a722188..b3d265b 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3071,16 +3071,6 @@ void qmp_x_block_dirty_bitmap_merge(const char *node, const char *dst_name, + return; + } + +- if (bdrv_dirty_bitmap_frozen(dst)) { +- error_setg(errp, "Bitmap '%s' is frozen and cannot be modified", +- dst_name); +- return; +- } else if (bdrv_dirty_bitmap_readonly(dst)) { +- error_setg(errp, "Bitmap '%s' is readonly and cannot be modified", +- dst_name); +- return; +- } +- + src = bdrv_find_dirty_bitmap(bs, src_name); + if (!src) { + error_setg(errp, "Dirty bitmap '%s' not found", src_name); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-dirty-bitmaps-clean-up-bitmaps-loading-and-migration.patch b/SOURCES/kvm-dirty-bitmaps-clean-up-bitmaps-loading-and-migration.patch new file mode 100644 index 0000000..1ecc375 --- /dev/null +++ b/SOURCES/kvm-dirty-bitmaps-clean-up-bitmaps-loading-and-migration.patch @@ -0,0 +1,323 @@ +From ae232c169a5ae37d3989919775332979d4e3b7d5 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:26 +0000 +Subject: [PATCH 32/35] dirty-bitmaps: clean-up bitmaps loading and migration + logic + +RH-Author: John Snow +Message-id: <20181120181828.15132-23-jsnow@redhat.com> +Patchwork-id: 83074 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 22/24] dirty-bitmaps: clean-up bitmaps loading and migration logic +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +This patch aims to bring the following behavior: + +1. We don't load bitmaps, when started in inactive mode. It's the case +of incoming migration. In this case we wait for bitmaps migration +through migration channel (if 'dirty-bitmaps' capability is enabled) or +for invalidation (to load bitmaps from the image). + +2. We don't remove persistent bitmaps on inactivation. Instead, we only +remove bitmaps after storing. This is the only way to restore bitmaps, +if we decided to resume source after [failed] migration with +'dirty-bitmaps' capability enabled (which means, that bitmaps were not +stored). + +3. We load bitmaps on open and any invalidation, it's ok for all cases: + - normal open + - migration target invalidation with dirty-bitmaps capability + (bitmaps are migrating through migration channel, the are not + stored, so they should have IN_USE flag set and will be skipped + when loading. However, it would fail if bitmaps are read-only[1]) + - migration target invalidation without dirty-bitmaps capability + (normal load of the bitmaps, if migrated with shared storage) + - source invalidation with dirty-bitmaps capability + (skip because IN_USE) + - source invalidation without dirty-bitmaps capability + (bitmaps were dropped, reload them) + +[1]: to accurately handle this, migration of read-only bitmaps is + explicitly forbidden in this patch. + +New mechanism for not storing bitmaps when migrate with dirty-bitmaps +capability is introduced: migration filed in BdrvDirtyBitmap. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +(cherry picked from commit 9c98f145dfb994e1e9d68a4d606ee5693891280d) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block.c | 11 ++++--- + block/dirty-bitmap.c | 36 +++++++++-------------- + block/qcow2-bitmap.c | 16 +++++++++++ + block/qcow2.c | 65 ++++++++++++++++++++++++++++++++++++++++-- + include/block/dirty-bitmap.h | 2 +- + migration/block-dirty-bitmap.c | 10 +++++-- + 6 files changed, 109 insertions(+), 31 deletions(-) + +diff --git a/block.c b/block.c +index fbd569c..18ae591 100644 +--- a/block.c ++++ b/block.c +@@ -4316,6 +4316,7 @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, + uint64_t perm, shared_perm; + Error *local_err = NULL; + int ret; ++ BdrvDirtyBitmap *bm; + + if (!bs->drv) { + return; +@@ -4365,6 +4366,12 @@ static void coroutine_fn bdrv_co_invalidate_cache(BlockDriverState *bs, + } + } + ++ for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm; ++ bm = bdrv_dirty_bitmap_next(bs, bm)) ++ { ++ bdrv_dirty_bitmap_set_migration(bm, false); ++ } ++ + ret = refresh_total_sectors(bs, bs->total_sectors); + if (ret < 0) { + bs->open_flags |= BDRV_O_INACTIVE; +@@ -4479,10 +4486,6 @@ static int bdrv_inactivate_recurse(BlockDriverState *bs, + } + } + +- /* At this point persistent bitmaps should be already stored by the format +- * driver */ +- bdrv_release_persistent_dirty_bitmaps(bs); +- + return 0; + } + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 9b9ebd7..89fd1d7 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -55,6 +55,10 @@ struct BdrvDirtyBitmap { + and this bitmap must remain unchanged while + this flag is set. */ + bool persistent; /* bitmap must be saved to owner disk image */ ++ bool migration; /* Bitmap is selected for migration, it should ++ not be stored on the next inactivation ++ (persistent flag doesn't matter until next ++ invalidation).*/ + QLIST_ENTRY(BdrvDirtyBitmap) list; + }; + +@@ -390,26 +394,6 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs) + } + + /** +- * Release all persistent dirty bitmaps attached to a BDS (for use in +- * bdrv_inactivate_recurse()). +- * There must not be any frozen bitmaps attached. +- * This function does not remove persistent bitmaps from the storage. +- * Called with BQL taken. +- */ +-void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs) +-{ +- BdrvDirtyBitmap *bm, *next; +- +- bdrv_dirty_bitmaps_lock(bs); +- QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) { +- if (bdrv_dirty_bitmap_get_persistance(bm)) { +- bdrv_release_dirty_bitmap_locked(bm); +- } +- } +- bdrv_dirty_bitmaps_unlock(bs); +-} +- +-/** + * Remove persistent dirty bitmap from the storage if it exists. + * Absence of bitmap is not an error, because we have the following scenario: + * BdrvDirtyBitmap can have .persistent = true but not yet saved and have no +@@ -761,16 +745,24 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, bool persistent) + qemu_mutex_unlock(bitmap->mutex); + } + ++/* Called with BQL taken. */ ++void bdrv_dirty_bitmap_set_migration(BdrvDirtyBitmap *bitmap, bool migration) ++{ ++ qemu_mutex_lock(bitmap->mutex); ++ bitmap->migration = migration; ++ qemu_mutex_unlock(bitmap->mutex); ++} ++ + bool bdrv_dirty_bitmap_get_persistance(BdrvDirtyBitmap *bitmap) + { +- return bitmap->persistent; ++ return bitmap->persistent && !bitmap->migration; + } + + bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs) + { + BdrvDirtyBitmap *bm; + QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) { +- if (bm->persistent && !bm->readonly) { ++ if (bm->persistent && !bm->readonly && !bm->migration) { + return true; + } + } +diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c +index 14050eb..a36773c 100644 +--- a/block/qcow2-bitmap.c ++++ b/block/qcow2-bitmap.c +@@ -1418,6 +1418,22 @@ void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp) + g_free(tb); + } + ++ QSIMPLEQ_FOREACH(bm, bm_list, entry) { ++ /* For safety, we remove bitmap after storing. ++ * We may be here in two cases: ++ * 1. bdrv_close. It's ok to drop bitmap. ++ * 2. inactivation. It means migration without 'dirty-bitmaps' ++ * capability, so bitmaps are not marked with ++ * BdrvDirtyBitmap.migration flags. It's not bad to drop them too, ++ * and reload on invalidation. ++ */ ++ if (bm->dirty_bitmap == NULL) { ++ continue; ++ } ++ ++ bdrv_release_dirty_bitmap(bs, bm->dirty_bitmap); ++ } ++ + bitmap_list_free(bm_list); + return; + +diff --git a/block/qcow2.c b/block/qcow2.c +index d260cd6..36d1152 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1487,8 +1487,69 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, + s->autoclear_features &= QCOW2_AUTOCLEAR_MASK; + } + +- if (qcow2_load_dirty_bitmaps(bs, &local_err)) { +- update_header = false; ++ /* == Handle persistent dirty bitmaps == ++ * ++ * We want load dirty bitmaps in three cases: ++ * ++ * 1. Normal open of the disk in active mode, not related to invalidation ++ * after migration. ++ * ++ * 2. Invalidation of the target vm after pre-copy phase of migration, if ++ * bitmaps are _not_ migrating through migration channel, i.e. ++ * 'dirty-bitmaps' capability is disabled. ++ * ++ * 3. Invalidation of source vm after failed or canceled migration. ++ * This is a very interesting case. There are two possible types of ++ * bitmaps: ++ * ++ * A. Stored on inactivation and removed. They should be loaded from the ++ * image. ++ * ++ * B. Not stored: not-persistent bitmaps and bitmaps, migrated through ++ * the migration channel (with dirty-bitmaps capability). ++ * ++ * On the other hand, there are two possible sub-cases: ++ * ++ * 3.1 disk was changed by somebody else while were inactive. In this ++ * case all in-RAM dirty bitmaps (both persistent and not) are ++ * definitely invalid. And we don't have any method to determine ++ * this. ++ * ++ * Simple and safe thing is to just drop all the bitmaps of type B on ++ * inactivation. But in this case we lose bitmaps in valid 4.2 case. ++ * ++ * On the other hand, resuming source vm, if disk was already changed ++ * is a bad thing anyway: not only bitmaps, the whole vm state is ++ * out of sync with disk. ++ * ++ * This means, that user or management tool, who for some reason ++ * decided to resume source vm, after disk was already changed by ++ * target vm, should at least drop all dirty bitmaps by hand. ++ * ++ * So, we can ignore this case for now, but TODO: "generation" ++ * extension for qcow2, to determine, that image was changed after ++ * last inactivation. And if it is changed, we will drop (or at least ++ * mark as 'invalid' all the bitmaps of type B, both persistent ++ * and not). ++ * ++ * 3.2 disk was _not_ changed while were inactive. Bitmaps may be saved ++ * to disk ('dirty-bitmaps' capability disabled), or not saved ++ * ('dirty-bitmaps' capability enabled), but we don't need to care ++ * of: let's load bitmaps as always: stored bitmaps will be loaded, ++ * and not stored has flag IN_USE=1 in the image and will be skipped ++ * on loading. ++ * ++ * One remaining possible case when we don't want load bitmaps: ++ * ++ * 4. Open disk in inactive mode in target vm (bitmaps are migrating or ++ * will be loaded on invalidation, no needs try loading them before) ++ */ ++ ++ if (!(bdrv_get_flags(bs) & BDRV_O_INACTIVE)) { ++ /* It's case 1, 2 or 3.2. Or 3.1 which is BUG in management layer. */ ++ bool header_updated = qcow2_load_dirty_bitmaps(bs, &local_err); ++ ++ update_header = update_header && !header_updated; + } + if (local_err != NULL) { + error_propagate(errp, local_err); +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index 1463943..8f38a3d 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -26,7 +26,6 @@ BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, + const char *name); + void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap); + void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs); +-void bdrv_release_persistent_dirty_bitmaps(BlockDriverState *bs); + void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, + const char *name, + Error **errp); +@@ -72,6 +71,7 @@ void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, + void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked); + void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, + HBitmap **backup, Error **errp); ++void bdrv_dirty_bitmap_set_migration(BdrvDirtyBitmap *bitmap, bool migration); + + /* Functions that require manual locking. */ + void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); +diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c +index 47251af..ffe7aca 100644 +--- a/migration/block-dirty-bitmap.c ++++ b/migration/block-dirty-bitmap.c +@@ -307,6 +307,12 @@ static int init_dirty_bitmap_migration(void) + goto fail; + } + ++ if (bdrv_dirty_bitmap_readonly(bitmap)) { ++ error_report("Can't migrate read-only dirty bitmap: '%s", ++ bdrv_dirty_bitmap_name(bitmap)); ++ goto fail; ++ } ++ + bdrv_ref(bs); + bdrv_dirty_bitmap_set_qmp_locked(bitmap, true); + +@@ -329,9 +335,9 @@ static int init_dirty_bitmap_migration(void) + } + } + +- /* unset persistance here, to not roll back it */ ++ /* unset migration flags here, to not roll back it */ + QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { +- bdrv_dirty_bitmap_set_persistance(dbms->bitmap, false); ++ bdrv_dirty_bitmap_set_migration(dbms->bitmap, true); + } + + if (QSIMPLEQ_EMPTY(&dirty_bitmap_mig_state.dbms_list)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-docs-Document-the-new-default-sizes-of-the-qcow2-cac.patch b/SOURCES/kvm-docs-Document-the-new-default-sizes-of-the-qcow2-cac.patch new file mode 100644 index 0000000..19d6b10 --- /dev/null +++ b/SOURCES/kvm-docs-Document-the-new-default-sizes-of-the-qcow2-cac.patch @@ -0,0 +1,87 @@ +From e3455ccc52d50010fdbd1940d7efebd28c65f3ad Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:27 +0000 +Subject: [PATCH 02/15] docs: Document the new default sizes of the qcow2 + caches + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-3-kwolf@redhat.com> +Patchwork-id: 83297 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 02/15] docs: Document the new default sizes of the qcow2 caches +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Alberto Garcia + +We have just reduced the refcount cache size to the minimum unless +the user explicitly requests a larger one, so we have to update the +documentation to reflect this change. + +Signed-off-by: Alberto Garcia +Message-id: c5f0bde23558dd9d33b21fffc76ac9953cc19c56.1523968389.git.berto@igalia.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 603790ef3aec6a19b1c095188a1d2171934a27de) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + docs/qcow2-cache.txt | 33 ++++++++++++++++----------------- + 1 file changed, 16 insertions(+), 17 deletions(-) + +diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt +index 170191a..8a09a5c 100644 +--- a/docs/qcow2-cache.txt ++++ b/docs/qcow2-cache.txt +@@ -116,31 +116,30 @@ There are three options available, and all of them take bytes: + "refcount-cache-size": maximum size of the refcount block cache + "cache-size": maximum size of both caches combined + +-There are two things that need to be taken into account: ++There are a few things that need to be taken into account: + + - Both caches must have a size that is a multiple of the cluster size + (or the cache entry size: see "Using smaller cache sizes" below). + +- - If you only set one of the options above, QEMU will automatically +- adjust the others so that the L2 cache is 4 times bigger than the +- refcount cache. ++ - The default L2 cache size is 8 clusters or 1MB (whichever is more), ++ and the minimum is 2 clusters (or 2 cache entries, see below). + +-This means that these options are equivalent: ++ - The default (and minimum) refcount cache size is 4 clusters. + +- -drive file=hd.qcow2,l2-cache-size=2097152 +- -drive file=hd.qcow2,refcount-cache-size=524288 +- -drive file=hd.qcow2,cache-size=2621440 ++ - If only "cache-size" is specified then QEMU will assign as much ++ memory as possible to the L2 cache before increasing the refcount ++ cache size. + +-The reason for this 1/4 ratio is to ensure that both caches cover the +-same amount of disk space. Note however that this is only valid with +-the default value of refcount_bits (16). If you are using a different +-value you might want to calculate both cache sizes yourself since QEMU +-will always use the same 1/4 ratio. ++Unlike L2 tables, refcount blocks are not used during normal I/O but ++only during allocations and internal snapshots. In most cases they are ++accessed sequentially (even during random guest I/O) so increasing the ++refcount cache size won't have any measurable effect in performance ++(this can change if you are using internal snapshots, so you may want ++to think about increasing the cache size if you use them heavily). + +-It's also worth mentioning that there's no strict need for both caches +-to cover the same amount of disk space. The refcount cache is used +-much less often than the L2 cache, so it's perfectly reasonable to +-keep it small. ++Before QEMU 2.12 the refcount cache had a default size of 1/4 of the ++L2 cache size. This resulted in unnecessarily large caches, so now the ++refcount cache is as small as possible unless overridden by the user. + + + Using smaller cache entries +-- +1.8.3.1 + diff --git a/SOURCES/kvm-docs-interop-add-nbd.txt.patch b/SOURCES/kvm-docs-interop-add-nbd.txt.patch new file mode 100644 index 0000000..4aa0694 --- /dev/null +++ b/SOURCES/kvm-docs-interop-add-nbd.txt.patch @@ -0,0 +1,92 @@ +From 4ae2c1a9deaea0ed095bd960da92d87eda2eb70a Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:05 +0200 +Subject: [PATCH 247/268] docs/interop: add nbd.txt + +RH-Author: John Snow +Message-id: <20180718225511.14878-30-jsnow@redhat.com> +Patchwork-id: 81416 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 29/35] docs/interop: add nbd.txt +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Describe new metadata namespace: "qemu". + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180609151758.17343-7-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: grammar tweaks] +Signed-off-by: Eric Blake +(cherry picked from commit 3229a835a3c574a8ebc605e007785c4e01c61623) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + MAINTAINERS | 1 + + docs/interop/nbd.txt | 38 ++++++++++++++++++++++++++++++++++++++ + 2 files changed, 39 insertions(+) + create mode 100644 docs/interop/nbd.txt + +diff --git a/MAINTAINERS b/MAINTAINERS +index a783c92..53d5216 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1927,6 +1927,7 @@ F: nbd/ + F: include/block/nbd* + F: qemu-nbd.* + F: blockdev-nbd.c ++F: docs/interop/nbd.txt + T: git git://repo.or.cz/qemu/ericb.git nbd + + NFS +diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt +new file mode 100644 +index 0000000..77b5f45 +--- /dev/null ++++ b/docs/interop/nbd.txt +@@ -0,0 +1,38 @@ ++Qemu supports the NBD protocol, and has an internal NBD client (see ++block/nbd.c), an internal NBD server (see blockdev-nbd.c), and an ++external NBD server tool (see qemu-nbd.c). The common code is placed ++in nbd/*. ++ ++The NBD protocol is specified here: ++https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md ++ ++The following paragraphs describe some specific properties of NBD ++protocol realization in Qemu. ++ ++= Metadata namespaces = ++ ++Qemu supports the "base:allocation" metadata context as defined in the ++NBD protocol specification, and also defines an additional metadata ++namespace "qemu". ++ ++ ++== "qemu" namespace == ++ ++The "qemu" namespace currently contains only one type of context, ++related to exposing the contents of a dirty bitmap alongside the ++associated disk contents. That context has the following form: ++ ++ qemu:dirty-bitmap: ++ ++Each dirty-bitmap metadata context defines only one flag for extents ++in reply for NBD_CMD_BLOCK_STATUS: ++ ++ bit 0: NBD_STATE_DIRTY, means that the extent is "dirty" ++ ++For NBD_OPT_LIST_META_CONTEXT the following queries are supported ++in addition to "qemu:dirty-bitmap:": ++ ++* "qemu:" - returns list of all available metadata contexts in the ++ namespace. ++* "qemu:dirty-bitmap:" - returns list of all available dirty-bitmap ++ metadata contexts. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-e1000-Fix-tso_props-compat-for-82540em.patch b/SOURCES/kvm-e1000-Fix-tso_props-compat-for-82540em.patch new file mode 100644 index 0000000..1725be9 --- /dev/null +++ b/SOURCES/kvm-e1000-Fix-tso_props-compat-for-82540em.patch @@ -0,0 +1,47 @@ +From 8198c8d2a4f05691c2fd41ddf0b66392e6a28d65 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Thu, 26 Jul 2018 16:40:11 +0200 +Subject: [PATCH 265/268] e1000: Fix tso_props compat for 82540em + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180726164011.21660-2-dgilbert@redhat.com> +Patchwork-id: 81522 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 1/1] e1000: Fix tso_props compat for 82540em +Bugzilla: 1608778 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Juan Quintela +RH-Acked-by: Xiao Wang + +From: "Dr. David Alan Gilbert" + +In RHEL when you ask for an e1000-82540em that's the device you get, +however in upstream it's aliased to an e1000. +Because of the difference, we need to change the compat entry +for migrate_tso_props to the base type that changes e1000 as well. + +Signed-off-by: Dr. David Alan Gilbert +Signed-off-by: Miroslav Rezanina +--- + include/hw/compat.h | 5 +++-- + 1 file changed, 3 insertions(+), 2 deletions(-) + +diff --git a/include/hw/compat.h b/include/hw/compat.h +index f4cc6e0..c343e8f 100644 +--- a/include/hw/compat.h ++++ b/include/hw/compat.h +@@ -460,8 +460,9 @@ + .driver = "vhost-user-blk-pci",\ + .property = "vectors",\ + .value = "2",\ +- },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_11 */ \ +- .driver = "e1000",\ ++ },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_11 but \ ++ bz 1608778 modified for our naming */ \ ++ .driver = "e1000-82540em",\ + .property = "migrate_tso_props",\ + .value = "off",\ + },{ /* HW_COMPAT_RHEL7_5 from HW_COMPAT_2_10 */ \ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-e1000e-Do-not-auto-clear-ICR-bits-which-aren-t-set-i.patch b/SOURCES/kvm-e1000e-Do-not-auto-clear-ICR-bits-which-aren-t-set-i.patch new file mode 100644 index 0000000..c52297b --- /dev/null +++ b/SOURCES/kvm-e1000e-Do-not-auto-clear-ICR-bits-which-aren-t-set-i.patch @@ -0,0 +1,49 @@ +From 4b19800a747e124fed795b013c743c0297fff241 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Thu, 16 Aug 2018 08:17:06 +0100 +Subject: [PATCH 1/4] e1000e: Do not auto-clear ICR bits which aren't set in + EIAC + +RH-Author: Xiao Wang +Message-id: <1534407427-44794-2-git-send-email-jasowang@redhat.com> +Patchwork-id: 81854 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 1/2] e1000e: Do not auto-clear ICR bits which aren't set in EIAC +Bugzilla: 1596024 +RH-Acked-by: wexu@redhat.com +RH-Acked-by: Thomas Huth +RH-Acked-by: Paolo Bonzini + +From: Jan Kiszka + +The spec does not justify clearing of any E1000_ICR_OTHER_CAUSES when +E1000_ICR_OTHER is set in EIAC. In fact, removing this code fixes the +issue the Linux driver runs into since 4aea7a5c5e94 ("e1000e: Avoid +receiver overrun interrupt bursts") and was worked around by +745d0bd3af99 ("e1000e: Remove Other from EIAC"). + +Signed-off-by: Jan Kiszka +Signed-off-by: Jason Wang +(cherry picked from commit 2285a00c113469bb3e750ca4921cdb7baaae9e25) +Signed-off-by: Danilo C. L. de Paula +--- + hw/net/e1000e_core.c | 4 ---- + 1 file changed, 4 deletions(-) + +diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c +index c93c466..9504891 100644 +--- a/hw/net/e1000e_core.c ++++ b/hw/net/e1000e_core.c +@@ -2022,10 +2022,6 @@ e1000e_msix_notify_one(E1000ECore *core, uint32_t cause, uint32_t int_cfg) + + effective_eiac = core->mac[EIAC] & cause; + +- if (effective_eiac == E1000_ICR_OTHER) { +- effective_eiac |= E1000_ICR_OTHER_CAUSES; +- } +- + core->mac[ICR] &= ~effective_eiac; + + if (!(core->mac[CTRL_EXT] & E1000_CTRL_EXT_IAME)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-e1000e-Prevent-MSI-MSI-X-storms.patch b/SOURCES/kvm-e1000e-Prevent-MSI-MSI-X-storms.patch new file mode 100644 index 0000000..4808d52 --- /dev/null +++ b/SOURCES/kvm-e1000e-Prevent-MSI-MSI-X-storms.patch @@ -0,0 +1,85 @@ +From 6bbba130993de09d0623eafe648b978351cb49f9 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Thu, 16 Aug 2018 08:17:07 +0100 +Subject: [PATCH 2/4] e1000e: Prevent MSI/MSI-X storms + +RH-Author: Xiao Wang +Message-id: <1534407427-44794-3-git-send-email-jasowang@redhat.com> +Patchwork-id: 81853 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 2/2] e1000e: Prevent MSI/MSI-X storms +Bugzilla: 1596024 +RH-Acked-by: wexu@redhat.com +RH-Acked-by: Thomas Huth +RH-Acked-by: Paolo Bonzini + +From: Jan Kiszka + +Only signal MSI/MSI-X events on rising edges. So far we re-triggered the +interrupt sources even if the guest did no consumed the pending one, +easily causing interrupt storms. + +Issue was observable with Linux 4.16 e1000e driver when MSI-X was used. +Vector 2 was causing interrupt storms after the driver activated the +device. + +Signed-off-by: Jan Kiszka +Signed-off-by: Jason Wang +(cherry picked from commit 4712c158c5276fd3c401152f4bb5c3fccf185946) +Signed-off-by: Danilo C. L. de Paula +--- + hw/net/e1000e_core.c | 11 +++++++++++ + hw/net/e1000e_core.h | 2 ++ + 2 files changed, 13 insertions(+) + +diff --git a/hw/net/e1000e_core.c b/hw/net/e1000e_core.c +index 9504891..2a221c2 100644 +--- a/hw/net/e1000e_core.c ++++ b/hw/net/e1000e_core.c +@@ -2023,6 +2023,7 @@ e1000e_msix_notify_one(E1000ECore *core, uint32_t cause, uint32_t int_cfg) + effective_eiac = core->mac[EIAC] & cause; + + core->mac[ICR] &= ~effective_eiac; ++ core->msi_causes_pending &= ~effective_eiac; + + if (!(core->mac[CTRL_EXT] & E1000_CTRL_EXT_IAME)) { + core->mac[IMS] &= ~effective_eiac; +@@ -2119,6 +2120,13 @@ e1000e_send_msi(E1000ECore *core, bool msix) + { + uint32_t causes = core->mac[ICR] & core->mac[IMS] & ~E1000_ICR_ASSERTED; + ++ core->msi_causes_pending &= causes; ++ causes ^= core->msi_causes_pending; ++ if (causes == 0) { ++ return; ++ } ++ core->msi_causes_pending |= causes; ++ + if (msix) { + e1000e_msix_notify(core, causes); + } else { +@@ -2156,6 +2164,9 @@ e1000e_update_interrupt_state(E1000ECore *core) + core->mac[ICS] = core->mac[ICR]; + + interrupts_pending = (core->mac[IMS] & core->mac[ICR]) ? true : false; ++ if (!interrupts_pending) { ++ core->msi_causes_pending = 0; ++ } + + trace_e1000e_irq_pending_interrupts(core->mac[ICR] & core->mac[IMS], + core->mac[ICR], core->mac[IMS]); +diff --git a/hw/net/e1000e_core.h b/hw/net/e1000e_core.h +index 7d8ff41..63a1551 100644 +--- a/hw/net/e1000e_core.h ++++ b/hw/net/e1000e_core.h +@@ -109,6 +109,8 @@ struct E1000Core { + NICState *owner_nic; + PCIDevice *owner; + void (*owner_start_recv)(PCIDevice *d); ++ ++ uint32_t msi_causes_pending; + }; + + void +-- +1.8.3.1 + diff --git a/SOURCES/kvm-exec-check-that-alignment-is-a-power-of-two.patch b/SOURCES/kvm-exec-check-that-alignment-is-a-power-of-two.patch new file mode 100644 index 0000000..aac78a0 --- /dev/null +++ b/SOURCES/kvm-exec-check-that-alignment-is-a-power-of-two.patch @@ -0,0 +1,61 @@ +From f5eecfb16d182d3c37df06bf20e6843f126b33fe Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Sep 2018 10:18:00 +0100 +Subject: [PATCH 3/4] exec: check that alignment is a power of two + +RH-Author: David Hildenbrand +Message-id: <20180921101800.27360-1-david@redhat.com> +Patchwork-id: 82229 +O-Subject: [RHEL-8.0 qemu-kvm PATCH] exec: check that alignment is a power of two +Bugzilla: 1630746 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Igor Mammedov +RH-Acked-by: Paolo Bonzini + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1630746 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18439951 +Branch: rhel8/master-2.12.0 +Upstream: 61362b71c105ccb151ca16897a7d56534423f390 + +Right now we can crash QEMU using e.g. + +qemu-system-x86_64 -m 256M,maxmem=20G,slots=2 \ + -object memory-backend-file,id=mem0,size=12288,mem-path=/dev/zero,align=12288 \ + -device pc-dimm,id=dimm1,memdev=mem0 + +qemu-system-x86_64: util/mmap-alloc.c:115: + qemu_ram_mmap: Assertion `is_power_of_2(align)' failed + +Fix this by adding a proper check. + +Signed-off-by: David Hildenbrand +Message-Id: <20180607154705.6316-3-david@redhat.com> +Reviewed-by: Michael S. Tsirkin +Reviewed-by: Igor Mammedov +Signed-off-by: Paolo Bonzini +(cherry picked from commit 61362b71c105ccb151ca16897a7d56534423f390) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + exec.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/exec.c b/exec.c +index 02b1efe..22cc7ef 100644 +--- a/exec.c ++++ b/exec.c +@@ -1646,6 +1646,10 @@ static void *file_ram_alloc(RAMBlock *block, + " must be multiples of page size 0x%zx", + block->mr->align, block->page_size); + return NULL; ++ } else if (block->mr->align && !is_power_of_2(block->mr->align)) { ++ error_setg(errp, "alignment 0x%" PRIx64 ++ " must be a power of two", block->mr->align); ++ return NULL; + } + block->mr->align = MAX(block->page_size, block->mr->align); + #if defined(__s390x__) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-Fix-EINTR-handling.patch b/SOURCES/kvm-file-posix-Fix-EINTR-handling.patch new file mode 100644 index 0000000..ffd0faf --- /dev/null +++ b/SOURCES/kvm-file-posix-Fix-EINTR-handling.patch @@ -0,0 +1,65 @@ +From 8c14192b6417a5f07658c9434c10c4fb296574c8 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:53 +0200 +Subject: [PATCH 179/268] file-posix: Fix EINTR handling + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-14-famz@redhat.com> +Patchwork-id: 81164 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 13/13] file-posix: Fix EINTR handling +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +EINTR should be checked against errno, not ret. While fixing the bug, +collect the branches with a switch block. + +Also, change the return value from -ENOSTUP to -ENOSPC when the actual +issue is request range passes EOF, which should be distinguishable from +the case of error == ENOSYS by the caller, so that it could still retry +with other byte ranges, whereas it shouldn't retry anymore upon ENOSYS. + +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 17 +++++++++-------- + 1 file changed, 9 insertions(+), 8 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index 29ff699..0a9df5b 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -1449,20 +1449,21 @@ static ssize_t handle_aiocb_copy_range(RawPosixAIOData *aiocb) + ssize_t ret = copy_file_range(aiocb->aio_fildes, &in_off, + aiocb->aio_fd2, &out_off, + bytes, 0); +- if (ret == -EINTR) { +- continue; ++ if (ret == 0) { ++ /* No progress (e.g. when beyond EOF), let the caller fall back to ++ * buffer I/O. */ ++ return -ENOSPC; + } + if (ret < 0) { +- if (errno == ENOSYS) { ++ switch (errno) { ++ case ENOSYS: + return -ENOTSUP; +- } else { ++ case EINTR: ++ continue; ++ default: + return -errno; + } + } +- if (!ret) { +- /* No progress (e.g. when beyond EOF), fall back to buffer I/O. */ +- return -ENOTSUP; +- } + bytes -= ret; + } + return 0; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-Fix-creation-locking.patch b/SOURCES/kvm-file-posix-Fix-creation-locking.patch new file mode 100644 index 0000000..bbe87f2 --- /dev/null +++ b/SOURCES/kvm-file-posix-Fix-creation-locking.patch @@ -0,0 +1,56 @@ +From 36ce5f680149086b2feda220735255c2b31b0e00 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 9 Jul 2018 15:11:21 +0200 +Subject: [PATCH 203/268] file-posix: Fix creation locking + +RH-Author: Max Reitz +Message-id: <20180709151122.27541-2-mreitz@redhat.com> +Patchwork-id: 81271 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/2] file-posix: Fix creation locking +Bugzilla: 1599335 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +raw_apply_lock_bytes() takes a bit mask of "permissions that are NOT +shared". + +Also, make the "perm" and "shared" variables uint64_t, because I do not +particularly like using ~ on signed integers (and other permission masks +are usually uint64_t, too). + +Reported-by: Kevin Wolf +Signed-off-by: Max Reitz +Signed-off-by: Kevin Wolf +(cherry picked from commit d815efcaf01b1698e2fdf0f3e125201025c53191) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index 0a9df5b..e876770 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -2052,7 +2052,7 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + { + BlockdevCreateOptionsFile *file_opts; + int fd; +- int perm, shared; ++ uint64_t perm, shared; + int result = 0; + + /* Validate options and set default values */ +@@ -2088,7 +2088,7 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + shared = BLK_PERM_ALL & ~BLK_PERM_RESIZE; + + /* Step one: Take locks */ +- result = raw_apply_lock_bytes(fd, perm, shared, false, errp); ++ result = raw_apply_lock_bytes(fd, perm, ~shared, false, errp); + if (result < 0) { + goto out_close; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-Implement-bdrv_co_copy_range.patch b/SOURCES/kvm-file-posix-Implement-bdrv_co_copy_range.patch new file mode 100644 index 0000000..86d5ea8 --- /dev/null +++ b/SOURCES/kvm-file-posix-Implement-bdrv_co_copy_range.patch @@ -0,0 +1,261 @@ +From 38a444dbad000639aa36f51f10319be7bc78dabf Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:45 +0200 +Subject: [PATCH 171/268] file-posix: Implement bdrv_co_copy_range + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-6-famz@redhat.com> +Patchwork-id: 81156 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 05/13] file-posix: Implement bdrv_co_copy_range +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +With copy_file_range(2), we can implement the bdrv_co_copy_range +semantics. + +Signed-off-by: Fam Zheng +Message-id: 20180601092648.24614-6-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 1efad060d7e131dd52ecd1e038a6ddd37a3940c8) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++-- + configure | 17 +++++++++ + include/block/raw-aio.h | 10 ++++- + 3 files changed, 120 insertions(+), 5 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index 370a483..29ff699 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -59,6 +59,7 @@ + #ifdef __linux__ + #include + #include ++#include + #include + #include + #include +@@ -185,6 +186,8 @@ typedef struct RawPosixAIOData { + #define aio_ioctl_cmd aio_nbytes /* for QEMU_AIO_IOCTL */ + off_t aio_offset; + int aio_type; ++ int aio_fd2; ++ off_t aio_offset2; + } RawPosixAIOData; + + #if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) +@@ -1422,6 +1425,49 @@ static ssize_t handle_aiocb_write_zeroes(RawPosixAIOData *aiocb) + return -ENOTSUP; + } + ++#ifndef HAVE_COPY_FILE_RANGE ++static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd, ++ off_t *out_off, size_t len, unsigned int flags) ++{ ++#ifdef __NR_copy_file_range ++ return syscall(__NR_copy_file_range, in_fd, in_off, out_fd, ++ out_off, len, flags); ++#else ++ errno = ENOSYS; ++ return -1; ++#endif ++} ++#endif ++ ++static ssize_t handle_aiocb_copy_range(RawPosixAIOData *aiocb) ++{ ++ uint64_t bytes = aiocb->aio_nbytes; ++ off_t in_off = aiocb->aio_offset; ++ off_t out_off = aiocb->aio_offset2; ++ ++ while (bytes) { ++ ssize_t ret = copy_file_range(aiocb->aio_fildes, &in_off, ++ aiocb->aio_fd2, &out_off, ++ bytes, 0); ++ if (ret == -EINTR) { ++ continue; ++ } ++ if (ret < 0) { ++ if (errno == ENOSYS) { ++ return -ENOTSUP; ++ } else { ++ return -errno; ++ } ++ } ++ if (!ret) { ++ /* No progress (e.g. when beyond EOF), fall back to buffer I/O. */ ++ return -ENOTSUP; ++ } ++ bytes -= ret; ++ } ++ return 0; ++} ++ + static ssize_t handle_aiocb_discard(RawPosixAIOData *aiocb) + { + int ret = -EOPNOTSUPP; +@@ -1502,6 +1548,9 @@ static int aio_worker(void *arg) + case QEMU_AIO_WRITE_ZEROES: + ret = handle_aiocb_write_zeroes(aiocb); + break; ++ case QEMU_AIO_COPY_RANGE: ++ ret = handle_aiocb_copy_range(aiocb); ++ break; + default: + fprintf(stderr, "invalid aio request (0x%x)\n", aiocb->aio_type); + ret = -EINVAL; +@@ -1512,9 +1561,10 @@ static int aio_worker(void *arg) + return ret; + } + +-static int paio_submit_co(BlockDriverState *bs, int fd, +- int64_t offset, QEMUIOVector *qiov, +- int bytes, int type) ++static int paio_submit_co_full(BlockDriverState *bs, int fd, ++ int64_t offset, int fd2, int64_t offset2, ++ QEMUIOVector *qiov, ++ int bytes, int type) + { + RawPosixAIOData *acb = g_new(RawPosixAIOData, 1); + ThreadPool *pool; +@@ -1522,6 +1572,8 @@ static int paio_submit_co(BlockDriverState *bs, int fd, + acb->bs = bs; + acb->aio_type = type; + acb->aio_fildes = fd; ++ acb->aio_fd2 = fd2; ++ acb->aio_offset2 = offset2; + + acb->aio_nbytes = bytes; + acb->aio_offset = offset; +@@ -1537,6 +1589,13 @@ static int paio_submit_co(BlockDriverState *bs, int fd, + return thread_pool_submit_co(pool, aio_worker, acb); + } + ++static inline int paio_submit_co(BlockDriverState *bs, int fd, ++ int64_t offset, QEMUIOVector *qiov, ++ int bytes, int type) ++{ ++ return paio_submit_co_full(bs, fd, offset, -1, 0, qiov, bytes, type); ++} ++ + static BlockAIOCB *paio_submit(BlockDriverState *bs, int fd, + int64_t offset, QEMUIOVector *qiov, int bytes, + BlockCompletionFunc *cb, void *opaque, int type) +@@ -2346,6 +2405,35 @@ static void raw_abort_perm_update(BlockDriverState *bs) + raw_handle_perm_lock(bs, RAW_PL_ABORT, 0, 0, NULL); + } + ++static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, ++ BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, flags); ++} ++ ++static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, ++ BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ BDRVRawState *s = bs->opaque; ++ BDRVRawState *src_s; ++ ++ assert(dst->bs == bs); ++ if (src->bs->drv->bdrv_co_copy_range_to != raw_co_copy_range_to) { ++ return -ENOTSUP; ++ } ++ ++ src_s = src->bs->opaque; ++ if (fd_open(bs) < 0 || fd_open(bs) < 0) { ++ return -EIO; ++ } ++ return paio_submit_co_full(bs, src_s->fd, src_offset, s->fd, dst_offset, ++ NULL, bytes, QEMU_AIO_COPY_RANGE); ++} ++ + BlockDriver bdrv_file = { + .format_name = "file", + .protocol_name = "file", +@@ -2368,6 +2456,8 @@ BlockDriver bdrv_file = { + .bdrv_co_pwritev = raw_co_pwritev, + .bdrv_aio_flush = raw_aio_flush, + .bdrv_aio_pdiscard = raw_aio_pdiscard, ++ .bdrv_co_copy_range_from = raw_co_copy_range_from, ++ .bdrv_co_copy_range_to = raw_co_copy_range_to, + .bdrv_refresh_limits = raw_refresh_limits, + .bdrv_io_plug = raw_aio_plug, + .bdrv_io_unplug = raw_aio_unplug, +@@ -2845,6 +2935,8 @@ static BlockDriver bdrv_host_device = { + .bdrv_co_pwritev = raw_co_pwritev, + .bdrv_aio_flush = raw_aio_flush, + .bdrv_aio_pdiscard = hdev_aio_pdiscard, ++ .bdrv_co_copy_range_from = raw_co_copy_range_from, ++ .bdrv_co_copy_range_to = raw_co_copy_range_to, + .bdrv_refresh_limits = raw_refresh_limits, + .bdrv_io_plug = raw_aio_plug, + .bdrv_io_unplug = raw_aio_unplug, +diff --git a/configure b/configure +index 7358269..23d8d18 100755 +--- a/configure ++++ b/configure +@@ -5147,6 +5147,20 @@ if test "$fortify_source" != "no"; then + fi + fi + ++############################################### ++# Check if copy_file_range is provided by glibc ++have_copy_file_range=no ++cat > $TMPC << EOF ++#include ++int main(void) { ++ copy_file_range(0, NULL, 0, NULL, 0, 0); ++ return 0; ++} ++EOF ++if compile_prog "" "" ; then ++ have_copy_file_range=yes ++fi ++ + ########################################## + # check if struct fsxattr is available via linux/fs.h + +@@ -6221,6 +6235,9 @@ fi + if test "$have_fsxattr" = "yes" ; then + echo "HAVE_FSXATTR=y" >> $config_host_mak + fi ++if test "$have_copy_file_range" = "yes" ; then ++ echo "HAVE_COPY_FILE_RANGE=y" >> $config_host_mak ++fi + if test "$vte" = "yes" ; then + echo "CONFIG_VTE=y" >> $config_host_mak + echo "VTE_CFLAGS=$vte_cflags" >> $config_host_mak +diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h +index a4cdbbf..3240530 100644 +--- a/include/block/raw-aio.h ++++ b/include/block/raw-aio.h +@@ -25,9 +25,15 @@ + #define QEMU_AIO_FLUSH 0x0008 + #define QEMU_AIO_DISCARD 0x0010 + #define QEMU_AIO_WRITE_ZEROES 0x0020 ++#define QEMU_AIO_COPY_RANGE 0x0040 + #define QEMU_AIO_TYPE_MASK \ +- (QEMU_AIO_READ|QEMU_AIO_WRITE|QEMU_AIO_IOCTL|QEMU_AIO_FLUSH| \ +- QEMU_AIO_DISCARD|QEMU_AIO_WRITE_ZEROES) ++ (QEMU_AIO_READ | \ ++ QEMU_AIO_WRITE | \ ++ QEMU_AIO_IOCTL | \ ++ QEMU_AIO_FLUSH | \ ++ QEMU_AIO_DISCARD | \ ++ QEMU_AIO_WRITE_ZEROES | \ ++ QEMU_AIO_COPY_RANGE) + + /* AIO flags */ + #define QEMU_AIO_MISALIGNED 0x1000 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-Make-.bdrv_co_truncate-asynchronous.patch b/SOURCES/kvm-file-posix-Make-.bdrv_co_truncate-asynchronous.patch new file mode 100644 index 0000000..7ad2d35 --- /dev/null +++ b/SOURCES/kvm-file-posix-Make-.bdrv_co_truncate-asynchronous.patch @@ -0,0 +1,384 @@ +From f585ea7b67febdd86a2a1c9d53724a5bce625cad Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 24 Jul 2018 09:23:26 +0200 +Subject: [PATCH 213/268] file-posix: Make .bdrv_co_truncate asynchronous + +RH-Author: Kevin Wolf +Message-id: <20180712144258.17303-7-kwolf@redhat.com> +Patchwork-id: 81324 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 6/6] file-posix: Make .bdrv_co_truncate asynchronous +Bugzilla: 1595173 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: John Snow + +This moves the code to resize an image file to the thread pool to avoid +blocking. + +Creating large images with preallocation with blockdev-create is now +actually a background job instead of blocking the monitor (and most +other things) until the preallocation has completed. + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 93f4e2ff4b31205d8bab0856631a52ed442b8b1c) +Signed-off-by: Kevin Wolf +--- + block/file-posix.c | 266 +++++++++++++++++++++++++++--------------------- + include/block/raw-aio.h | 4 +- + 2 files changed, 154 insertions(+), 116 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index f8488ec..24c2367 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -186,8 +186,16 @@ typedef struct RawPosixAIOData { + #define aio_ioctl_cmd aio_nbytes /* for QEMU_AIO_IOCTL */ + off_t aio_offset; + int aio_type; +- int aio_fd2; +- off_t aio_offset2; ++ union { ++ struct { ++ int aio_fd2; ++ off_t aio_offset2; ++ }; ++ struct { ++ PreallocMode prealloc; ++ Error **errp; ++ }; ++ }; + } RawPosixAIOData; + + #if defined(__FreeBSD__) || defined(__FreeBSD_kernel__) +@@ -1509,6 +1517,122 @@ static ssize_t handle_aiocb_discard(RawPosixAIOData *aiocb) + return ret; + } + ++static int handle_aiocb_truncate(RawPosixAIOData *aiocb) ++{ ++ int result = 0; ++ int64_t current_length = 0; ++ char *buf = NULL; ++ struct stat st; ++ int fd = aiocb->aio_fildes; ++ int64_t offset = aiocb->aio_offset; ++ Error **errp = aiocb->errp; ++ ++ if (fstat(fd, &st) < 0) { ++ result = -errno; ++ error_setg_errno(errp, -result, "Could not stat file"); ++ return result; ++ } ++ ++ current_length = st.st_size; ++ if (current_length > offset && aiocb->prealloc != PREALLOC_MODE_OFF) { ++ error_setg(errp, "Cannot use preallocation for shrinking files"); ++ return -ENOTSUP; ++ } ++ ++ switch (aiocb->prealloc) { ++#ifdef CONFIG_POSIX_FALLOCATE ++ case PREALLOC_MODE_FALLOC: ++ /* ++ * Truncating before posix_fallocate() makes it about twice slower on ++ * file systems that do not support fallocate(), trying to check if a ++ * block is allocated before allocating it, so don't do that here. ++ */ ++ if (offset != current_length) { ++ result = -posix_fallocate(fd, current_length, ++ offset - current_length); ++ if (result != 0) { ++ /* posix_fallocate() doesn't set errno. */ ++ error_setg_errno(errp, -result, ++ "Could not preallocate new data"); ++ } ++ } else { ++ result = 0; ++ } ++ goto out; ++#endif ++ case PREALLOC_MODE_FULL: ++ { ++ int64_t num = 0, left = offset - current_length; ++ off_t seek_result; ++ ++ /* ++ * Knowing the final size from the beginning could allow the file ++ * system driver to do less allocations and possibly avoid ++ * fragmentation of the file. ++ */ ++ if (ftruncate(fd, offset) != 0) { ++ result = -errno; ++ error_setg_errno(errp, -result, "Could not resize file"); ++ goto out; ++ } ++ ++ buf = g_malloc0(65536); ++ ++ seek_result = lseek(fd, current_length, SEEK_SET); ++ if (seek_result < 0) { ++ result = -errno; ++ error_setg_errno(errp, -result, ++ "Failed to seek to the old end of file"); ++ goto out; ++ } ++ ++ while (left > 0) { ++ num = MIN(left, 65536); ++ result = write(fd, buf, num); ++ if (result < 0) { ++ result = -errno; ++ error_setg_errno(errp, -result, ++ "Could not write zeros for preallocation"); ++ goto out; ++ } ++ left -= result; ++ } ++ if (result >= 0) { ++ result = fsync(fd); ++ if (result < 0) { ++ result = -errno; ++ error_setg_errno(errp, -result, ++ "Could not flush file to disk"); ++ goto out; ++ } ++ } ++ goto out; ++ } ++ case PREALLOC_MODE_OFF: ++ if (ftruncate(fd, offset) != 0) { ++ result = -errno; ++ error_setg_errno(errp, -result, "Could not resize file"); ++ } ++ return result; ++ default: ++ result = -ENOTSUP; ++ error_setg(errp, "Unsupported preallocation mode: %s", ++ PreallocMode_str(aiocb->prealloc)); ++ return result; ++ } ++ ++out: ++ if (result < 0) { ++ if (ftruncate(fd, current_length) < 0) { ++ error_report("Failed to restore old file length: %s", ++ strerror(errno)); ++ } ++ } ++ ++ g_free(buf); ++ return result; ++} ++ + static int aio_worker(void *arg) + { + RawPosixAIOData *aiocb = arg; +@@ -1552,6 +1676,9 @@ static int aio_worker(void *arg) + case QEMU_AIO_COPY_RANGE: + ret = handle_aiocb_copy_range(aiocb); + break; ++ case QEMU_AIO_TRUNCATE: ++ ret = handle_aiocb_truncate(aiocb); ++ break; + default: + fprintf(stderr, "invalid aio request (0x%x)\n", aiocb->aio_type); + ret = -EINVAL; +@@ -1719,117 +1846,25 @@ static void raw_close(BlockDriverState *bs) + * + * Returns: 0 on success, -errno on failure. + */ +-static int raw_regular_truncate(int fd, int64_t offset, PreallocMode prealloc, +- Error **errp) ++static int coroutine_fn ++raw_regular_truncate(BlockDriverState *bs, int fd, int64_t offset, ++ PreallocMode prealloc, Error **errp) + { +- int result = 0; +- int64_t current_length = 0; +- char *buf = NULL; +- struct stat st; +- +- if (fstat(fd, &st) < 0) { +- result = -errno; +- error_setg_errno(errp, -result, "Could not stat file"); +- return result; +- } +- +- current_length = st.st_size; +- if (current_length > offset && prealloc != PREALLOC_MODE_OFF) { +- error_setg(errp, "Cannot use preallocation for shrinking files"); +- return -ENOTSUP; +- } +- +- switch (prealloc) { +-#ifdef CONFIG_POSIX_FALLOCATE +- case PREALLOC_MODE_FALLOC: +- /* +- * Truncating before posix_fallocate() makes it about twice slower on +- * file systems that do not support fallocate(), trying to check if a +- * block is allocated before allocating it, so don't do that here. +- */ +- if (offset != current_length) { +- result = -posix_fallocate(fd, current_length, offset - current_length); +- if (result != 0) { +- /* posix_fallocate() doesn't set errno. */ +- error_setg_errno(errp, -result, +- "Could not preallocate new data"); +- } +- } else { +- result = 0; +- } +- goto out; +-#endif +- case PREALLOC_MODE_FULL: +- { +- int64_t num = 0, left = offset - current_length; +- off_t seek_result; +- +- /* +- * Knowing the final size from the beginning could allow the file +- * system driver to do less allocations and possibly avoid +- * fragmentation of the file. +- */ +- if (ftruncate(fd, offset) != 0) { +- result = -errno; +- error_setg_errno(errp, -result, "Could not resize file"); +- goto out; +- } +- +- buf = g_malloc0(65536); +- +- seek_result = lseek(fd, current_length, SEEK_SET); +- if (seek_result < 0) { +- result = -errno; +- error_setg_errno(errp, -result, +- "Failed to seek to the old end of file"); +- goto out; +- } +- +- while (left > 0) { +- num = MIN(left, 65536); +- result = write(fd, buf, num); +- if (result < 0) { +- result = -errno; +- error_setg_errno(errp, -result, +- "Could not write zeros for preallocation"); +- goto out; +- } +- left -= result; +- } +- if (result >= 0) { +- result = fsync(fd); +- if (result < 0) { +- result = -errno; +- error_setg_errno(errp, -result, +- "Could not flush file to disk"); +- goto out; +- } +- } +- goto out; +- } +- case PREALLOC_MODE_OFF: +- if (ftruncate(fd, offset) != 0) { +- result = -errno; +- error_setg_errno(errp, -result, "Could not resize file"); +- } +- return result; +- default: +- result = -ENOTSUP; +- error_setg(errp, "Unsupported preallocation mode: %s", +- PreallocMode_str(prealloc)); +- return result; +- } ++ RawPosixAIOData *acb = g_new(RawPosixAIOData, 1); ++ ThreadPool *pool; + +-out: +- if (result < 0) { +- if (ftruncate(fd, current_length) < 0) { +- error_report("Failed to restore old file length: %s", +- strerror(errno)); +- } +- } ++ *acb = (RawPosixAIOData) { ++ .bs = bs, ++ .aio_fildes = fd, ++ .aio_type = QEMU_AIO_TRUNCATE, ++ .aio_offset = offset, ++ .prealloc = prealloc, ++ .errp = errp, ++ }; + +- g_free(buf); +- return result; ++ /* @bs can be NULL, bdrv_get_aio_context() returns the main context then */ ++ pool = aio_get_thread_pool(bdrv_get_aio_context(bs)); ++ return thread_pool_submit_co(pool, aio_worker, acb); + } + + static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, +@@ -1846,7 +1881,7 @@ static int coroutine_fn raw_co_truncate(BlockDriverState *bs, int64_t offset, + } + + if (S_ISREG(st.st_mode)) { +- return raw_regular_truncate(s->fd, offset, prealloc, errp); ++ return raw_regular_truncate(bs, s->fd, offset, prealloc, errp); + } + + if (prealloc != PREALLOC_MODE_OFF) { +@@ -2048,7 +2083,8 @@ static int64_t raw_get_allocated_file_size(BlockDriverState *bs) + return (int64_t)st.st_blocks * 512; + } + +-static int raw_co_create(BlockdevCreateOptions *options, Error **errp) ++static int coroutine_fn ++raw_co_create(BlockdevCreateOptions *options, Error **errp) + { + BlockdevCreateOptionsFile *file_opts; + Error *local_err = NULL; +@@ -2101,7 +2137,7 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + } + + /* Clear the file by truncating it to 0 */ +- result = raw_regular_truncate(fd, 0, PREALLOC_MODE_OFF, errp); ++ result = raw_regular_truncate(NULL, fd, 0, PREALLOC_MODE_OFF, errp); + if (result < 0) { + goto out_unlock; + } +@@ -2123,8 +2159,8 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + + /* Resize and potentially preallocate the file to the desired + * final size */ +- result = raw_regular_truncate(fd, file_opts->size, file_opts->preallocation, +- errp); ++ result = raw_regular_truncate(NULL, fd, file_opts->size, ++ file_opts->preallocation, errp); + if (result < 0) { + goto out_unlock; + } +diff --git a/include/block/raw-aio.h b/include/block/raw-aio.h +index 3240530..2ffcd9d 100644 +--- a/include/block/raw-aio.h ++++ b/include/block/raw-aio.h +@@ -26,6 +26,7 @@ + #define QEMU_AIO_DISCARD 0x0010 + #define QEMU_AIO_WRITE_ZEROES 0x0020 + #define QEMU_AIO_COPY_RANGE 0x0040 ++#define QEMU_AIO_TRUNCATE 0x0080 + #define QEMU_AIO_TYPE_MASK \ + (QEMU_AIO_READ | \ + QEMU_AIO_WRITE | \ +@@ -33,7 +34,8 @@ + QEMU_AIO_FLUSH | \ + QEMU_AIO_DISCARD | \ + QEMU_AIO_WRITE_ZEROES | \ +- QEMU_AIO_COPY_RANGE) ++ QEMU_AIO_COPY_RANGE | \ ++ QEMU_AIO_TRUNCATE) + + /* AIO flags */ + #define QEMU_AIO_MISALIGNED 0x1000 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-Support-auto-read-only-option.patch b/SOURCES/kvm-file-posix-Support-auto-read-only-option.patch new file mode 100644 index 0000000..9dcb3eb --- /dev/null +++ b/SOURCES/kvm-file-posix-Support-auto-read-only-option.patch @@ -0,0 +1,60 @@ +From 8aff951dc5acb2965131e43e10b1cd9fce17992e Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:36 +0000 +Subject: [PATCH 06/14] file-posix: Support auto-read-only option + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-7-kwolf@redhat.com> +Patchwork-id: 83955 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 06/12] file-posix: Support auto-read-only option +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +If read-only=off, but auto-read-only=on is given, open the file +read-write if we have the permissions, but instead of erroring out for +read-only files, just degrade to read-only. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 64107dc044a54ebe46348ac0fe87584be2eb3e81) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/file-posix.c | 19 ++++++++++++++++--- + 1 file changed, 16 insertions(+), 3 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index c12cdb7..7e6869d 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -517,9 +517,22 @@ static int raw_open_common(BlockDriverState *bs, QDict *options, + + s->fd = -1; + fd = qemu_open(filename, s->open_flags, 0644); +- if (fd < 0) { +- ret = -errno; +- error_setg_errno(errp, errno, "Could not open '%s'", filename); ++ ret = fd < 0 ? -errno : 0; ++ ++ if (ret == -EACCES || ret == -EROFS) { ++ /* Try to degrade to read-only, but if it doesn't work, still use the ++ * normal error message. */ ++ if (bdrv_apply_auto_read_only(bs, NULL, NULL) == 0) { ++ bdrv_flags &= ~BDRV_O_RDWR; ++ raw_parse_flags(bdrv_flags, &s->open_flags); ++ assert(!(s->open_flags & O_CREAT)); ++ fd = qemu_open(filename, s->open_flags); ++ ret = fd < 0 ? -errno : 0; ++ } ++ } ++ ++ if (ret < 0) { ++ error_setg_errno(errp, -ret, "Could not open '%s'", filename); + if (ret == -EROFS) { + ret = -EACCES; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-Unlock-FD-after-creation.patch b/SOURCES/kvm-file-posix-Unlock-FD-after-creation.patch new file mode 100644 index 0000000..39b00ca --- /dev/null +++ b/SOURCES/kvm-file-posix-Unlock-FD-after-creation.patch @@ -0,0 +1,77 @@ +From ecf9ad9048427f1b63a250d9dd595264cf03e256 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 9 Jul 2018 15:11:22 +0200 +Subject: [PATCH 204/268] file-posix: Unlock FD after creation + +RH-Author: Max Reitz +Message-id: <20180709151122.27541-3-mreitz@redhat.com> +Patchwork-id: 81269 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/2] file-posix: Unlock FD after creation +Bugzilla: 1599335 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +Closing the FD does not necessarily mean that it is unlocked. Fix this +by relinquishing all permission locks before qemu_close(). + +Reported-by: Kevin Wolf +Signed-off-by: Max Reitz +Signed-off-by: Kevin Wolf +(cherry picked from commit 7c20c808a5cbf5d244735bc78fc3138c739c1946) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 17 ++++++++++++++--- + 1 file changed, 14 insertions(+), 3 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index e876770..cbf7c11 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -2051,6 +2051,7 @@ static int64_t raw_get_allocated_file_size(BlockDriverState *bs) + static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + { + BlockdevCreateOptionsFile *file_opts; ++ Error *local_err = NULL; + int fd; + uint64_t perm, shared; + int result = 0; +@@ -2096,13 +2097,13 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + /* Step two: Check that nobody else has taken conflicting locks */ + result = raw_check_lock_bytes(fd, perm, shared, errp); + if (result < 0) { +- goto out_close; ++ goto out_unlock; + } + + /* Clear the file by truncating it to 0 */ + result = raw_regular_truncate(fd, 0, PREALLOC_MODE_OFF, errp); + if (result < 0) { +- goto out_close; ++ goto out_unlock; + } + + if (file_opts->nocow) { +@@ -2125,7 +2126,17 @@ static int raw_co_create(BlockdevCreateOptions *options, Error **errp) + result = raw_regular_truncate(fd, file_opts->size, file_opts->preallocation, + errp); + if (result < 0) { +- goto out_close; ++ goto out_unlock; ++ } ++ ++out_unlock: ++ raw_apply_lock_bytes(fd, 0, 0, true, &local_err); ++ if (local_err) { ++ /* The above call should not fail, and if it does, that does ++ * not mean the whole creation operation has failed. So ++ * report it the user for their convenience, but do not report ++ * it to the caller. */ ++ error_report_err(local_err); + } + + out_close: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-file-posix-specify-expected-filetypes.patch b/SOURCES/kvm-file-posix-specify-expected-filetypes.patch new file mode 100644 index 0000000..3ec8d73 --- /dev/null +++ b/SOURCES/kvm-file-posix-specify-expected-filetypes.patch @@ -0,0 +1,151 @@ +From 2e757d702c366a1eb58abe33ed39331253bfa851 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Fri, 13 Jul 2018 14:50:01 +0200 +Subject: [PATCH 217/268] file-posix: specify expected filetypes + +RH-Author: Kevin Wolf +Message-id: <20180713145002.20953-2-kwolf@redhat.com> +Patchwork-id: 81350 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/2] file-posix: specify expected filetypes +Bugzilla: 1525829 +RH-Acked-by: John Snow +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: John Snow + +Adjust each caller of raw_open_common to specify if they are expecting +host and character devices or not. Tighten expectations of file types upon +open in the common code and refuse types that are not expected. + +This has two effects: + +(1) Character and block devices are now considered deprecated for the + 'file' driver, which expects only S_IFREG, and +(2) no file-posix driver (file, host_cdrom, or host_device) can open + directories now. + +I don't think there's a legitimate reason to open directories as if +they were files. This prevents QEMU from opening and attempting to probe +a directory inode, which can break in exciting ways. One of those ways +is lseek on ext4/xfs, which will return 0x7fffffffffffffff as the file +size instead of EISDIR. This can coax QEMU into responding with a +confusing "file too big" instead of "Hey, that's not a file". + +See: https://bugs.launchpad.net/qemu/+bug/1739304/ +Signed-off-by: John Snow +Signed-off-by: Kevin Wolf +(cherry picked from commit 230ff73904e72dde2d7718c2da407786a1c72e57) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/file-posix.c | 39 +++++++++++++++++++++++++++++++-------- + qemu-doc.texi | 6 ++++++ + 2 files changed, 37 insertions(+), 8 deletions(-) + +diff --git a/block/file-posix.c b/block/file-posix.c +index 24c2367..06ec67d 100644 +--- a/block/file-posix.c ++++ b/block/file-posix.c +@@ -431,7 +431,8 @@ static QemuOptsList raw_runtime_opts = { + }; + + static int raw_open_common(BlockDriverState *bs, QDict *options, +- int bdrv_flags, int open_flags, Error **errp) ++ int bdrv_flags, int open_flags, ++ bool device, Error **errp) + { + BDRVRawState *s = bs->opaque; + QemuOpts *opts; +@@ -569,10 +570,32 @@ static int raw_open_common(BlockDriverState *bs, QDict *options, + error_setg_errno(errp, errno, "Could not stat file"); + goto fail; + } +- if (S_ISREG(st.st_mode)) { +- s->discard_zeroes = true; +- s->has_fallocate = true; ++ ++ if (!device) { ++ if (S_ISBLK(st.st_mode)) { ++ warn_report("Opening a block device as a file using the '%s' " ++ "driver is deprecated", bs->drv->format_name); ++ } else if (S_ISCHR(st.st_mode)) { ++ warn_report("Opening a character device as a file using the '%s' " ++ "driver is deprecated", bs->drv->format_name); ++ } else if (!S_ISREG(st.st_mode)) { ++ error_setg(errp, "A regular file was expected by the '%s' driver, " ++ "but something else was given", bs->drv->format_name); ++ ret = -EINVAL; ++ goto fail; ++ } else { ++ s->discard_zeroes = true; ++ s->has_fallocate = true; ++ } ++ } else { ++ if (!(S_ISCHR(st.st_mode) || S_ISBLK(st.st_mode))) { ++ error_setg(errp, "'%s' driver expects either " ++ "a character or block device", bs->drv->format_name); ++ ret = -EINVAL; ++ goto fail; ++ } + } ++ + if (S_ISBLK(st.st_mode)) { + #ifdef BLKDISCARDZEROES + unsigned int arg; +@@ -625,7 +648,7 @@ static int raw_open(BlockDriverState *bs, QDict *options, int flags, + BDRVRawState *s = bs->opaque; + + s->type = FTYPE_FILE; +- return raw_open_common(bs, options, flags, 0, errp); ++ return raw_open_common(bs, options, flags, 0, false, errp); + } + + typedef enum { +@@ -2794,7 +2817,7 @@ hdev_open_Mac_error: + + s->type = FTYPE_FILE; + +- ret = raw_open_common(bs, options, flags, 0, &local_err); ++ ret = raw_open_common(bs, options, flags, 0, true, &local_err); + if (ret < 0) { + error_propagate(errp, local_err); + #if defined(__APPLE__) && defined(__MACH__) +@@ -3023,7 +3046,7 @@ static int cdrom_open(BlockDriverState *bs, QDict *options, int flags, + s->type = FTYPE_CD; + + /* open will not fail even if no CD is inserted, so add O_NONBLOCK */ +- return raw_open_common(bs, options, flags, O_NONBLOCK, errp); ++ return raw_open_common(bs, options, flags, O_NONBLOCK, true, errp); + } + + static int cdrom_probe_device(const char *filename) +@@ -3136,7 +3159,7 @@ static int cdrom_open(BlockDriverState *bs, QDict *options, int flags, + + s->type = FTYPE_CD; + +- ret = raw_open_common(bs, options, flags, 0, &local_err); ++ ret = raw_open_common(bs, options, flags, 0, true, &local_err); + if (ret) { + error_propagate(errp, local_err); + return ret; +diff --git a/qemu-doc.texi b/qemu-doc.texi +index de5097a..985e0f2 100644 +--- a/qemu-doc.texi ++++ b/qemu-doc.texi +@@ -2938,6 +2938,12 @@ The @code{-startdate} option has been replaced by @code{-rtc base=@var{date}}. + The ``convert -s snapshot_id_or_name'' argument is obsoleted + by the ``convert -l snapshot_param'' argument instead. + ++@subsection -drive file=json:@{...@{'driver':'file'@}@} (since 3.0) ++ ++The 'file' driver for drives is no longer appropriate for character or host ++devices and will only accept regular files (S_IFREG). The correct driver ++for these file types is 'host_cdrom' or 'host_device' as appropriate. ++ + @section QEMU Machine Protocol (QMP) commands + + @subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-gluster-Support-auto-read-only-option.patch b/SOURCES/kvm-gluster-Support-auto-read-only-option.patch new file mode 100644 index 0000000..e137fcc --- /dev/null +++ b/SOURCES/kvm-gluster-Support-auto-read-only-option.patch @@ -0,0 +1,53 @@ +From 6508e23a5053680509754d719a19b04754e1fbdc Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:38 +0000 +Subject: [PATCH 08/14] gluster: Support auto-read-only option + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-9-kwolf@redhat.com> +Patchwork-id: 83959 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 08/12] gluster: Support auto-read-only option +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +If read-only=off, but auto-read-only=on is given, open the file +read-write if we have the permissions, but instead of erroring out for +read-only files, just degrade to read-only. + +Signed-off-by: Kevin Wolf +Reviewed-by: Niels de Vos +(cherry picked from commit 54ea21bd16202c4a3e43c67b573b5d1aa2ec1c0c) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/gluster.c | 12 ++++++++++-- + 1 file changed, 10 insertions(+), 2 deletions(-) + +diff --git a/block/gluster.c b/block/gluster.c +index cecfe09..8c13002 100644 +--- a/block/gluster.c ++++ b/block/gluster.c +@@ -849,8 +849,16 @@ static int qemu_gluster_open(BlockDriverState *bs, QDict *options, + qemu_gluster_parse_flags(bdrv_flags, &open_flags); + + s->fd = glfs_open(s->glfs, gconf->path, open_flags); +- if (!s->fd) { +- ret = -errno; ++ ret = s->fd ? 0 : -errno; ++ ++ if (ret == -EACCES || ret == -EROFS) { ++ /* Try to degrade to read-only, but if it doesn't work, still use the ++ * normal error message. */ ++ if (bdrv_apply_auto_read_only(bs, NULL, NULL) == 0) { ++ open_flags = (open_flags & ~O_RDWR) | O_RDONLY; ++ s->fd = glfs_open(s->glfs, gconf->path, open_flags); ++ ret = s->fd ? 0 : -errno; ++ } + } + + s->supports_seek_data = qemu_gluster_test_seek(s->fd); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hbitmap-Add-advance-param-to-hbitmap_iter_next.patch b/SOURCES/kvm-hbitmap-Add-advance-param-to-hbitmap_iter_next.patch new file mode 100644 index 0000000..18d6e01 --- /dev/null +++ b/SOURCES/kvm-hbitmap-Add-advance-param-to-hbitmap_iter_next.patch @@ -0,0 +1,182 @@ +From d3cab6731c84ccc36d7a2655451486d1877afcf0 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:07 +0000 +Subject: [PATCH 13/35] hbitmap: Add @advance param to hbitmap_iter_next() + +RH-Author: John Snow +Message-id: <20181120181828.15132-4-jsnow@redhat.com> +Patchwork-id: 83063 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 03/24] hbitmap: Add @advance param to hbitmap_iter_next() +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Max Reitz + +This new parameter allows the caller to just query the next dirty +position without moving the iterator. + +Signed-off-by: Max Reitz +Reviewed-by: Fam Zheng +Reviewed-by: John Snow +Message-id: 20180613181823.13618-8-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit a33fbb4f8b64226becf502a123733776ce319b24) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/backup.c | 2 +- + block/dirty-bitmap.c | 2 +- + include/qemu/hbitmap.h | 5 ++++- + tests/test-hbitmap.c | 26 +++++++++++++------------- + util/hbitmap.c | 10 +++++++--- + 5 files changed, 26 insertions(+), 19 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 524e0ff..ac17db6 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -409,7 +409,7 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job) + HBitmapIter hbi; + + hbitmap_iter_init(&hbi, job->copy_bitmap, 0); +- while ((cluster = hbitmap_iter_next(&hbi)) != -1) { ++ while ((cluster = hbitmap_iter_next(&hbi, true)) != -1) { + do { + if (yield_and_check(job)) { + return 0; +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index dac8d74..236dce1 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -525,7 +525,7 @@ void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter) + + int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter) + { +- return hbitmap_iter_next(&iter->hbi); ++ return hbitmap_iter_next(&iter->hbi, true); + } + + /* Called within bdrv_dirty_bitmap_lock..unlock */ +diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h +index 6b6490e..ddca52c 100644 +--- a/include/qemu/hbitmap.h ++++ b/include/qemu/hbitmap.h +@@ -324,11 +324,14 @@ void hbitmap_free_meta(HBitmap *hb); + /** + * hbitmap_iter_next: + * @hbi: HBitmapIter to operate on. ++ * @advance: If true, advance the iterator. Otherwise, the next call ++ * of this function will return the same result (if that ++ * position is still dirty). + * + * Return the next bit that is set in @hbi's associated HBitmap, + * or -1 if all remaining bits are zero. + */ +-int64_t hbitmap_iter_next(HBitmapIter *hbi); ++int64_t hbitmap_iter_next(HBitmapIter *hbi, bool advance); + + /** + * hbitmap_iter_next_word: +diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c +index f29631f..f2158f7 100644 +--- a/tests/test-hbitmap.c ++++ b/tests/test-hbitmap.c +@@ -46,7 +46,7 @@ static void hbitmap_test_check(TestHBitmapData *data, + + i = first; + for (;;) { +- next = hbitmap_iter_next(&hbi); ++ next = hbitmap_iter_next(&hbi, true); + if (next < 0) { + next = data->size; + } +@@ -435,25 +435,25 @@ static void test_hbitmap_iter_granularity(TestHBitmapData *data, + /* Note that hbitmap_test_check has to be invoked manually in this test. */ + hbitmap_test_init(data, 131072 << 7, 7); + hbitmap_iter_init(&hbi, data->hb, 0); +- g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); + + hbitmap_test_set(data, ((L2 + L1 + 1) << 7) + 8, 8); + hbitmap_iter_init(&hbi, data->hb, 0); +- g_assert_cmpint(hbitmap_iter_next(&hbi), ==, (L2 + L1 + 1) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, (L2 + L1 + 1) << 7); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); + + hbitmap_iter_init(&hbi, data->hb, (L2 + L1 + 2) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); + + hbitmap_test_set(data, (131072 << 7) - 8, 8); + hbitmap_iter_init(&hbi, data->hb, 0); +- g_assert_cmpint(hbitmap_iter_next(&hbi), ==, (L2 + L1 + 1) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi), ==, 131071 << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, (L2 + L1 + 1) << 7); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, 131071 << 7); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); + + hbitmap_iter_init(&hbi, data->hb, (L2 + L1 + 2) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi), ==, 131071 << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi), <, 0); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, 131071 << 7); ++ g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); + } + + static void hbitmap_test_set_boundary_bits(TestHBitmapData *data, ssize_t diff) +@@ -893,7 +893,7 @@ static void test_hbitmap_serialize_zeroes(TestHBitmapData *data, + for (i = 0; i < num_positions; i++) { + hbitmap_deserialize_zeroes(data->hb, positions[i], min_l1, true); + hbitmap_iter_init(&iter, data->hb, 0); +- next = hbitmap_iter_next(&iter); ++ next = hbitmap_iter_next(&iter, true); + if (i == num_positions - 1) { + g_assert_cmpint(next, ==, -1); + } else { +@@ -919,10 +919,10 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData *data, + + hbitmap_iter_init(&hbi, data->hb, BITS_PER_LONG - 1); + +- hbitmap_iter_next(&hbi); ++ hbitmap_iter_next(&hbi, true); + + hbitmap_reset_all(data->hb); +- hbitmap_iter_next(&hbi); ++ hbitmap_iter_next(&hbi, true); + } + + static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start) +diff --git a/util/hbitmap.c b/util/hbitmap.c +index 58a2c93..bcd3040 100644 +--- a/util/hbitmap.c ++++ b/util/hbitmap.c +@@ -141,7 +141,7 @@ unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi) + return cur; + } + +-int64_t hbitmap_iter_next(HBitmapIter *hbi) ++int64_t hbitmap_iter_next(HBitmapIter *hbi, bool advance) + { + unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1] & + hbi->hb->levels[HBITMAP_LEVELS - 1][hbi->pos]; +@@ -154,8 +154,12 @@ int64_t hbitmap_iter_next(HBitmapIter *hbi) + } + } + +- /* The next call will resume work from the next bit. */ +- hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1); ++ if (advance) { ++ /* The next call will resume work from the next bit. */ ++ hbi->cur[HBITMAP_LEVELS - 1] = cur & (cur - 1); ++ } else { ++ hbi->cur[HBITMAP_LEVELS - 1] = cur; ++ } + item = ((uint64_t)hbi->pos << BITS_PER_LEVEL) + ctzl(cur); + + return item << hbi->granularity; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hostmem-file-add-the-pmem-option.patch b/SOURCES/kvm-hostmem-file-add-the-pmem-option.patch new file mode 100644 index 0000000..c715327 --- /dev/null +++ b/SOURCES/kvm-hostmem-file-add-the-pmem-option.patch @@ -0,0 +1,274 @@ +From b872c9566d3f18b4d6580a7b271bbbc21c026a36 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:19 +0000 +Subject: [PATCH 18/22] hostmem-file: add the 'pmem' option + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-7-git-send-email-plai@redhat.com> +Patchwork-id: 83892 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 06/10] hostmem-file: add the 'pmem' option +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +When QEMU emulates vNVDIMM labels and migrates vNVDIMM devices, it +needs to know whether the backend storage is a real persistent memory, +in order to decide whether special operations should be performed to +ensure the data persistence. + +This boolean option 'pmem' allows users to specify whether the backend +storage of memory-backend-file is a real persistent memory. If +'pmem=on', QEMU will set the flag RAM_PMEM in the RAM block of the +corresponding memory region. If 'pmem' is set while lack of libpmem +support, a error is generated. + +Signed-off-by: Junyan He +Signed-off-by: Haozhong Zhang +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Richard Henderson +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit a4de8552b2580adf6fa4874439217b65d3bdd88b) +Signed-off-by: Paul Lai + +Resolved Conflicts: + docs/nvdimm.txt + +Signed-off-by: Danilo C. L. de Paula +--- + backends/hostmem-file.c | 43 +++++++++++++++++++++++++++++++++++++++++-- + docs/nvdimm.txt | 42 ++++++++++++++++++++++++++++++++++++++++++ + exec.c | 8 ++++++++ + include/exec/memory.h | 4 ++++ + include/exec/ram_addr.h | 3 +++ + qemu-options.hx | 7 +++++++ + 6 files changed, 105 insertions(+), 2 deletions(-) + +diff --git a/backends/hostmem-file.c b/backends/hostmem-file.c +index 34c68bb..2476dcb 100644 +--- a/backends/hostmem-file.c ++++ b/backends/hostmem-file.c +@@ -12,6 +12,7 @@ + #include "qemu/osdep.h" + #include "qapi/error.h" + #include "qemu-common.h" ++#include "qemu/error-report.h" + #include "sysemu/hostmem.h" + #include "sysemu/sysemu.h" + #include "qom/object_interfaces.h" +@@ -31,9 +32,10 @@ typedef struct HostMemoryBackendFile HostMemoryBackendFile; + struct HostMemoryBackendFile { + HostMemoryBackend parent_obj; + +- bool discard_data; + char *mem_path; + uint64_t align; ++ bool discard_data; ++ bool is_pmem; + }; + + static void +@@ -59,7 +61,8 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) + memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), + path, + backend->size, fb->align, +- backend->share ? RAM_SHARED : 0, ++ (backend->share ? RAM_SHARED : 0) | ++ (fb->is_pmem ? RAM_PMEM : 0), + fb->mem_path, errp); + g_free(path); + } +@@ -131,6 +134,39 @@ static void file_memory_backend_set_align(Object *o, Visitor *v, + error_propagate(errp, local_err); + } + ++static bool file_memory_backend_get_pmem(Object *o, Error **errp) ++{ ++ return MEMORY_BACKEND_FILE(o)->is_pmem; ++} ++ ++static void file_memory_backend_set_pmem(Object *o, bool value, Error **errp) ++{ ++ HostMemoryBackend *backend = MEMORY_BACKEND(o); ++ HostMemoryBackendFile *fb = MEMORY_BACKEND_FILE(o); ++ ++ if (host_memory_backend_mr_inited(backend)) { ++ error_setg(errp, "cannot change property 'pmem' of %s '%s'", ++ object_get_typename(o), ++ object_get_canonical_path_component(o)); ++ return; ++ } ++ ++#ifndef CONFIG_LIBPMEM ++ if (value) { ++ Error *local_err = NULL; ++ error_setg(&local_err, ++ "Lack of libpmem support while setting the 'pmem=on'" ++ " of %s '%s'. We can't ensure data persistence.", ++ object_get_typename(o), ++ object_get_canonical_path_component(o)); ++ error_propagate(errp, local_err); ++ return; ++ } ++#endif ++ ++ fb->is_pmem = value; ++} ++ + static void file_backend_unparent(Object *obj) + { + HostMemoryBackend *backend = MEMORY_BACKEND(obj); +@@ -162,6 +198,9 @@ file_backend_class_init(ObjectClass *oc, void *data) + file_memory_backend_get_align, + file_memory_backend_set_align, + NULL, NULL, &error_abort); ++ object_class_property_add_bool(oc, "pmem", ++ file_memory_backend_get_pmem, file_memory_backend_set_pmem, ++ &error_abort); + } + + static void file_backend_instance_finalize(Object *o) +diff --git a/docs/nvdimm.txt b/docs/nvdimm.txt +index e903d8b..5f158a6 100644 +--- a/docs/nvdimm.txt ++++ b/docs/nvdimm.txt +@@ -153,3 +153,45 @@ guest NVDIMM region mapping structure. This unarmed flag indicates + guest software that this vNVDIMM device contains a region that cannot + accept persistent writes. In result, for example, the guest Linux + NVDIMM driver, marks such vNVDIMM device as read-only. ++ ++NVDIMM Persistence ++------------------ ++ ++ACPI 6.2 Errata A added support for a new Platform Capabilities Structure ++which allows the platform to communicate what features it supports related to ++NVDIMM data persistence. Users can provide a persistence value to a guest via ++the optional "nvdimm-persistence" machine command line option: ++ ++ -machine pc,accel=kvm,nvdimm,nvdimm-persistence=cpu ++ ++There are currently two valid values for this option: ++ ++"mem-ctrl" - The platform supports flushing dirty data from the memory ++ controller to the NVDIMMs in the event of power loss. ++ ++"cpu" - The platform supports flushing dirty data from the CPU cache to ++ the NVDIMMs in the event of power loss. This implies that the ++ platform also supports flushing dirty data through the memory ++ controller on power loss. ++ ++If the vNVDIMM backend is in host persistent memory that can be accessed in ++SNIA NVM Programming Model [1] (e.g., Intel NVDIMM), it's suggested to set ++the 'pmem' option of memory-backend-file to 'on'. When 'pmem' is 'on' and QEMU ++is built with libpmem [2] support (configured with --enable-libpmem), QEMU ++will take necessary operations to guarantee the persistence of its own writes ++to the vNVDIMM backend(e.g., in vNVDIMM label emulation and live migration). ++If 'pmem' is 'on' while there is no libpmem support, qemu will exit and report ++a "lack of libpmem support" message to ensure the persistence is available. ++For example, if we want to ensure the persistence for some backend file, ++use the QEMU command line: ++ ++ -object memory-backend-file,id=nv_mem,mem-path=/XXX/yyy,size=4G,pmem=on ++ ++References ++---------- ++ ++[1] NVM Programming Model (NPM) ++ Version 1.2 ++ https://www.snia.org/sites/default/files/technical_work/final/NVMProgrammingModel_v1.2.pdf ++[2] Persistent Memory Development Kit (PMDK), formerly known as NVML project, home page: ++ http://pmem.io/pmdk/ +diff --git a/exec.c b/exec.c +index 8d58e8f..9028700 100644 +--- a/exec.c ++++ b/exec.c +@@ -2049,6 +2049,9 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, + Error *local_err = NULL; + int64_t file_size; + ++ /* Just support these ram flags by now. */ ++ assert((ram_flags & ~(RAM_SHARED | RAM_PMEM)) == 0); ++ + if (xen_enabled()) { + error_setg(errp, "-mem-path not supported with Xen"); + return NULL; +@@ -3867,6 +3870,11 @@ err: + return ret; + } + ++bool ramblock_is_pmem(RAMBlock *rb) ++{ ++ return rb->flags & RAM_PMEM; ++} ++ + #endif + + void page_size_init(void) +diff --git a/include/exec/memory.h b/include/exec/memory.h +index b3abe61..fd2c574 100644 +--- a/include/exec/memory.h ++++ b/include/exec/memory.h +@@ -122,6 +122,9 @@ typedef struct IOMMUNotifier IOMMUNotifier; + /* RAM can be migrated */ + #define RAM_MIGRATABLE (1 << 4) + ++/* RAM is a persistent kind memory */ ++#define RAM_PMEM (1 << 5) ++ + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, + IOMMUNotifierFlag flags, + hwaddr start, hwaddr end) +@@ -541,6 +544,7 @@ void memory_region_init_resizeable_ram(MemoryRegion *mr, + * (getpagesize()) will be used. + * @ram_flags: Memory region features: + * - RAM_SHARED: memory must be mmaped with the MAP_SHARED flag ++ * - RAM_PMEM: the memory is persistent memory + * Other bits are ignored now. + * @path: the path in which to allocate the RAM. + * @errp: pointer to Error*, to store an error if it happens. +diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h +index 67e163e..922305d 100644 +--- a/include/exec/ram_addr.h ++++ b/include/exec/ram_addr.h +@@ -70,6 +70,8 @@ static inline unsigned long int ramblock_recv_bitmap_offset(void *host_addr, + return host_addr_offset >> TARGET_PAGE_BITS; + } + ++bool ramblock_is_pmem(RAMBlock *rb); ++ + long qemu_getrampagesize(void); + unsigned long last_ram_page(void); + +@@ -84,6 +86,7 @@ unsigned long last_ram_page(void); + * @ram_flags: specify the properties of the ram block, which can be one + * or bit-or of following values + * - RAM_SHARED: mmap the backing file or device with MAP_SHARED ++ * - RAM_PMEM: the backend @mem_path or @fd is persistent memory + * Other bits are ignored. + * @mem_path or @fd: specify the backing file or device + * @errp: pointer to Error*, to store an error if it happens +diff --git a/qemu-options.hx b/qemu-options.hx +index 683ab0d..1b6786b 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -4051,6 +4051,13 @@ requires an alignment different than the default one used by QEMU, eg + the device DAX /dev/dax0.0 requires 2M alignment rather than 4K. In + such cases, users can specify the required alignment via this option. + ++The @option{pmem} option specifies whether the backing file specified ++by @option{mem-path} is in host persistent memory that can be accessed ++using the SNIA NVM programming model (e.g. Intel NVDIMM). ++If @option{pmem} is set to 'on', QEMU will take necessary operations to ++guarantee the persistence of its own writes to @option{mem-path} ++(e.g. in vNVDIMM label emulation and live migration). ++ + @item -object memory-backend-ram,id=@var{id},merge=@var{on|off},dump=@var{on|off},share=@var{on|off},prealloc=@var{on|off},size=@var{size},host-nodes=@var{host-nodes},policy=@var{default|preferred|bind|interleave} + + Creates a memory backend object, which can be used to back the guest RAM. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-char-serial-Only-retry-if-qemu_chr_fe_write-retur.patch b/SOURCES/kvm-hw-char-serial-Only-retry-if-qemu_chr_fe_write-retur.patch new file mode 100644 index 0000000..a55b5d8 --- /dev/null +++ b/SOURCES/kvm-hw-char-serial-Only-retry-if-qemu_chr_fe_write-retur.patch @@ -0,0 +1,59 @@ +From 59609c04618734b168df1b751e0126c03671b93c Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Fri, 20 Jul 2018 12:17:59 +0200 +Subject: [PATCH 254/268] hw/char/serial: Only retry if qemu_chr_fe_write + returns 0 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20180720121800.18952-2-marcandre.lureau@redhat.com> +Patchwork-id: 81454 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 1/2] hw/char/serial: Only retry if qemu_chr_fe_write returns 0 +Bugzilla: 1592817 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Markus Armbruster +RH-Acked-by: John Snow + +From: Sergio Lopez + +Only retry on serial_xmit if qemu_chr_fe_write returns 0, as this is the +only recoverable error. + +Retrying with any other scenario, in addition to being a waste of CPU +cycles, can compromise the Guest stability if by the vCPU issuing the +write and the main loop thread are, by chance or explicit pinning, +running on the same pCPU. + +Previous discussion: + +https://lists.nongnu.org/archive/html/qemu-devel/2018-05/msg06998.html + +Signed-off-by: Sergio Lopez +Message-Id: <1528185295-14199-1-git-send-email-slp@redhat.com> +Signed-off-by: Paolo Bonzini + +(cherry picked from commit 019288bf137183bf3407c9824655b753bfafc99f) +Signed-off-by: Marc-André Lureau +Signed-off-by: Miroslav Rezanina +--- + hw/char/serial.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/char/serial.c b/hw/char/serial.c +index d6d9b18..d4057bf 100644 +--- a/hw/char/serial.c ++++ b/hw/char/serial.c +@@ -262,7 +262,7 @@ static void serial_xmit(SerialState *s) + if (s->mcr & UART_MCR_LOOP) { + /* in loopback mode, say that we just received a char */ + serial_receive1(s, &s->tsr, 1); +- } else if (qemu_chr_fe_write(&s->chr, &s->tsr, 1) != 1 && ++ } else if (qemu_chr_fe_write(&s->chr, &s->tsr, 1) == 0 && + s->tsr_retry < MAX_XMIT_RETRY) { + assert(s->watch_tag == 0); + s->watch_tag = +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-char-serial-retry-write-if-EAGAIN.patch b/SOURCES/kvm-hw-char-serial-retry-write-if-EAGAIN.patch new file mode 100644 index 0000000..4c02b56 --- /dev/null +++ b/SOURCES/kvm-hw-char-serial-retry-write-if-EAGAIN.patch @@ -0,0 +1,72 @@ +From 46979f6e61d0afc31be31dab797f60c4f41c3de5 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Fri, 20 Jul 2018 12:18:00 +0200 +Subject: [PATCH 255/268] hw/char/serial: retry write if EAGAIN +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20180720121800.18952-3-marcandre.lureau@redhat.com> +Patchwork-id: 81455 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 2/2] hw/char/serial: retry write if EAGAIN +Bugzilla: 1592817 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Markus Armbruster +RH-Acked-by: John Snow + +If the chardev returns -1 with EAGAIN errno on write(), it should try +to send it again (EINTR is handled by the chardev itself). + +This fixes commit 019288bf137183bf3407c9824655b753bfafc99f +"hw/char/serial: Only retry if qemu_chr_fe_write returns 0" + +Tested-by: Igor Mammedov +Signed-off-by: Marc-André Lureau +Message-Id: <20180716110755.12499-1-marcandre.lureau@redhat.com> +Signed-off-by: Paolo Bonzini + +(cherry picked from commit f3575af130c700cea060b51a89008a76dae22259) +Signed-off-by: Marc-André Lureau +Signed-off-by: Miroslav Rezanina +--- + hw/char/serial.c | 23 ++++++++++++++--------- + 1 file changed, 14 insertions(+), 9 deletions(-) + +diff --git a/hw/char/serial.c b/hw/char/serial.c +index d4057bf..478f9a8 100644 +--- a/hw/char/serial.c ++++ b/hw/char/serial.c +@@ -262,15 +262,20 @@ static void serial_xmit(SerialState *s) + if (s->mcr & UART_MCR_LOOP) { + /* in loopback mode, say that we just received a char */ + serial_receive1(s, &s->tsr, 1); +- } else if (qemu_chr_fe_write(&s->chr, &s->tsr, 1) == 0 && +- s->tsr_retry < MAX_XMIT_RETRY) { +- assert(s->watch_tag == 0); +- s->watch_tag = +- qemu_chr_fe_add_watch(&s->chr, G_IO_OUT | G_IO_HUP, +- serial_watch_cb, s); +- if (s->watch_tag > 0) { +- s->tsr_retry++; +- return; ++ } else { ++ int rc = qemu_chr_fe_write(&s->chr, &s->tsr, 1); ++ ++ if ((rc == 0 || ++ (rc == -1 && errno == EAGAIN)) && ++ s->tsr_retry < MAX_XMIT_RETRY) { ++ assert(s->watch_tag == 0); ++ s->watch_tag = ++ qemu_chr_fe_add_watch(&s->chr, G_IO_OUT | G_IO_HUP, ++ serial_watch_cb, s); ++ if (s->watch_tag > 0) { ++ s->tsr_retry++; ++ return; ++ } + } + } + s->tsr_retry = 0; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-s390x-Fix-bad-mask-in-time2tod.patch b/SOURCES/kvm-hw-s390x-Fix-bad-mask-in-time2tod.patch new file mode 100644 index 0000000..2e6c3b3 --- /dev/null +++ b/SOURCES/kvm-hw-s390x-Fix-bad-mask-in-time2tod.patch @@ -0,0 +1,50 @@ +From e851c8ce61bd343961c59afee9062d04a880b1d6 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:14 +0000 +Subject: [PATCH 12/22] hw/s390x: Fix bad mask in time2tod() + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-13-david@redhat.com> +Patchwork-id: 83756 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 12/12] hw/s390x: Fix bad mask in time2tod() +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +Since "s390x/tcg: avoid overflows in time2tod/tod2time", the +time2tod() function tries to deal with the 9 uppermost bits in the +time value, but uses the wrong mask for this: 0xff80000000000000 should +be used instead of 0xff10000000000000 here. + +Fixes: 14055ce53c2d901d826ffad7fb7d6bb8ab46bdfd +Cc: qemu-stable@nongnu.org +Signed-off-by: Thomas Huth +Message-Id: <1544792887-14575-1-git-send-email-thuth@redhat.com> +Reviewed-by: David Hildenbrand +[CH: tweaked commit message] +Signed-off-by: Cornelia Huck +(cherry picked from commit aba7a5a2de3dba5917024df25441f715b9249e31) +Signed-off-by: David Hildenbrand + +Signed-off-by: Danilo C. L. de Paula +--- + include/hw/s390x/tod.h | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/include/hw/s390x/tod.h b/include/hw/s390x/tod.h +index cbd7552..47ef9de 100644 +--- a/include/hw/s390x/tod.h ++++ b/include/hw/s390x/tod.h +@@ -56,7 +56,7 @@ typedef struct S390TODClass { + /* Converts ns to s390's clock format */ + static inline uint64_t time2tod(uint64_t ns) + { +- return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9); ++ return (ns << 9) / 125 + (((ns & 0xff80000000000000ull) / 125) << 9); + } + + /* Converts s390's clock format to ns */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-s390x-Include-the-tod-qemu-also-for-builds-with-d.patch b/SOURCES/kvm-hw-s390x-Include-the-tod-qemu-also-for-builds-with-d.patch new file mode 100644 index 0000000..80fb261 --- /dev/null +++ b/SOURCES/kvm-hw-s390x-Include-the-tod-qemu-also-for-builds-with-d.patch @@ -0,0 +1,61 @@ +From 5481b8bcb37cfaf3016a09df49dbcfdeb5ae9d68 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:12 +0000 +Subject: [PATCH 10/22] hw/s390x: Include the tod-qemu also for builds with + --disable-tcg + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-11-david@redhat.com> +Patchwork-id: 83753 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 10/12] hw/s390x: Include the tod-qemu also for builds with --disable-tcg +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +The device is required for running qtests, see hw/s390x/tod.c: + +void s390_init_tod(void) +{ + Object *obj; + + if (kvm_enabled()) { + obj = object_new(TYPE_KVM_S390_TOD); + } else { + obj = object_new(TYPE_QEMU_S390_TOD); + } + [...] + } + +During qtests, we're running without kvm, so TYPE_QEMU_S390_TOD is +required to avoid that QEMU aborts here. + +Fixes: 8046f374a6 ("s390x/tod: factor out TOD into separate device") +Signed-off-by: Thomas Huth +Message-Id: <1539264723-741-1-git-send-email-thuth@redhat.com> +Reviewed-by: David Hildenbrand +Signed-off-by: Cornelia Huck +(cherry picked from commit 0161215d435ef5680c4623bcbdfe89ce5b35cf42) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + hw/s390x/Makefile.objs | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs +index add89b1..c57d706 100644 +--- a/hw/s390x/Makefile.objs ++++ b/hw/s390x/Makefile.objs +@@ -15,8 +15,8 @@ obj-$(call lnot,$(CONFIG_PCI)) += s390-pci-stub.o + obj-y += s390-skeys.o + obj-y += s390-stattrib.o + obj-y += tod.o ++obj-y += tod-qemu.o + obj-$(CONFIG_KVM) += tod-kvm.o +-obj-$(CONFIG_TCG) += tod-qemu.o + obj-$(CONFIG_KVM) += s390-skeys-kvm.o + obj-$(CONFIG_KVM) += s390-stattrib-kvm.o + obj-y += s390-ccw.o +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-scsi-add-VPD-Block-Limits-emulation.patch b/SOURCES/kvm-hw-scsi-add-VPD-Block-Limits-emulation.patch new file mode 100644 index 0000000..38e651d --- /dev/null +++ b/SOURCES/kvm-hw-scsi-add-VPD-Block-Limits-emulation.patch @@ -0,0 +1,335 @@ +From 51c8629ca55d0b6e51faffcddfc5e3b10898fc5c Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:30:58 +0000 +Subject: [PATCH 3/8] hw/scsi: add VPD Block Limits emulation + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-4-pbonzini@redhat.com> +Patchwork-id: 83713 +O-Subject: [PATCH 3/8] hw/scsi: add VPD Block Limits emulation +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +From: Daniel Henrique Barboza + +The VPD Block Limits Inquiry page is optional, allowing SCSI devices +to not implement it. This is the case for devices like the MegaRAID +SAS 9361-8i and Microsemi PM8069. + +In case of SCSI passthrough, the response of this request is used by +the QEMU SCSI layer to set the max_io_sectors that the guest +device will support, based on the value of the max_sectors_kb that +the device has set in the host at that time. Without this response, +the guest kernel is free to assume any value of max_io_sectors +for the SCSI device. If this value is greater than the value from +the host, SCSI Sense errors will occur because the guest will send +read/write requests that are larger than the underlying host device +is configured to support. An example of this behavior can be seen +in [1]. + +A workaround is to set the max_sectors_kb host value back in the guest +kernel (a process that can be automated using rc.local startup scripts +and the like), but this has several drawbacks: + +- it can be troublesome if the guest has many passthrough devices that +needs this tuning; + +- if a change in max_sectors_kb is made in the host side, manual change +in the guests will also be required; + +- during an OS install it is difficult, and sometimes not possible, to +go to a terminal and change the max_sectors_kb prior to the installation. +This means that the disk can't be used during the install process. The +easiest alternative here is to roll back to scsi-hd, install the guest +and then go back to SCSI passthrough when the installation is done and +max_sectors_kb can be set. + +An easier way would be to QEMU handle the absence of the Block Limits +VPD device response, setting max_io_sectors accordingly and allowing +the guest to use the device without the hassle. + +This patch adds emulation of the Block Limits VPD response for +SCSI passthrough devices of type TYPE_DISK that doesn't support +it. The following changes were made: + +- scsi_handle_inquiry_reply will now check the available VPD +pages from the Inquiry EVPD reply. In case the device does not + +- a new function called scsi_generic_set_vpd_bl_emulation, +that is called during device realize, was created to set a +new flag 'needs_vpd_bl_emulation' of the device. This function +retrieves the Inquiry EVPD response of the device to check for +VPD BL support. + +- scsi_handle_inquiry_reply will now check the available VPD +pages from the Inquiry EVPD reply in case the device needs +VPD BL emulation, adding the Block Limits page (0xb0) to +the list. This will make the guest kernel aware of the +support that we're now providing by emulation. + +- a new function scsi_emulate_block_limits creates the +emulated Block Limits response. This function is called +inside scsi_read_complete in case the device requires +Block Limits VPD emulation and we detected a SCSI Sense +error in the VPD Block Limits reply that was issued +from the guest kernel to the device. This error is +expected: we're reporting support from our side, but +the device isn't aware of it. + +With this patch, the guest now queries the Block Limits +page during the device configuration because it is being +advertised in the Supported Pages response. It will either +receive the Block Limits page from the hardware, if it supports +it, or will receive an emulated response from QEMU. At any rate, +the guest now has the information to set the max_sectors_kb +parameter accordingly, sparing the user of SCSI sense errors +that would happen without the emulated response and in the +absence of Block Limits support from the hardware. + +[1] https://bugzilla.redhat.com/show_bug.cgi?id=1566195 + +Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1566195 +Reported-by: Dac Nguyen +Signed-off-by: Daniel Henrique Barboza +Message-Id: <20180627172432.11120-4-danielhb413@gmail.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit a71c775b24ebc664129eb1d9b4c360590353efd5) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-disk.c | 2 +- + hw/scsi/scsi-generic.c | 132 +++++++++++++++++++++++++++++++++++++++++++++---- + include/hw/scsi/scsi.h | 3 +- + 3 files changed, 125 insertions(+), 12 deletions(-) + +diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c +index ea86849..b3d53ec 100644 +--- a/hw/scsi/scsi-disk.c ++++ b/hw/scsi/scsi-disk.c +@@ -2646,7 +2646,7 @@ static void scsi_block_realize(SCSIDevice *dev, Error **errp) + s->features |= (1 << SCSI_DISK_F_NO_REMOVABLE_DEVOPS); + + scsi_realize(&s->qdev, errp); +- scsi_generic_read_device_identification(&s->qdev); ++ scsi_generic_read_device_inquiry(&s->qdev); + } + + typedef struct SCSIBlockReq { +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index c6307a8..4266003 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -145,6 +145,8 @@ static int execute_command(BlockBackend *blk, + + static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + { ++ uint8_t page, page_len; ++ + /* + * EVPD set to zero returns the standard INQUIRY data. + * +@@ -168,22 +170,57 @@ static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + s->scsi_version = r->buf[2]; + } + } +- if (s->type == TYPE_DISK && r->req.cmd.buf[2] == 0xb0) { +- uint32_t max_transfer = +- blk_get_max_transfer(s->conf.blk) / s->blocksize; + +- assert(max_transfer); +- stl_be_p(&r->buf[8], max_transfer); +- /* Also take care of the opt xfer len. */ +- stl_be_p(&r->buf[12], +- MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); ++ if (s->type == TYPE_DISK && (r->req.cmd.buf[1] & 0x01)) { ++ page = r->req.cmd.buf[2]; ++ if (page == 0xb0) { ++ uint32_t max_transfer = ++ blk_get_max_transfer(s->conf.blk) / s->blocksize; ++ ++ assert(max_transfer); ++ stl_be_p(&r->buf[8], max_transfer); ++ /* Also take care of the opt xfer len. */ ++ stl_be_p(&r->buf[12], ++ MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); ++ } else if (page == 0x00 && s->needs_vpd_bl_emulation) { ++ /* ++ * Now we're capable of supplying the VPD Block Limits ++ * response if the hardware can't. Add it in the INQUIRY ++ * Supported VPD pages response in case we are using the ++ * emulation for this device. ++ * ++ * This way, the guest kernel will be aware of the support ++ * and will use it to proper setup the SCSI device. ++ */ ++ page_len = r->buf[3]; ++ r->buf[page_len + 4] = 0xb0; ++ r->buf[3] = ++page_len; ++ } + } + } + ++static int scsi_emulate_block_limits(SCSIGenericReq *r) ++{ ++ r->buflen = scsi_disk_emulate_vpd_page(&r->req, r->buf); ++ r->io_header.sb_len_wr = 0; ++ ++ /* ++ * We have valid contents in the reply buffer but the ++ * io_header can report a sense error coming from ++ * the hardware in scsi_command_complete_noio. Clean ++ * up the io_header to avoid reporting it. ++ */ ++ r->io_header.driver_status = 0; ++ r->io_header.status = 0; ++ ++ return r->buflen; ++} ++ + static void scsi_read_complete(void * opaque, int ret) + { + SCSIGenericReq *r = (SCSIGenericReq *)opaque; + SCSIDevice *s = r->req.dev; ++ SCSISense sense; + int len; + + assert(r->req.aiocb != NULL); +@@ -200,6 +237,27 @@ static void scsi_read_complete(void * opaque, int ret) + DPRINTF("Data ready tag=0x%x len=%d\n", r->req.tag, len); + + r->len = -1; ++ ++ /* ++ * Check if this is a VPD Block Limits request that ++ * resulted in sense error but would need emulation. ++ * In this case, emulate a valid VPD response. ++ */ ++ if (s->needs_vpd_bl_emulation) { ++ int is_vpd_bl = r->req.cmd.buf[0] == INQUIRY && ++ r->req.cmd.buf[1] & 0x01 && ++ r->req.cmd.buf[2] == 0xb0; ++ ++ if (is_vpd_bl && sg_io_sense_from_errno(-ret, &r->io_header, &sense)) { ++ len = scsi_emulate_block_limits(r); ++ /* ++ * No need to let scsi_read_complete go on and handle an ++ * INQUIRY VPD BL request we created manually. ++ */ ++ goto req_complete; ++ } ++ } ++ + if (len == 0) { + scsi_command_complete_noio(r, 0); + goto done; +@@ -234,6 +292,8 @@ static void scsi_read_complete(void * opaque, int ret) + if (r->req.cmd.buf[0] == INQUIRY) { + scsi_handle_inquiry_reply(r, s); + } ++ ++req_complete: + scsi_req_data(&r->req, len); + scsi_req_unref(&r->req); + +@@ -435,7 +495,49 @@ int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, + return 0; + } + +-void scsi_generic_read_device_identification(SCSIDevice *s) ++/* ++ * Executes an INQUIRY request with EVPD set to retrieve the ++ * available VPD pages of the device. If the device does ++ * not support the Block Limits page (page 0xb0), set ++ * the needs_vpd_bl_emulation flag for future use. ++ */ ++static void scsi_generic_set_vpd_bl_emulation(SCSIDevice *s) ++{ ++ uint8_t cmd[6]; ++ uint8_t buf[250]; ++ uint8_t page_len; ++ int ret, i; ++ ++ memset(cmd, 0, sizeof(cmd)); ++ memset(buf, 0, sizeof(buf)); ++ cmd[0] = INQUIRY; ++ cmd[1] = 1; ++ cmd[2] = 0x00; ++ cmd[4] = sizeof(buf); ++ ++ ret = scsi_SG_IO_FROM_DEV(s->conf.blk, cmd, sizeof(cmd), ++ buf, sizeof(buf)); ++ if (ret < 0) { ++ /* ++ * Do not assume anything if we can't retrieve the ++ * INQUIRY response to assert the VPD Block Limits ++ * support. ++ */ ++ s->needs_vpd_bl_emulation = false; ++ return; ++ } ++ ++ page_len = buf[3]; ++ for (i = 4; i < page_len + 4; i++) { ++ if (buf[i] == 0xb0) { ++ s->needs_vpd_bl_emulation = false; ++ return; ++ } ++ } ++ s->needs_vpd_bl_emulation = true; ++} ++ ++static void scsi_generic_read_device_identification(SCSIDevice *s) + { + uint8_t cmd[6]; + uint8_t buf[250]; +@@ -480,6 +582,16 @@ void scsi_generic_read_device_identification(SCSIDevice *s) + } + } + ++void scsi_generic_read_device_inquiry(SCSIDevice *s) ++{ ++ scsi_generic_read_device_identification(s); ++ if (s->type == TYPE_DISK) { ++ scsi_generic_set_vpd_bl_emulation(s); ++ } else { ++ s->needs_vpd_bl_emulation = false; ++ } ++} ++ + static int get_stream_blocksize(BlockBackend *blk) + { + uint8_t cmd[6]; +@@ -581,7 +693,7 @@ static void scsi_generic_realize(SCSIDevice *s, Error **errp) + + /* Only used by scsi-block, but initialize it nevertheless to be clean. */ + s->default_scsi_version = -1; +- scsi_generic_read_device_identification(s); ++ scsi_generic_read_device_inquiry(s); + } + + const SCSIReqOps scsi_generic_req_ops = { +diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h +index b6e05c4..ee3a411 100644 +--- a/include/hw/scsi/scsi.h ++++ b/include/hw/scsi/scsi.h +@@ -87,6 +87,7 @@ struct SCSIDevice + uint64_t port_wwn; + int scsi_version; + int default_scsi_version; ++ bool needs_vpd_bl_emulation; + }; + + extern const VMStateDescription vmstate_scsi_device; +@@ -186,7 +187,7 @@ void scsi_device_purge_requests(SCSIDevice *sdev, SCSISense sense); + void scsi_device_set_ua(SCSIDevice *sdev, SCSISense sense); + void scsi_device_report_change(SCSIDevice *dev, SCSISense sense); + void scsi_device_unit_attention_reported(SCSIDevice *dev); +-void scsi_generic_read_device_identification(SCSIDevice *dev); ++void scsi_generic_read_device_inquiry(SCSIDevice *dev); + int scsi_device_get_sense(SCSIDevice *dev, uint8_t *buf, int len, bool fixed); + int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf); + int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-scsi-centralize-SG_IO-calls-into-single-function.patch b/SOURCES/kvm-hw-scsi-centralize-SG_IO-calls-into-single-function.patch new file mode 100644 index 0000000..1a9385f --- /dev/null +++ b/SOURCES/kvm-hw-scsi-centralize-SG_IO-calls-into-single-function.patch @@ -0,0 +1,205 @@ +From 84621d8aa40cb29508b5730c46666c645851bbfd Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:30:57 +0000 +Subject: [PATCH 2/8] hw/scsi: centralize SG_IO calls into single function +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-3-pbonzini@redhat.com> +Patchwork-id: 83711 +O-Subject: [PATCH 2/8] hw/scsi: centralize SG_IO calls into single function +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +From: Daniel Henrique Barboza + +For the VPD Block Limits emulation with SCSI passthrough, +we'll issue an Inquiry request with EVPD set to retrieve +the available VPD pages of the device. This would be done in +a way similar of what scsi_generic_read_device_identification +does: create a SCSI command and a reply buffer, fill in the +sg_io_hdr_t structure, call blk_ioctl, check if an error +occurred, process the response. + +This same process is done in other 2 functions, get_device_type +and get_stream_blocksize. They differ in the command/reply +buffer and post-processing, everything else is almost a +copy/paste. + +Instead of adding a forth copy/pasted-ish code when adding +the passthrough VPD BL emulation, this patch extirpates +this repetition of those 3 functions and put it into +a new one called scsi_SG_IO_FROM_DEV. Any future code that +wants to execute an SG_DXFER_FROM_DEV to the device can +use it, avoiding filling sg_io_hdr_t again and et cetera. + +Signed-off-by: Daniel Henrique Barboza +Message-Id: <20180627172432.11120-3-danielhb413@gmail.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit a0c7e35b17b3d2cade8a5fc8e57904e02fb91fe4) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-disk.c | 18 +++------------ + hw/scsi/scsi-generic.c | 61 +++++++++++++++++++++++++------------------------- + include/hw/scsi/scsi.h | 2 ++ + 3 files changed, 36 insertions(+), 45 deletions(-) + +diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c +index ae5b4c0..ea86849 100644 +--- a/hw/scsi/scsi-disk.c ++++ b/hw/scsi/scsi-disk.c +@@ -2579,8 +2579,6 @@ static int get_device_type(SCSIDiskState *s) + { + uint8_t cmd[16]; + uint8_t buf[36]; +- uint8_t sensebuf[8]; +- sg_io_hdr_t io_header; + int ret; + + memset(cmd, 0, sizeof(cmd)); +@@ -2588,19 +2586,9 @@ static int get_device_type(SCSIDiskState *s) + cmd[0] = INQUIRY; + cmd[4] = sizeof(buf); + +- memset(&io_header, 0, sizeof(io_header)); +- io_header.interface_id = 'S'; +- io_header.dxfer_direction = SG_DXFER_FROM_DEV; +- io_header.dxfer_len = sizeof(buf); +- io_header.dxferp = buf; +- io_header.cmdp = cmd; +- io_header.cmd_len = sizeof(cmd); +- io_header.mx_sb_len = sizeof(sensebuf); +- io_header.sbp = sensebuf; +- io_header.timeout = 6000; /* XXX */ +- +- ret = blk_ioctl(s->qdev.conf.blk, SG_IO, &io_header); +- if (ret < 0 || io_header.driver_status || io_header.host_status) { ++ ret = scsi_SG_IO_FROM_DEV(s->qdev.conf.blk, cmd, sizeof(cmd), ++ buf, sizeof(buf)); ++ if (ret < 0) { + return -1; + } + s->qdev.type = buf[0]; +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index 796162c..c6307a8 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -410,35 +410,48 @@ static int read_naa_id(const uint8_t *p, uint64_t *p_wwn) + return -EINVAL; + } + +-void scsi_generic_read_device_identification(SCSIDevice *s) ++int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, ++ uint8_t *buf, uint8_t buf_size) + { +- uint8_t cmd[6]; +- uint8_t buf[250]; +- uint8_t sensebuf[8]; + sg_io_hdr_t io_header; ++ uint8_t sensebuf[8]; + int ret; +- int i, len; +- +- memset(cmd, 0, sizeof(cmd)); +- memset(buf, 0, sizeof(buf)); +- cmd[0] = INQUIRY; +- cmd[1] = 1; +- cmd[2] = 0x83; +- cmd[4] = sizeof(buf); + + memset(&io_header, 0, sizeof(io_header)); + io_header.interface_id = 'S'; + io_header.dxfer_direction = SG_DXFER_FROM_DEV; +- io_header.dxfer_len = sizeof(buf); ++ io_header.dxfer_len = buf_size; + io_header.dxferp = buf; + io_header.cmdp = cmd; +- io_header.cmd_len = sizeof(cmd); ++ io_header.cmd_len = cmd_size; + io_header.mx_sb_len = sizeof(sensebuf); + io_header.sbp = sensebuf; + io_header.timeout = 6000; /* XXX */ + +- ret = blk_ioctl(s->conf.blk, SG_IO, &io_header); ++ ret = blk_ioctl(blk, SG_IO, &io_header); + if (ret < 0 || io_header.driver_status || io_header.host_status) { ++ return -1; ++ } ++ return 0; ++} ++ ++void scsi_generic_read_device_identification(SCSIDevice *s) ++{ ++ uint8_t cmd[6]; ++ uint8_t buf[250]; ++ int ret; ++ int i, len; ++ ++ memset(cmd, 0, sizeof(cmd)); ++ memset(buf, 0, sizeof(buf)); ++ cmd[0] = INQUIRY; ++ cmd[1] = 1; ++ cmd[2] = 0x83; ++ cmd[4] = sizeof(buf); ++ ++ ret = scsi_SG_IO_FROM_DEV(s->conf.blk, cmd, sizeof(cmd), ++ buf, sizeof(buf)); ++ if (ret < 0) { + return; + } + +@@ -471,8 +484,6 @@ static int get_stream_blocksize(BlockBackend *blk) + { + uint8_t cmd[6]; + uint8_t buf[12]; +- uint8_t sensebuf[8]; +- sg_io_hdr_t io_header; + int ret; + + memset(cmd, 0, sizeof(cmd)); +@@ -480,21 +491,11 @@ static int get_stream_blocksize(BlockBackend *blk) + cmd[0] = MODE_SENSE; + cmd[4] = sizeof(buf); + +- memset(&io_header, 0, sizeof(io_header)); +- io_header.interface_id = 'S'; +- io_header.dxfer_direction = SG_DXFER_FROM_DEV; +- io_header.dxfer_len = sizeof(buf); +- io_header.dxferp = buf; +- io_header.cmdp = cmd; +- io_header.cmd_len = sizeof(cmd); +- io_header.mx_sb_len = sizeof(sensebuf); +- io_header.sbp = sensebuf; +- io_header.timeout = 6000; /* XXX */ +- +- ret = blk_ioctl(blk, SG_IO, &io_header); +- if (ret < 0 || io_header.driver_status || io_header.host_status) { ++ ret = scsi_SG_IO_FROM_DEV(blk, cmd, sizeof(cmd), buf, sizeof(buf)); ++ if (ret < 0) { + return -1; + } ++ + return (buf[9] << 16) | (buf[10] << 8) | buf[11]; + } + +diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h +index 5930a43..b6e05c4 100644 +--- a/include/hw/scsi/scsi.h ++++ b/include/hw/scsi/scsi.h +@@ -189,6 +189,8 @@ void scsi_device_unit_attention_reported(SCSIDevice *dev); + void scsi_generic_read_device_identification(SCSIDevice *dev); + int scsi_device_get_sense(SCSIDevice *dev, uint8_t *buf, int len, bool fixed); + int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf); ++int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, ++ uint8_t *buf, uint8_t buf_size); + SCSIDevice *scsi_device_find(SCSIBus *bus, int channel, int target, int lun); + + /* scsi-generic.c. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-hw-scsi-cleanups-before-VPD-BL-emulation.patch b/SOURCES/kvm-hw-scsi-cleanups-before-VPD-BL-emulation.patch new file mode 100644 index 0000000..fb53c64 --- /dev/null +++ b/SOURCES/kvm-hw-scsi-cleanups-before-VPD-BL-emulation.patch @@ -0,0 +1,594 @@ +From a9bfa7b102f1de3dbff308806099a7529b4e96b4 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:30:56 +0000 +Subject: [PATCH 1/8] hw/scsi: cleanups before VPD BL emulation + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-2-pbonzini@redhat.com> +Patchwork-id: 83712 +O-Subject: [PATCH 1/8] hw/scsi: cleanups before VPD BL emulation +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +From: Daniel Henrique Barboza + +To add support for the emulation of Block Limits VPD page +for passthrough devices, a few adjustments in the current code +base is required to avoid repetition and improve clarity. + +In scsi-generic.c, detach the Inquiry handling from +scsi_read_complete and put it into a new function called +scsi_handle_inquiry_reply. This change aims to avoid +cluttering of scsi_read_complete when we more logic in the +Inquiry response handling is added in the next patches, +centralizing the changes in the new function. + +In scsi-disk.c, take the build of all emulated VPD pages +from scsi_disk_emulate_inquiry and make it available to +other files into a non-static function called +scsi_disk_emulate_vpd_page. Making it public will allow +the future VPD BL emulation code for passthrough devices +to use it from scsi-generic.c, avoiding copy/pasting this +code solely for that purpose. It also has the advantage of +providing emulation of all VPD pages in case we need to +emulate other pages in other scenarios. As a bonus, +scsi_disk_emulate_inquiry got tidier. + +Signed-off-by: Daniel Henrique Barboza +Message-Id: <20180627172432.11120-2-danielhb413@gmail.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit 0a96ca2437646bad197b0108c5f4a93e7ead05a9) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-disk.c | 407 +++++++++++++++++++++++++------------------------ + hw/scsi/scsi-generic.c | 71 +++++---- + include/hw/scsi/scsi.h | 1 + + 3 files changed, 249 insertions(+), 230 deletions(-) + +diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c +index ded23d3..ae5b4c0 100644 +--- a/hw/scsi/scsi-disk.c ++++ b/hw/scsi/scsi-disk.c +@@ -585,219 +585,228 @@ static uint8_t *scsi_get_buf(SCSIRequest *req) + return (uint8_t *)r->iov.iov_base; + } + +-static int scsi_disk_emulate_inquiry(SCSIRequest *req, uint8_t *outbuf) ++int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf) + { + SCSIDiskState *s = DO_UPCAST(SCSIDiskState, qdev, req->dev); +- int buflen = 0; +- int start; +- +- if (req->cmd.buf[1] & 0x1) { +- /* Vital product data */ +- uint8_t page_code = req->cmd.buf[2]; +- +- outbuf[buflen++] = s->qdev.type & 0x1f; +- outbuf[buflen++] = page_code ; // this page +- outbuf[buflen++] = 0x00; +- outbuf[buflen++] = 0x00; +- start = buflen; +- +- switch (page_code) { +- case 0x00: /* Supported page codes, mandatory */ +- { +- DPRINTF("Inquiry EVPD[Supported pages] " +- "buffer size %zd\n", req->cmd.xfer); +- outbuf[buflen++] = 0x00; // list of supported pages (this page) +- if (s->serial) { +- outbuf[buflen++] = 0x80; // unit serial number +- } +- outbuf[buflen++] = 0x83; // device identification +- if (s->qdev.type == TYPE_DISK) { +- outbuf[buflen++] = 0xb0; // block limits +- outbuf[buflen++] = 0xb1; /* block device characteristics */ +- outbuf[buflen++] = 0xb2; // thin provisioning +- } +- break; +- } +- case 0x80: /* Device serial number, optional */ +- { +- int l; ++ uint8_t page_code = req->cmd.buf[2]; ++ int start, buflen = 0; + +- if (!s->serial) { +- DPRINTF("Inquiry (EVPD[Serial number] not supported\n"); +- return -1; +- } ++ outbuf[buflen++] = s->qdev.type & 0x1f; ++ outbuf[buflen++] = page_code; ++ outbuf[buflen++] = 0x00; ++ outbuf[buflen++] = 0x00; ++ start = buflen; + +- l = strlen(s->serial); +- if (l > 36) { +- l = 36; +- } ++ switch (page_code) { ++ case 0x00: /* Supported page codes, mandatory */ ++ { ++ DPRINTF("Inquiry EVPD[Supported pages] " ++ "buffer size %zd\n", req->cmd.xfer); ++ outbuf[buflen++] = 0x00; /* list of supported pages (this page) */ ++ if (s->serial) { ++ outbuf[buflen++] = 0x80; /* unit serial number */ ++ } ++ outbuf[buflen++] = 0x83; /* device identification */ ++ if (s->qdev.type == TYPE_DISK) { ++ outbuf[buflen++] = 0xb0; /* block limits */ ++ outbuf[buflen++] = 0xb1; /* block device characteristics */ ++ outbuf[buflen++] = 0xb2; /* thin provisioning */ ++ } ++ break; ++ } ++ case 0x80: /* Device serial number, optional */ ++ { ++ int l; + +- DPRINTF("Inquiry EVPD[Serial number] " +- "buffer size %zd\n", req->cmd.xfer); +- memcpy(outbuf+buflen, s->serial, l); +- buflen += l; +- break; ++ if (!s->serial) { ++ DPRINTF("Inquiry (EVPD[Serial number] not supported\n"); ++ return -1; + } + +- case 0x83: /* Device identification page, mandatory */ +- { +- const char *str = s->serial ?: blk_name(s->qdev.conf.blk); +- int max_len = s->serial ? 20 : 255 - 8; +- int id_len = strlen(str); ++ l = strlen(s->serial); ++ if (l > 36) { ++ l = 36; ++ } + +- if (id_len > max_len) { +- id_len = max_len; +- } +- DPRINTF("Inquiry EVPD[Device identification] " +- "buffer size %zd\n", req->cmd.xfer); +- +- outbuf[buflen++] = 0x2; // ASCII +- outbuf[buflen++] = 0; // not officially assigned +- outbuf[buflen++] = 0; // reserved +- outbuf[buflen++] = id_len; // length of data following +- memcpy(outbuf+buflen, str, id_len); +- buflen += id_len; +- +- if (s->qdev.wwn) { +- outbuf[buflen++] = 0x1; // Binary +- outbuf[buflen++] = 0x3; // NAA +- outbuf[buflen++] = 0; // reserved +- outbuf[buflen++] = 8; +- stq_be_p(&outbuf[buflen], s->qdev.wwn); +- buflen += 8; +- } ++ DPRINTF("Inquiry EVPD[Serial number] " ++ "buffer size %zd\n", req->cmd.xfer); ++ memcpy(outbuf + buflen, s->serial, l); ++ buflen += l; ++ break; ++ } + +- if (s->qdev.port_wwn) { +- outbuf[buflen++] = 0x61; // SAS / Binary +- outbuf[buflen++] = 0x93; // PIV / Target port / NAA +- outbuf[buflen++] = 0; // reserved +- outbuf[buflen++] = 8; +- stq_be_p(&outbuf[buflen], s->qdev.port_wwn); +- buflen += 8; +- } ++ case 0x83: /* Device identification page, mandatory */ ++ { ++ const char *str = s->serial ?: blk_name(s->qdev.conf.blk); ++ int max_len = s->serial ? 20 : 255 - 8; ++ int id_len = strlen(str); + +- if (s->port_index) { +- outbuf[buflen++] = 0x61; // SAS / Binary +- outbuf[buflen++] = 0x94; // PIV / Target port / relative target port +- outbuf[buflen++] = 0; // reserved +- outbuf[buflen++] = 4; +- stw_be_p(&outbuf[buflen + 2], s->port_index); +- buflen += 4; +- } +- break; ++ if (id_len > max_len) { ++ id_len = max_len; + } +- case 0xb0: /* block limits */ +- { +- unsigned int unmap_sectors = +- s->qdev.conf.discard_granularity / s->qdev.blocksize; +- unsigned int min_io_size = +- s->qdev.conf.min_io_size / s->qdev.blocksize; +- unsigned int opt_io_size = +- s->qdev.conf.opt_io_size / s->qdev.blocksize; +- unsigned int max_unmap_sectors = +- s->max_unmap_size / s->qdev.blocksize; +- unsigned int max_io_sectors = +- s->max_io_size / s->qdev.blocksize; +- +- if (s->qdev.type == TYPE_ROM) { +- DPRINTF("Inquiry (EVPD[%02X] not supported for CDROM\n", +- page_code); +- return -1; +- } +- if (s->qdev.type == TYPE_DISK) { +- int max_transfer_blk = blk_get_max_transfer(s->qdev.conf.blk); +- int max_io_sectors_blk = +- max_transfer_blk / s->qdev.blocksize; +- +- max_io_sectors = +- MIN_NON_ZERO(max_io_sectors_blk, max_io_sectors); +- +- /* min_io_size and opt_io_size can't be greater than +- * max_io_sectors */ +- if (min_io_size) { +- min_io_size = MIN(min_io_size, max_io_sectors); +- } +- if (opt_io_size) { +- opt_io_size = MIN(opt_io_size, max_io_sectors); +- } +- } +- /* required VPD size with unmap support */ +- buflen = 0x40; +- memset(outbuf + 4, 0, buflen - 4); +- +- outbuf[4] = 0x1; /* wsnz */ +- +- /* optimal transfer length granularity */ +- outbuf[6] = (min_io_size >> 8) & 0xff; +- outbuf[7] = min_io_size & 0xff; +- +- /* maximum transfer length */ +- outbuf[8] = (max_io_sectors >> 24) & 0xff; +- outbuf[9] = (max_io_sectors >> 16) & 0xff; +- outbuf[10] = (max_io_sectors >> 8) & 0xff; +- outbuf[11] = max_io_sectors & 0xff; +- +- /* optimal transfer length */ +- outbuf[12] = (opt_io_size >> 24) & 0xff; +- outbuf[13] = (opt_io_size >> 16) & 0xff; +- outbuf[14] = (opt_io_size >> 8) & 0xff; +- outbuf[15] = opt_io_size & 0xff; +- +- /* max unmap LBA count, default is 1GB */ +- outbuf[20] = (max_unmap_sectors >> 24) & 0xff; +- outbuf[21] = (max_unmap_sectors >> 16) & 0xff; +- outbuf[22] = (max_unmap_sectors >> 8) & 0xff; +- outbuf[23] = max_unmap_sectors & 0xff; +- +- /* max unmap descriptors, 255 fit in 4 kb with an 8-byte header. */ +- outbuf[24] = 0; +- outbuf[25] = 0; +- outbuf[26] = 0; +- outbuf[27] = 255; +- +- /* optimal unmap granularity */ +- outbuf[28] = (unmap_sectors >> 24) & 0xff; +- outbuf[29] = (unmap_sectors >> 16) & 0xff; +- outbuf[30] = (unmap_sectors >> 8) & 0xff; +- outbuf[31] = unmap_sectors & 0xff; +- +- /* max write same size */ +- outbuf[36] = 0; +- outbuf[37] = 0; +- outbuf[38] = 0; +- outbuf[39] = 0; +- +- outbuf[40] = (max_io_sectors >> 24) & 0xff; +- outbuf[41] = (max_io_sectors >> 16) & 0xff; +- outbuf[42] = (max_io_sectors >> 8) & 0xff; +- outbuf[43] = max_io_sectors & 0xff; +- break; ++ DPRINTF("Inquiry EVPD[Device identification] " ++ "buffer size %zd\n", req->cmd.xfer); ++ ++ outbuf[buflen++] = 0x2; /* ASCII */ ++ outbuf[buflen++] = 0; /* not officially assigned */ ++ outbuf[buflen++] = 0; /* reserved */ ++ outbuf[buflen++] = id_len; /* length of data following */ ++ memcpy(outbuf + buflen, str, id_len); ++ buflen += id_len; ++ ++ if (s->qdev.wwn) { ++ outbuf[buflen++] = 0x1; /* Binary */ ++ outbuf[buflen++] = 0x3; /* NAA */ ++ outbuf[buflen++] = 0; /* reserved */ ++ outbuf[buflen++] = 8; ++ stq_be_p(&outbuf[buflen], s->qdev.wwn); ++ buflen += 8; + } +- case 0xb1: /* block device characteristics */ +- { +- buflen = 8; +- outbuf[4] = (s->rotation_rate >> 8) & 0xff; +- outbuf[5] = s->rotation_rate & 0xff; +- outbuf[6] = 0; +- outbuf[7] = 0; +- break; ++ ++ if (s->qdev.port_wwn) { ++ outbuf[buflen++] = 0x61; /* SAS / Binary */ ++ outbuf[buflen++] = 0x93; /* PIV / Target port / NAA */ ++ outbuf[buflen++] = 0; /* reserved */ ++ outbuf[buflen++] = 8; ++ stq_be_p(&outbuf[buflen], s->qdev.port_wwn); ++ buflen += 8; + } +- case 0xb2: /* thin provisioning */ +- { +- buflen = 8; +- outbuf[4] = 0; +- outbuf[5] = 0xe0; /* unmap & write_same 10/16 all supported */ +- outbuf[6] = s->qdev.conf.discard_granularity ? 2 : 1; +- outbuf[7] = 0; +- break; ++ ++ if (s->port_index) { ++ outbuf[buflen++] = 0x61; /* SAS / Binary */ ++ ++ /* PIV/Target port/relative target port */ ++ outbuf[buflen++] = 0x94; ++ ++ outbuf[buflen++] = 0; /* reserved */ ++ outbuf[buflen++] = 4; ++ stw_be_p(&outbuf[buflen + 2], s->port_index); ++ buflen += 4; + } +- default: ++ break; ++ } ++ case 0xb0: /* block limits */ ++ { ++ unsigned int unmap_sectors = ++ s->qdev.conf.discard_granularity / s->qdev.blocksize; ++ unsigned int min_io_size = ++ s->qdev.conf.min_io_size / s->qdev.blocksize; ++ unsigned int opt_io_size = ++ s->qdev.conf.opt_io_size / s->qdev.blocksize; ++ unsigned int max_unmap_sectors = ++ s->max_unmap_size / s->qdev.blocksize; ++ unsigned int max_io_sectors = ++ s->max_io_size / s->qdev.blocksize; ++ ++ if (s->qdev.type == TYPE_ROM) { ++ DPRINTF("Inquiry (EVPD[%02X] not supported for CDROM\n", ++ page_code); + return -1; + } +- /* done with EVPD */ +- assert(buflen - start <= 255); +- outbuf[start - 1] = buflen - start; +- return buflen; ++ if (s->qdev.type == TYPE_DISK) { ++ int max_transfer_blk = blk_get_max_transfer(s->qdev.conf.blk); ++ int max_io_sectors_blk = ++ max_transfer_blk / s->qdev.blocksize; ++ ++ max_io_sectors = ++ MIN_NON_ZERO(max_io_sectors_blk, max_io_sectors); ++ ++ /* min_io_size and opt_io_size can't be greater than ++ * max_io_sectors */ ++ if (min_io_size) { ++ min_io_size = MIN(min_io_size, max_io_sectors); ++ } ++ if (opt_io_size) { ++ opt_io_size = MIN(opt_io_size, max_io_sectors); ++ } ++ } ++ /* required VPD size with unmap support */ ++ buflen = 0x40; ++ memset(outbuf + 4, 0, buflen - 4); ++ ++ outbuf[4] = 0x1; /* wsnz */ ++ ++ /* optimal transfer length granularity */ ++ outbuf[6] = (min_io_size >> 8) & 0xff; ++ outbuf[7] = min_io_size & 0xff; ++ ++ /* maximum transfer length */ ++ outbuf[8] = (max_io_sectors >> 24) & 0xff; ++ outbuf[9] = (max_io_sectors >> 16) & 0xff; ++ outbuf[10] = (max_io_sectors >> 8) & 0xff; ++ outbuf[11] = max_io_sectors & 0xff; ++ ++ /* optimal transfer length */ ++ outbuf[12] = (opt_io_size >> 24) & 0xff; ++ outbuf[13] = (opt_io_size >> 16) & 0xff; ++ outbuf[14] = (opt_io_size >> 8) & 0xff; ++ outbuf[15] = opt_io_size & 0xff; ++ ++ /* max unmap LBA count, default is 1GB */ ++ outbuf[20] = (max_unmap_sectors >> 24) & 0xff; ++ outbuf[21] = (max_unmap_sectors >> 16) & 0xff; ++ outbuf[22] = (max_unmap_sectors >> 8) & 0xff; ++ outbuf[23] = max_unmap_sectors & 0xff; ++ ++ /* max unmap descriptors, 255 fit in 4 kb with an 8-byte header */ ++ outbuf[24] = 0; ++ outbuf[25] = 0; ++ outbuf[26] = 0; ++ outbuf[27] = 255; ++ ++ /* optimal unmap granularity */ ++ outbuf[28] = (unmap_sectors >> 24) & 0xff; ++ outbuf[29] = (unmap_sectors >> 16) & 0xff; ++ outbuf[30] = (unmap_sectors >> 8) & 0xff; ++ outbuf[31] = unmap_sectors & 0xff; ++ ++ /* max write same size */ ++ outbuf[36] = 0; ++ outbuf[37] = 0; ++ outbuf[38] = 0; ++ outbuf[39] = 0; ++ ++ outbuf[40] = (max_io_sectors >> 24) & 0xff; ++ outbuf[41] = (max_io_sectors >> 16) & 0xff; ++ outbuf[42] = (max_io_sectors >> 8) & 0xff; ++ outbuf[43] = max_io_sectors & 0xff; ++ break; ++ } ++ case 0xb1: /* block device characteristics */ ++ { ++ buflen = 8; ++ outbuf[4] = (s->rotation_rate >> 8) & 0xff; ++ outbuf[5] = s->rotation_rate & 0xff; ++ outbuf[6] = 0; ++ outbuf[7] = 0; ++ break; ++ } ++ case 0xb2: /* thin provisioning */ ++ { ++ buflen = 8; ++ outbuf[4] = 0; ++ outbuf[5] = 0xe0; /* unmap & write_same 10/16 all supported */ ++ outbuf[6] = s->qdev.conf.discard_granularity ? 2 : 1; ++ outbuf[7] = 0; ++ break; ++ } ++ default: ++ return -1; ++ } ++ /* done with EVPD */ ++ assert(buflen - start <= 255); ++ outbuf[start - 1] = buflen - start; ++ return buflen; ++} ++ ++static int scsi_disk_emulate_inquiry(SCSIRequest *req, uint8_t *outbuf) ++{ ++ SCSIDiskState *s = DO_UPCAST(SCSIDiskState, qdev, req->dev); ++ int buflen = 0; ++ ++ if (req->cmd.buf[1] & 0x1) { ++ /* Vital product data */ ++ return scsi_disk_emulate_vpd_page(req, outbuf); + } + + /* Standard INQUIRY data */ +@@ -3040,6 +3049,10 @@ static Property scsi_block_properties[] = { + DEFINE_PROP_DRIVE("drive", SCSIDiskState, qdev.conf.blk), + DEFINE_PROP_BOOL("share-rw", SCSIDiskState, qdev.conf.share_rw, false), + DEFINE_PROP_UINT16("rotation_rate", SCSIDiskState, rotation_rate, 0), ++ DEFINE_PROP_UINT64("max_unmap_size", SCSIDiskState, max_unmap_size, ++ DEFAULT_MAX_UNMAP_SIZE), ++ DEFINE_PROP_UINT64("max_io_size", SCSIDiskState, max_io_size, ++ DEFAULT_MAX_IO_SIZE), + DEFINE_PROP_INT32("scsi_version", SCSIDiskState, qdev.default_scsi_version, + -1), + DEFINE_PROP_END_OF_LIST(), +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index 381f04e..796162c 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -143,6 +143,43 @@ static int execute_command(BlockBackend *blk, + return 0; + } + ++static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) ++{ ++ /* ++ * EVPD set to zero returns the standard INQUIRY data. ++ * ++ * Check if scsi_version is unset (-1) to avoid re-defining it ++ * each time an INQUIRY with standard data is received. ++ * scsi_version is initialized with -1 in scsi_generic_reset ++ * and scsi_disk_reset, making sure that we'll set the ++ * scsi_version after a reset. If the version field of the ++ * INQUIRY response somehow changes after a guest reboot, ++ * we'll be able to keep track of it. ++ * ++ * On SCSI-2 and older, first 3 bits of byte 2 is the ++ * ANSI-approved version, while on later versions the ++ * whole byte 2 contains the version. Check if we're dealing ++ * with a newer version and, in that case, assign the ++ * whole byte. ++ */ ++ if (s->scsi_version == -1 && !(r->req.cmd.buf[1] & 0x01)) { ++ s->scsi_version = r->buf[2] & 0x07; ++ if (s->scsi_version > 2) { ++ s->scsi_version = r->buf[2]; ++ } ++ } ++ if (s->type == TYPE_DISK && r->req.cmd.buf[2] == 0xb0) { ++ uint32_t max_transfer = ++ blk_get_max_transfer(s->conf.blk) / s->blocksize; ++ ++ assert(max_transfer); ++ stl_be_p(&r->buf[8], max_transfer); ++ /* Also take care of the opt xfer len. */ ++ stl_be_p(&r->buf[12], ++ MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); ++ } ++} ++ + static void scsi_read_complete(void * opaque, int ret) + { + SCSIGenericReq *r = (SCSIGenericReq *)opaque; +@@ -195,39 +232,7 @@ static void scsi_read_complete(void * opaque, int ret) + } + } + if (r->req.cmd.buf[0] == INQUIRY) { +- /* +- * EVPD set to zero returns the standard INQUIRY data. +- * +- * Check if scsi_version is unset (-1) to avoid re-defining it +- * each time an INQUIRY with standard data is received. +- * scsi_version is initialized with -1 in scsi_generic_reset +- * and scsi_disk_reset, making sure that we'll set the +- * scsi_version after a reset. If the version field of the +- * INQUIRY response somehow changes after a guest reboot, +- * we'll be able to keep track of it. +- * +- * On SCSI-2 and older, first 3 bits of byte 2 is the +- * ANSI-approved version, while on later versions the +- * whole byte 2 contains the version. Check if we're dealing +- * with a newer version and, in that case, assign the +- * whole byte. +- */ +- if (s->scsi_version == -1 && !(r->req.cmd.buf[1] & 0x01)) { +- s->scsi_version = r->buf[2] & 0x07; +- if (s->scsi_version > 2) { +- s->scsi_version = r->buf[2]; +- } +- } +- if (s->type == TYPE_DISK && r->req.cmd.buf[2] == 0xb0) { +- uint32_t max_transfer = +- blk_get_max_transfer(s->conf.blk) / s->blocksize; +- +- assert(max_transfer); +- stl_be_p(&r->buf[8], max_transfer); +- /* Also take care of the opt xfer len. */ +- stl_be_p(&r->buf[12], +- MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); +- } ++ scsi_handle_inquiry_reply(r, s); + } + scsi_req_data(&r->req, len); + scsi_req_unref(&r->req); +diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h +index 1a7290d..5930a43 100644 +--- a/include/hw/scsi/scsi.h ++++ b/include/hw/scsi/scsi.h +@@ -188,6 +188,7 @@ void scsi_device_report_change(SCSIDevice *dev, SCSISense sense); + void scsi_device_unit_attention_reported(SCSIDevice *dev); + void scsi_generic_read_device_identification(SCSIDevice *dev); + int scsi_device_get_sense(SCSIDevice *dev, uint8_t *buf, int len, bool fixed); ++int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf); + SCSIDevice *scsi_device_find(SCSIBus *bus, int channel, int target, int lun); + + /* scsi-generic.c. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Add-cache-information-in-X86CPUDefinition.patch b/SOURCES/kvm-i386-Add-cache-information-in-X86CPUDefinition.patch new file mode 100644 index 0000000..0598621 --- /dev/null +++ b/SOURCES/kvm-i386-Add-cache-information-in-X86CPUDefinition.patch @@ -0,0 +1,71 @@ +From a05bef7298f7d30da342ef1a3f6d12bb33377fc5 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:18:55 +0100 +Subject: [PATCH 05/14] i386: Add cache information in X86CPUDefinition + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-3-ehabkost@redhat.com> +Patchwork-id: 81523 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 02/11] i386: Add cache information in X86CPUDefinition +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Add cache information in X86CPUDefinition and CPUX86State. + +Signed-off-by: Babu Moger +Tested-by: Geoffrey McRae +Reviewed-by: Eduardo Habkost +Message-Id: <20180510204148.11687-3-babu.moger@amd.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit 6aaeb05492ef668f415324f43e7d875c0f1e90b3) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 1 + + target/i386/cpu.h | 7 +++++++ + 2 files changed, 8 insertions(+) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 6c57b2f..50af741 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -1105,6 +1105,7 @@ struct X86CPUDefinition { + int stepping; + FeatureWordArray features; + const char *model_id; ++ CPUCaches *cache_info; + }; + + static X86CPUDefinition builtin_x86_defs[] = { +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index fa03e2c..372f8b7 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -1096,6 +1096,12 @@ typedef struct CPUCacheInfo { + } CPUCacheInfo; + + ++typedef struct CPUCaches { ++ CPUCacheInfo l1d_cache; ++ CPUCacheInfo l1i_cache; ++ CPUCacheInfo l2_cache; ++ CPUCacheInfo l3_cache; ++} CPUCaches; + + typedef struct CPUX86State { + /* standard registers */ +@@ -1282,6 +1288,7 @@ typedef struct CPUX86State { + /* Features that were explicitly enabled/disabled */ + FeatureWordArray user_features; + uint32_t cpuid_model[12]; ++ CPUCaches *cache_info; + + /* MTRRs */ + uint64_t mtrr_fixed[11]; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Add-new-property-to-control-cache-info.patch b/SOURCES/kvm-i386-Add-new-property-to-control-cache-info.patch new file mode 100644 index 0000000..3504bef --- /dev/null +++ b/SOURCES/kvm-i386-Add-new-property-to-control-cache-info.patch @@ -0,0 +1,287 @@ +From c78647f69c02d1004dfc1a2f1e2e24960634c795 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:18:57 +0100 +Subject: [PATCH 07/14] i386: Add new property to control cache info + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-5-ehabkost@redhat.com> +Patchwork-id: 81526 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 04/11] i386: Add new property to control cache info +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +The property legacy-cache will be used to control the cache information. +If user passes "-cpu legacy-cache" then older information will +be displayed even if the hardware supports new information. Otherwise +use the statically loaded cache definitions if available. + +Renamed the previous cache structures to legacy_*. If there is any change in +the cache information, then it needs to be initialized in builtin_x86_defs. + +Signed-off-by: Babu Moger +Tested-by: Geoffrey McRae +Message-Id: <20180514164156.27034-3-babu.moger@amd.com> +Reviewed-by: Eduardo Habkost +Signed-off-by: Eduardo Habkost +(cherry picked from commit ab8f992e3e63e91be257e4e343d386dae7be4bcb) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + include/hw/i386/pc.h | 4 +++ + target/i386/cpu.c | 97 ++++++++++++++++++++++++++++++++++++++-------------- + target/i386/cpu.h | 5 +++ + 3 files changed, 80 insertions(+), 26 deletions(-) + +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index ae84db4..5aebf6e 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -979,6 +979,10 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .driver = "Skylake-Server" "-" TYPE_X86_CPU,\ + .property = "clflushopt",\ + .value = "off",\ ++ },{ /* PC_RHEL7_5_COMPAT from PC_COMPAT_2_12 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "legacy-cache",\ ++ .value = "on",\ + }, + + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index bd0abc2..7dfc0fc 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -336,10 +336,14 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2, + } + } + +-/* Definitions of the hardcoded cache entries we expose: */ ++/* ++ * Definitions of the hardcoded cache entries we expose: ++ * These are legacy cache values. If there is a need to change any ++ * of these values please use builtin_x86_defs ++ */ + + /* L1 data cache: */ +-static CPUCacheInfo l1d_cache = { ++static CPUCacheInfo legacy_l1d_cache = { + .type = DCACHE, + .level = 1, + .size = 32 * KiB, +@@ -352,7 +356,7 @@ static CPUCacheInfo l1d_cache = { + }; + + /*FIXME: CPUID leaf 0x80000005 is inconsistent with leaves 2 & 4 */ +-static CPUCacheInfo l1d_cache_amd = { ++static CPUCacheInfo legacy_l1d_cache_amd = { + .type = DCACHE, + .level = 1, + .size = 64 * KiB, +@@ -366,7 +370,7 @@ static CPUCacheInfo l1d_cache_amd = { + }; + + /* L1 instruction cache: */ +-static CPUCacheInfo l1i_cache = { ++static CPUCacheInfo legacy_l1i_cache = { + .type = ICACHE, + .level = 1, + .size = 32 * KiB, +@@ -379,7 +383,7 @@ static CPUCacheInfo l1i_cache = { + }; + + /*FIXME: CPUID leaf 0x80000005 is inconsistent with leaves 2 & 4 */ +-static CPUCacheInfo l1i_cache_amd = { ++static CPUCacheInfo legacy_l1i_cache_amd = { + .type = ICACHE, + .level = 1, + .size = 64 * KiB, +@@ -393,7 +397,7 @@ static CPUCacheInfo l1i_cache_amd = { + }; + + /* Level 2 unified cache: */ +-static CPUCacheInfo l2_cache = { ++static CPUCacheInfo legacy_l2_cache = { + .type = UNIFIED_CACHE, + .level = 2, + .size = 4 * MiB, +@@ -406,7 +410,7 @@ static CPUCacheInfo l2_cache = { + }; + + /*FIXME: CPUID leaf 2 descriptor is inconsistent with CPUID leaf 4 */ +-static CPUCacheInfo l2_cache_cpuid2 = { ++static CPUCacheInfo legacy_l2_cache_cpuid2 = { + .type = UNIFIED_CACHE, + .level = 2, + .size = 2 * MiB, +@@ -416,7 +420,7 @@ static CPUCacheInfo l2_cache_cpuid2 = { + + + /*FIXME: CPUID leaf 0x80000006 is inconsistent with leaves 2 & 4 */ +-static CPUCacheInfo l2_cache_amd = { ++static CPUCacheInfo legacy_l2_cache_amd = { + .type = UNIFIED_CACHE, + .level = 2, + .size = 512 * KiB, +@@ -428,7 +432,7 @@ static CPUCacheInfo l2_cache_amd = { + }; + + /* Level 3 unified cache: */ +-static CPUCacheInfo l3_cache = { ++static CPUCacheInfo legacy_l3_cache = { + .type = UNIFIED_CACHE, + .level = 3, + .size = 16 * MiB, +@@ -3321,6 +3325,10 @@ static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp) + env->features[w] = def->features[w]; + } + ++ /* Store Cache information from the X86CPUDefinition if available */ ++ env->cache_info = def->cache_info; ++ cpu->legacy_cache = def->cache_info ? 0 : 1; ++ + /* Special cases not set in the X86CPUDefinition structs: */ + /* TODO: in-kernel irqchip for hvf */ + if (kvm_enabled()) { +@@ -3670,11 +3678,21 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + if (!cpu->enable_l3_cache) { + *ecx = 0; + } else { +- *ecx = cpuid2_cache_descriptor(&l3_cache); ++ if (env->cache_info && !cpu->legacy_cache) { ++ *ecx = cpuid2_cache_descriptor(&env->cache_info->l3_cache); ++ } else { ++ *ecx = cpuid2_cache_descriptor(&legacy_l3_cache); ++ } ++ } ++ if (env->cache_info && !cpu->legacy_cache) { ++ *edx = (cpuid2_cache_descriptor(&env->cache_info->l1d_cache) << 16) | ++ (cpuid2_cache_descriptor(&env->cache_info->l1i_cache) << 8) | ++ (cpuid2_cache_descriptor(&env->cache_info->l2_cache)); ++ } else { ++ *edx = (cpuid2_cache_descriptor(&legacy_l1d_cache) << 16) | ++ (cpuid2_cache_descriptor(&legacy_l1i_cache) << 8) | ++ (cpuid2_cache_descriptor(&legacy_l2_cache_cpuid2)); + } +- *edx = (cpuid2_cache_descriptor(&l1d_cache) << 16) | +- (cpuid2_cache_descriptor(&l1i_cache) << 8) | +- (cpuid2_cache_descriptor(&l2_cache_cpuid2)); + break; + case 4: + /* cache info: needed for Core compatibility */ +@@ -3687,27 +3705,35 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + } + } else { + *eax = 0; ++ CPUCacheInfo *l1d, *l1i, *l2, *l3; ++ if (env->cache_info && !cpu->legacy_cache) { ++ l1d = &env->cache_info->l1d_cache; ++ l1i = &env->cache_info->l1i_cache; ++ l2 = &env->cache_info->l2_cache; ++ l3 = &env->cache_info->l3_cache; ++ } else { ++ l1d = &legacy_l1d_cache; ++ l1i = &legacy_l1i_cache; ++ l2 = &legacy_l2_cache; ++ l3 = &legacy_l3_cache; ++ } + switch (count) { + case 0: /* L1 dcache info */ +- encode_cache_cpuid4(&l1d_cache, +- 1, cs->nr_cores, ++ encode_cache_cpuid4(l1d, 1, cs->nr_cores, + eax, ebx, ecx, edx); + break; + case 1: /* L1 icache info */ +- encode_cache_cpuid4(&l1i_cache, +- 1, cs->nr_cores, ++ encode_cache_cpuid4(l1i, 1, cs->nr_cores, + eax, ebx, ecx, edx); + break; + case 2: /* L2 cache info */ +- encode_cache_cpuid4(&l2_cache, +- cs->nr_threads, cs->nr_cores, ++ encode_cache_cpuid4(l2, cs->nr_threads, cs->nr_cores, + eax, ebx, ecx, edx); + break; + case 3: /* L3 cache info */ + pkg_offset = apicid_pkg_offset(cs->nr_cores, cs->nr_threads); + if (cpu->enable_l3_cache) { +- encode_cache_cpuid4(&l3_cache, +- (1 << pkg_offset), cs->nr_cores, ++ encode_cache_cpuid4(l3, (1 << pkg_offset), cs->nr_cores, + eax, ebx, ecx, edx); + break; + } +@@ -3920,8 +3946,13 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + (L1_ITLB_2M_ASSOC << 8) | (L1_ITLB_2M_ENTRIES); + *ebx = (L1_DTLB_4K_ASSOC << 24) | (L1_DTLB_4K_ENTRIES << 16) | \ + (L1_ITLB_4K_ASSOC << 8) | (L1_ITLB_4K_ENTRIES); +- *ecx = encode_cache_cpuid80000005(&l1d_cache_amd); +- *edx = encode_cache_cpuid80000005(&l1i_cache_amd); ++ if (env->cache_info && !cpu->legacy_cache) { ++ *ecx = encode_cache_cpuid80000005(&env->cache_info->l1d_cache); ++ *edx = encode_cache_cpuid80000005(&env->cache_info->l1i_cache); ++ } else { ++ *ecx = encode_cache_cpuid80000005(&legacy_l1d_cache_amd); ++ *edx = encode_cache_cpuid80000005(&legacy_l1i_cache_amd); ++ } + break; + case 0x80000006: + /* cache info (L2 cache) */ +@@ -3937,9 +3968,17 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + (L2_DTLB_4K_ENTRIES << 16) | \ + (AMD_ENC_ASSOC(L2_ITLB_4K_ASSOC) << 12) | \ + (L2_ITLB_4K_ENTRIES); +- encode_cache_cpuid80000006(&l2_cache_amd, +- cpu->enable_l3_cache ? &l3_cache : NULL, +- ecx, edx); ++ if (env->cache_info && !cpu->legacy_cache) { ++ encode_cache_cpuid80000006(&env->cache_info->l2_cache, ++ cpu->enable_l3_cache ? ++ &env->cache_info->l3_cache : NULL, ++ ecx, edx); ++ } else { ++ encode_cache_cpuid80000006(&legacy_l2_cache_amd, ++ cpu->enable_l3_cache ? ++ &legacy_l3_cache : NULL, ++ ecx, edx); ++ } + break; + case 0x80000007: + *eax = 0; +@@ -5119,6 +5158,12 @@ static Property x86_cpu_properties[] = { + false), + DEFINE_PROP_BOOL("vmware-cpuid-freq", X86CPU, vmware_cpuid_freq, true), + DEFINE_PROP_BOOL("tcg-cpuid", X86CPU, expose_tcg, true), ++ /* ++ * lecacy_cache defaults to CPU model being chosen. This is set in ++ * x86_cpu_load_def based on cache_info which is initialized in ++ * builtin_x86_defs ++ */ ++ DEFINE_PROP_BOOL("legacy-cache", X86CPU, legacy_cache, false), + + /* + * From "Requirements for Implementing the Microsoft +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index 372f8b7..31715d1 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -1394,6 +1394,11 @@ struct X86CPU { + */ + bool enable_l3_cache; + ++ /* Compatibility bits for old machine types. ++ * If true present the old cache topology information ++ */ ++ bool legacy_cache; ++ + /* Compatibility bits for old machine types: */ + bool enable_cpuid_0xb; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Add-stibp-flag-name.patch b/SOURCES/kvm-i386-Add-stibp-flag-name.patch new file mode 100644 index 0000000..a42f7f4 --- /dev/null +++ b/SOURCES/kvm-i386-Add-stibp-flag-name.patch @@ -0,0 +1,51 @@ +From 0d13ff22c572978cbef5e0b9ad81143178315bf6 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Mon, 10 Dec 2018 19:25:34 +0000 +Subject: [PATCH 06/13] i386: Add "stibp" flag name + +RH-Author: Eduardo Habkost +Message-id: <20181210192534.21567-2-ehabkost@redhat.com> +Patchwork-id: 83359 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] i386: Add "stibp" flag name +Bugzilla: 1639446 +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Paolo Bonzini + +The STIBP flag may be supported by the host KVM module, so QEMU +can allow it to be configured manually, and it can be exposed to +guests when using "-cpu host". + +No additional migration code is required because the whole +contents of spec_ctrl is already migrated in the "cpu/spec_ctrl" +section. + +Corresponding KVM patch was submitted at: +https://lore.kernel.org/lkml/20181205191956.31480-1-ehabkost@redhat.com/ + +Signed-off-by: Eduardo Habkost +Message-Id: <20181210180250.31299-1-ehabkost@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit 218f1ef8f700b9a25cad3299cb0728d71b7634dd) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 228935f..a44912c 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -1007,7 +1007,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, +- NULL, NULL, "spec-ctrl", NULL, ++ NULL, NULL, "spec-ctrl", "stibp", + NULL, NULL, NULL, "ssbd", + }, + .cpuid_eax = 7, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Add-support-for-CPUID_8000_001E-for-AMD.patch b/SOURCES/kvm-i386-Add-support-for-CPUID_8000_001E-for-AMD.patch new file mode 100644 index 0000000..10e1a69 --- /dev/null +++ b/SOURCES/kvm-i386-Add-support-for-CPUID_8000_001E-for-AMD.patch @@ -0,0 +1,137 @@ +From f5560b5492153802d046fab1d873970f57ebb42f Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:19:00 +0100 +Subject: [PATCH 10/14] i386: Add support for CPUID_8000_001E for AMD + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-8-ehabkost@redhat.com> +Patchwork-id: 81533 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 07/11] i386: Add support for CPUID_8000_001E for AMD +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Add support for cpuid leaf CPUID_8000_001E. Build the config that closely +match the underlying hardware. Please refer to the Processor Programming +Reference (PPR) for AMD Family 17h Model for more details. + +Signed-off-by: Babu Moger +Message-Id: <1528498581-131037-2-git-send-email-babu.moger@amd.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit ed78467a214595a63af7800a073a03ffe37cd7db) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 86 insertions(+) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index d2474d7..3ed1e47 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -429,6 +429,87 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs, + (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0); + } + ++/* Data structure to hold the configuration info for a given core index */ ++struct core_topology { ++ /* core complex id of the current core index */ ++ int ccx_id; ++ /* ++ * Adjusted core index for this core in the topology ++ * This can be 0,1,2,3 with max 4 cores in a core complex ++ */ ++ int core_id; ++ /* Node id for this core index */ ++ int node_id; ++ /* Number of nodes in this config */ ++ int num_nodes; ++}; ++ ++/* ++ * Build the configuration closely match the EPYC hardware. Using the EPYC ++ * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE) ++ * right now. This could change in future. ++ * nr_cores : Total number of cores in the config ++ * core_id : Core index of the current CPU ++ * topo : Data structure to hold all the config info for this core index ++ */ ++static void build_core_topology(int nr_cores, int core_id, ++ struct core_topology *topo) ++{ ++ int nodes, cores_in_ccx; ++ ++ /* First get the number of nodes required */ ++ nodes = nodes_in_socket(nr_cores); ++ ++ cores_in_ccx = cores_in_core_complex(nr_cores); ++ ++ topo->node_id = core_id / (cores_in_ccx * MAX_CCX); ++ topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx; ++ topo->core_id = core_id % cores_in_ccx; ++ topo->num_nodes = nodes; ++} ++ ++/* Encode cache info for CPUID[8000001E] */ ++static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu, ++ uint32_t *eax, uint32_t *ebx, ++ uint32_t *ecx, uint32_t *edx) ++{ ++ struct core_topology topo = {0}; ++ ++ build_core_topology(cs->nr_cores, cpu->core_id, &topo); ++ *eax = cpu->apic_id; ++ /* ++ * CPUID_Fn8000001E_EBX ++ * 31:16 Reserved ++ * 15:8 Threads per core (The number of threads per core is ++ * Threads per core + 1) ++ * 7:0 Core id (see bit decoding below) ++ * SMT: ++ * 4:3 node id ++ * 2 Core complex id ++ * 1:0 Core id ++ * Non SMT: ++ * 5:4 node id ++ * 3 Core complex id ++ * 1:0 Core id ++ */ ++ if (cs->nr_threads - 1) { ++ *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) | ++ (topo.ccx_id << 2) | topo.core_id; ++ } else { ++ *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id; ++ } ++ /* ++ * CPUID_Fn8000001E_ECX ++ * 31:11 Reserved ++ * 10:8 Nodes per processor (Nodes per processor is number of nodes + 1) ++ * 7:0 Node id (see bit decoding below) ++ * 2 Socket id ++ * 1:0 Node id ++ */ ++ *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) | topo.node_id; ++ *edx = 0; ++} ++ + /* + * Definitions of the hardcoded cache entries we expose: + * These are legacy cache values. If there is a need to change any +@@ -4105,6 +4186,11 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + break; + } + break; ++ case 0x8000001E: ++ assert(cpu->core_id <= 255); ++ encode_topo_cpuid8000001e(cs, cpu, ++ eax, ebx, ecx, edx); ++ break; + case 0xC0000000: + *eax = env->cpuid_xlevel2; + *ebx = 0; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Allow-TOPOEXT-to-be-enabled-on-older-kernels.patch b/SOURCES/kvm-i386-Allow-TOPOEXT-to-be-enabled-on-older-kernels.patch new file mode 100644 index 0000000..7eb91d2 --- /dev/null +++ b/SOURCES/kvm-i386-Allow-TOPOEXT-to-be-enabled-on-older-kernels.patch @@ -0,0 +1,53 @@ +From 0a6e593487844fd00920c0caab64aa98eac368a1 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:19:04 +0100 +Subject: [PATCH 14/14] i386: Allow TOPOEXT to be enabled on older kernels + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-12-ehabkost@redhat.com> +Patchwork-id: 81534 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 11/11] i386: Allow TOPOEXT to be enabled on older kernels +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Enabling TOPOEXT feature might cause compatibility issues if +older kernels does not set this feature. Lets set this feature +unconditionally. + +Signed-off-by: Babu Moger +Message-Id: <1528939107-17193-2-git-send-email-babu.moger@amd.com> +[ehabkost: rewrite comment and commit message] +Signed-off-by: Eduardo Habkost +(cherry picked from commit f98bbd8304112187cafc3e636c31b2a3865d2717) +Signed-off-by: Eduardo Habkost + +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/kvm.c | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/target/i386/kvm.c b/target/i386/kvm.c +index 6e66f9c..bb5e485 100644 +--- a/target/i386/kvm.c ++++ b/target/i386/kvm.c +@@ -371,6 +371,13 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function, + if (host_tsx_blacklisted()) { + ret &= ~(CPUID_7_0_EBX_RTM | CPUID_7_0_EBX_HLE); + } ++ } else if (function == 0x80000001 && reg == R_ECX) { ++ /* ++ * It's safe to enable TOPOEXT even if it's not returned by ++ * GET_SUPPORTED_CPUID. Unconditionally enabling TOPOEXT here allows ++ * us to keep CPU models including TOPOEXT runnable on older kernels. ++ */ ++ ret |= CPUID_EXT3_TOPOEXT; + } else if (function == 0x80000001 && reg == R_EDX) { + /* On Intel, kvm returns cpuid according to the Intel spec, + * so add missing bits according to the AMD spec: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Clean-up-cache-CPUID-code.patch b/SOURCES/kvm-i386-Clean-up-cache-CPUID-code.patch new file mode 100644 index 0000000..77ce80d --- /dev/null +++ b/SOURCES/kvm-i386-Clean-up-cache-CPUID-code.patch @@ -0,0 +1,280 @@ +From 980b21f10d74e23a5eb458eaad67c8a2dea65929 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:18:58 +0100 +Subject: [PATCH 08/14] i386: Clean up cache CPUID code + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-6-ehabkost@redhat.com> +Patchwork-id: 81528 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 05/11] i386: Clean up cache CPUID code +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +Always initialize CPUCaches structs with cache information, even +if legacy_cache=true. Use different CPUCaches struct for +CPUID[2], CPUID[4], and the AMD CPUID leaves. + +This will simplify a lot the logic inside cpu_x86_cpuid(). + +Signed-off-by: Eduardo Habkost +Signed-off-by: Babu Moger +Message-Id: <1527176614-26271-2-git-send-email-babu.moger@amd.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit a9f27ea9adc8c695197bd08f2e938ef7b4183f07) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 117 +++++++++++++++++++++++++++--------------------------- + target/i386/cpu.h | 14 ++++--- + 2 files changed, 67 insertions(+), 64 deletions(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 7dfc0fc..f98a964 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -1113,7 +1113,7 @@ struct X86CPUDefinition { + }; + + static CPUCaches epyc_cache_info = { +- .l1d_cache = { ++ .l1d_cache = &(CPUCacheInfo) { + .type = DCACHE, + .level = 1, + .size = 32 * KiB, +@@ -1125,7 +1125,7 @@ static CPUCaches epyc_cache_info = { + .self_init = 1, + .no_invd_sharing = true, + }, +- .l1i_cache = { ++ .l1i_cache = &(CPUCacheInfo) { + .type = ICACHE, + .level = 1, + .size = 64 * KiB, +@@ -1137,7 +1137,7 @@ static CPUCaches epyc_cache_info = { + .self_init = 1, + .no_invd_sharing = true, + }, +- .l2_cache = { ++ .l2_cache = &(CPUCacheInfo) { + .type = UNIFIED_CACHE, + .level = 2, + .size = 512 * KiB, +@@ -1147,7 +1147,7 @@ static CPUCaches epyc_cache_info = { + .sets = 1024, + .lines_per_tag = 1, + }, +- .l3_cache = { ++ .l3_cache = &(CPUCacheInfo) { + .type = UNIFIED_CACHE, + .level = 3, + .size = 8 * MiB, +@@ -3325,9 +3325,8 @@ static void x86_cpu_load_def(X86CPU *cpu, X86CPUDefinition *def, Error **errp) + env->features[w] = def->features[w]; + } + +- /* Store Cache information from the X86CPUDefinition if available */ +- env->cache_info = def->cache_info; +- cpu->legacy_cache = def->cache_info ? 0 : 1; ++ /* legacy-cache defaults to 'off' if CPU model provides cache info */ ++ cpu->legacy_cache = !def->cache_info; + + /* Special cases not set in the X86CPUDefinition structs: */ + /* TODO: in-kernel irqchip for hvf */ +@@ -3678,21 +3677,11 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + if (!cpu->enable_l3_cache) { + *ecx = 0; + } else { +- if (env->cache_info && !cpu->legacy_cache) { +- *ecx = cpuid2_cache_descriptor(&env->cache_info->l3_cache); +- } else { +- *ecx = cpuid2_cache_descriptor(&legacy_l3_cache); +- } +- } +- if (env->cache_info && !cpu->legacy_cache) { +- *edx = (cpuid2_cache_descriptor(&env->cache_info->l1d_cache) << 16) | +- (cpuid2_cache_descriptor(&env->cache_info->l1i_cache) << 8) | +- (cpuid2_cache_descriptor(&env->cache_info->l2_cache)); +- } else { +- *edx = (cpuid2_cache_descriptor(&legacy_l1d_cache) << 16) | +- (cpuid2_cache_descriptor(&legacy_l1i_cache) << 8) | +- (cpuid2_cache_descriptor(&legacy_l2_cache_cpuid2)); ++ *ecx = cpuid2_cache_descriptor(env->cache_info_cpuid2.l3_cache); + } ++ *edx = (cpuid2_cache_descriptor(env->cache_info_cpuid2.l1d_cache) << 16) | ++ (cpuid2_cache_descriptor(env->cache_info_cpuid2.l1i_cache) << 8) | ++ (cpuid2_cache_descriptor(env->cache_info_cpuid2.l2_cache)); + break; + case 4: + /* cache info: needed for Core compatibility */ +@@ -3705,35 +3694,27 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + } + } else { + *eax = 0; +- CPUCacheInfo *l1d, *l1i, *l2, *l3; +- if (env->cache_info && !cpu->legacy_cache) { +- l1d = &env->cache_info->l1d_cache; +- l1i = &env->cache_info->l1i_cache; +- l2 = &env->cache_info->l2_cache; +- l3 = &env->cache_info->l3_cache; +- } else { +- l1d = &legacy_l1d_cache; +- l1i = &legacy_l1i_cache; +- l2 = &legacy_l2_cache; +- l3 = &legacy_l3_cache; +- } + switch (count) { + case 0: /* L1 dcache info */ +- encode_cache_cpuid4(l1d, 1, cs->nr_cores, ++ encode_cache_cpuid4(env->cache_info_cpuid4.l1d_cache, ++ 1, cs->nr_cores, + eax, ebx, ecx, edx); + break; + case 1: /* L1 icache info */ +- encode_cache_cpuid4(l1i, 1, cs->nr_cores, ++ encode_cache_cpuid4(env->cache_info_cpuid4.l1i_cache, ++ 1, cs->nr_cores, + eax, ebx, ecx, edx); + break; + case 2: /* L2 cache info */ +- encode_cache_cpuid4(l2, cs->nr_threads, cs->nr_cores, ++ encode_cache_cpuid4(env->cache_info_cpuid4.l2_cache, ++ cs->nr_threads, cs->nr_cores, + eax, ebx, ecx, edx); + break; + case 3: /* L3 cache info */ + pkg_offset = apicid_pkg_offset(cs->nr_cores, cs->nr_threads); + if (cpu->enable_l3_cache) { +- encode_cache_cpuid4(l3, (1 << pkg_offset), cs->nr_cores, ++ encode_cache_cpuid4(env->cache_info_cpuid4.l3_cache, ++ (1 << pkg_offset), cs->nr_cores, + eax, ebx, ecx, edx); + break; + } +@@ -3946,13 +3927,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + (L1_ITLB_2M_ASSOC << 8) | (L1_ITLB_2M_ENTRIES); + *ebx = (L1_DTLB_4K_ASSOC << 24) | (L1_DTLB_4K_ENTRIES << 16) | \ + (L1_ITLB_4K_ASSOC << 8) | (L1_ITLB_4K_ENTRIES); +- if (env->cache_info && !cpu->legacy_cache) { +- *ecx = encode_cache_cpuid80000005(&env->cache_info->l1d_cache); +- *edx = encode_cache_cpuid80000005(&env->cache_info->l1i_cache); +- } else { +- *ecx = encode_cache_cpuid80000005(&legacy_l1d_cache_amd); +- *edx = encode_cache_cpuid80000005(&legacy_l1i_cache_amd); +- } ++ *ecx = encode_cache_cpuid80000005(env->cache_info_amd.l1d_cache); ++ *edx = encode_cache_cpuid80000005(env->cache_info_amd.l1i_cache); + break; + case 0x80000006: + /* cache info (L2 cache) */ +@@ -3968,17 +3944,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + (L2_DTLB_4K_ENTRIES << 16) | \ + (AMD_ENC_ASSOC(L2_ITLB_4K_ASSOC) << 12) | \ + (L2_ITLB_4K_ENTRIES); +- if (env->cache_info && !cpu->legacy_cache) { +- encode_cache_cpuid80000006(&env->cache_info->l2_cache, +- cpu->enable_l3_cache ? +- &env->cache_info->l3_cache : NULL, +- ecx, edx); +- } else { +- encode_cache_cpuid80000006(&legacy_l2_cache_amd, +- cpu->enable_l3_cache ? +- &legacy_l3_cache : NULL, +- ecx, edx); +- } ++ encode_cache_cpuid80000006(env->cache_info_amd.l2_cache, ++ cpu->enable_l3_cache ? ++ env->cache_info_amd.l3_cache : NULL, ++ ecx, edx); + break; + case 0x80000007: + *eax = 0; +@@ -4675,6 +4644,37 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp) + cpu->phys_bits = 32; + } + } ++ ++ /* Cache information initialization */ ++ if (!cpu->legacy_cache) { ++ if (!xcc->cpu_def || !xcc->cpu_def->cache_info) { ++ char *name = x86_cpu_class_get_model_name(xcc); ++ error_setg(errp, ++ "CPU model '%s' doesn't support legacy-cache=off", name); ++ g_free(name); ++ return; ++ } ++ env->cache_info_cpuid2 = env->cache_info_cpuid4 = env->cache_info_amd = ++ *xcc->cpu_def->cache_info; ++ } else { ++ /* Build legacy cache information */ ++ env->cache_info_cpuid2.l1d_cache = &legacy_l1d_cache; ++ env->cache_info_cpuid2.l1i_cache = &legacy_l1i_cache; ++ env->cache_info_cpuid2.l2_cache = &legacy_l2_cache_cpuid2; ++ env->cache_info_cpuid2.l3_cache = &legacy_l3_cache; ++ ++ env->cache_info_cpuid4.l1d_cache = &legacy_l1d_cache; ++ env->cache_info_cpuid4.l1i_cache = &legacy_l1i_cache; ++ env->cache_info_cpuid4.l2_cache = &legacy_l2_cache; ++ env->cache_info_cpuid4.l3_cache = &legacy_l3_cache; ++ ++ env->cache_info_amd.l1d_cache = &legacy_l1d_cache_amd; ++ env->cache_info_amd.l1i_cache = &legacy_l1i_cache_amd; ++ env->cache_info_amd.l2_cache = &legacy_l2_cache_amd; ++ env->cache_info_amd.l3_cache = &legacy_l3_cache; ++ } ++ ++ + cpu_exec_realizefn(cs, &local_err); + if (local_err != NULL) { + error_propagate(errp, local_err); +@@ -5159,11 +5159,10 @@ static Property x86_cpu_properties[] = { + DEFINE_PROP_BOOL("vmware-cpuid-freq", X86CPU, vmware_cpuid_freq, true), + DEFINE_PROP_BOOL("tcg-cpuid", X86CPU, expose_tcg, true), + /* +- * lecacy_cache defaults to CPU model being chosen. This is set in +- * x86_cpu_load_def based on cache_info which is initialized in +- * builtin_x86_defs ++ * lecacy_cache defaults to true unless the CPU model provides its ++ * own cache information (see x86_cpu_load_def()). + */ +- DEFINE_PROP_BOOL("legacy-cache", X86CPU, legacy_cache, false), ++ DEFINE_PROP_BOOL("legacy-cache", X86CPU, legacy_cache, true), + + /* + * From "Requirements for Implementing the Microsoft +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index 31715d1..88fdf80 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -1097,10 +1097,10 @@ typedef struct CPUCacheInfo { + + + typedef struct CPUCaches { +- CPUCacheInfo l1d_cache; +- CPUCacheInfo l1i_cache; +- CPUCacheInfo l2_cache; +- CPUCacheInfo l3_cache; ++ CPUCacheInfo *l1d_cache; ++ CPUCacheInfo *l1i_cache; ++ CPUCacheInfo *l2_cache; ++ CPUCacheInfo *l3_cache; + } CPUCaches; + + typedef struct CPUX86State { +@@ -1288,7 +1288,11 @@ typedef struct CPUX86State { + /* Features that were explicitly enabled/disabled */ + FeatureWordArray user_features; + uint32_t cpuid_model[12]; +- CPUCaches *cache_info; ++ /* Cache information for CPUID. When legacy-cache=on, the cache data ++ * on each CPUID leaf will be different, because we keep compatibility ++ * with old QEMU versions. ++ */ ++ CPUCaches cache_info_cpuid2, cache_info_cpuid4, cache_info_amd; + + /* MTRRs */ + uint64_t mtrr_fixed[11]; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Define-the-Virt-SSBD-MSR-and-handling-of-it-CVE.patch b/SOURCES/kvm-i386-Define-the-Virt-SSBD-MSR-and-handling-of-it-CVE.patch new file mode 100644 index 0000000..a835a54 --- /dev/null +++ b/SOURCES/kvm-i386-Define-the-Virt-SSBD-MSR-and-handling-of-it-CVE.patch @@ -0,0 +1,164 @@ +From 96f6d8855246a66bde59fd3e663bee0da00f709b Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Wed, 13 Jun 2018 18:08:11 +0200 +Subject: [PATCH 152/268] i386: Define the Virt SSBD MSR and handling of it + (CVE-2018-3639) +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Habkost +Message-id: <20180613180812.28169-2-ehabkost@redhat.com> +Patchwork-id: 80677 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/2] i386: Define the Virt SSBD MSR and handling of it (CVE-2018-3639) +Bugzilla: 1574216 +RH-Acked-by: Igor Mammedov +RH-Acked-by: Miroslav Rezanina +RH-Acked-by: Laurent Vivier + +From: Konrad Rzeszutek Wilk + +"Some AMD processors only support a non-architectural means of enabling +speculative store bypass disable (SSBD). To allow a simplified view of +this to a guest, an architectural definition has been created through a new +CPUID bit, 0x80000008_EBX[25], and a new MSR, 0xc001011f. With this, a +hypervisor can virtualize the existence of this definition and provide an +architectural method for using SSBD to a guest. + +Add the new CPUID feature, the new MSR and update the existing SSBD +support to use this MSR when present." (from x86/speculation: Add virtualized +speculative store bypass disable support in Linux). + +Backport conflicts: + * target-i386/machine.c: trivial conflict with vmstate_xsave section + +Signed-off-by: Konrad Rzeszutek Wilk +Reviewed-by: Daniel P. Berrangé +Signed-off-by: Daniel P. Berrangé +Message-Id: <20180521215424.13520-4-berrange@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit cfeea0c021db6234c154dbc723730e81553924ff) +Signed-off-by: Eduardo Habkost +Signed-off-by: Miroslav Rezanina +--- + target/i386/cpu.h | 2 ++ + target/i386/kvm.c | 16 ++++++++++++++-- + target/i386/machine.c | 20 ++++++++++++++++++++ + 3 files changed, 36 insertions(+), 2 deletions(-) + +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index 88fdf80..c47db96 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -351,6 +351,7 @@ typedef enum X86Seg { + #define MSR_IA32_FEATURE_CONTROL 0x0000003a + #define MSR_TSC_ADJUST 0x0000003b + #define MSR_IA32_SPEC_CTRL 0x48 ++#define MSR_VIRT_SSBD 0xc001011f + #define MSR_IA32_TSCDEADLINE 0x6e0 + + #define FEATURE_CONTROL_LOCKED (1<<0) +@@ -1208,6 +1209,7 @@ typedef struct CPUX86State { + uint32_t pkru; + + uint64_t spec_ctrl; ++ uint64_t virt_ssbd; + + /* End of state preserved by INIT (dummy marker). */ + struct {} end_init_save; +diff --git a/target/i386/kvm.c b/target/i386/kvm.c +index bb5e485..00f2141 100644 +--- a/target/i386/kvm.c ++++ b/target/i386/kvm.c +@@ -92,6 +92,7 @@ static bool has_msr_hv_stimer; + static bool has_msr_hv_frequencies; + static bool has_msr_xss; + static bool has_msr_spec_ctrl; ++static bool has_msr_virt_ssbd; + static bool has_msr_smi_count; + + static uint32_t has_architectural_pmu_version; +@@ -1248,6 +1249,9 @@ static int kvm_get_supported_msrs(KVMState *s) + case MSR_IA32_SPEC_CTRL: + has_msr_spec_ctrl = true; + break; ++ case MSR_VIRT_SSBD: ++ has_msr_virt_ssbd = true; ++ break; + } + } + } +@@ -1736,6 +1740,10 @@ static int kvm_put_msrs(X86CPU *cpu, int level) + if (has_msr_spec_ctrl) { + kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, env->spec_ctrl); + } ++ if (has_msr_virt_ssbd) { ++ kvm_msr_entry_add(cpu, MSR_VIRT_SSBD, env->virt_ssbd); ++ } ++ + #ifdef TARGET_X86_64 + if (lm_capable_kernel) { + kvm_msr_entry_add(cpu, MSR_CSTAR, env->cstar); +@@ -2107,8 +2115,9 @@ static int kvm_get_msrs(X86CPU *cpu) + if (has_msr_spec_ctrl) { + kvm_msr_entry_add(cpu, MSR_IA32_SPEC_CTRL, 0); + } +- +- ++ if (has_msr_virt_ssbd) { ++ kvm_msr_entry_add(cpu, MSR_VIRT_SSBD, 0); ++ } + if (!env->tsc_valid) { + kvm_msr_entry_add(cpu, MSR_IA32_TSC, 0); + env->tsc_valid = !runstate_is_running(); +@@ -2474,6 +2483,9 @@ static int kvm_get_msrs(X86CPU *cpu) + case MSR_IA32_SPEC_CTRL: + env->spec_ctrl = msrs[i].data; + break; ++ case MSR_VIRT_SSBD: ++ env->virt_ssbd = msrs[i].data; ++ break; + case MSR_IA32_RTIT_CTL: + env->msr_rtit_ctrl = msrs[i].data; + break; +diff --git a/target/i386/machine.c b/target/i386/machine.c +index c9a3b5c..5e9a19b 100644 +--- a/target/i386/machine.c ++++ b/target/i386/machine.c +@@ -913,6 +913,25 @@ static const VMStateDescription vmstate_xsave ={ + } + }; + ++static bool virt_ssbd_needed(void *opaque) ++{ ++ X86CPU *cpu = opaque; ++ CPUX86State *env = &cpu->env; ++ ++ return env->virt_ssbd != 0; ++} ++ ++static const VMStateDescription vmstate_msr_virt_ssbd = { ++ .name = "cpu/virt_ssbd", ++ .version_id = 1, ++ .minimum_version_id = 1, ++ .needed = virt_ssbd_needed, ++ .fields = (VMStateField[]){ ++ VMSTATE_UINT64(env.virt_ssbd, X86CPU), ++ VMSTATE_END_OF_LIST() ++ } ++}; ++ + VMStateDescription vmstate_x86_cpu = { + .name = "cpu", + .version_id = 12, +@@ -1036,6 +1055,7 @@ VMStateDescription vmstate_x86_cpu = { + &vmstate_mcg_ext_ctl, + &vmstate_msr_intel_pt, + &vmstate_xsave, ++ &vmstate_msr_virt_ssbd, + NULL + } + }; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Disable-TOPOEXT-by-default-on-cpu-host.patch b/SOURCES/kvm-i386-Disable-TOPOEXT-by-default-on-cpu-host.patch new file mode 100644 index 0000000..bd82032 --- /dev/null +++ b/SOURCES/kvm-i386-Disable-TOPOEXT-by-default-on-cpu-host.patch @@ -0,0 +1,54 @@ +From 42405600b468b79523f7e8171c1aac4a3d012792 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Tue, 21 Aug 2018 19:15:41 +0100 +Subject: [PATCH 1/2] i386: Disable TOPOEXT by default on "-cpu host" + +RH-Author: Eduardo Habkost +Message-id: <20180821191541.31916-2-ehabkost@redhat.com> +Patchwork-id: 81904 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 1/1] i386: Disable TOPOEXT by default on "-cpu host" +Bugzilla: 1619804 +RH-Acked-by: Miroslav Rezanina +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Igor Mammedov + +Enabling TOPOEXT is always allowed, but it can't be enabled +blindly by "-cpu host" because it may make guests crash if the +rest of the cache topology information isn't provided or isn't +consistent. + +This addresses the bug reported at: +https://bugzilla.redhat.com/show_bug.cgi?id=1613277 + +Signed-off-by: Eduardo Habkost +Message-Id: <20180809221852.15285-1-ehabkost@redhat.com> +Tested-by: Richard W.M. Jones +Reviewed-by: Babu Moger +Signed-off-by: Eduardo Habkost +(cherry picked from commit 7210a02c58572b2686a3a8d610c6628f87864aed) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 605d0fa..e16dba7 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -852,6 +852,12 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { + }, + .cpuid_eax = 0x80000001, .cpuid_reg = R_ECX, + .tcg_features = TCG_EXT3_FEATURES, ++ /* ++ * TOPOEXT is always allowed but can't be enabled blindly by ++ * "-cpu host", as it requires consistent cache topology info ++ * to be provided so it doesn't confuse guests. ++ */ ++ .no_autoenable_flags = CPUID_EXT3_TOPOEXT, + }, + [FEAT_C000_0001_EDX] = { + .feat_names = { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Enable-TOPOEXT-feature-on-AMD-EPYC-CPU.patch b/SOURCES/kvm-i386-Enable-TOPOEXT-feature-on-AMD-EPYC-CPU.patch new file mode 100644 index 0000000..ca297b9 --- /dev/null +++ b/SOURCES/kvm-i386-Enable-TOPOEXT-feature-on-AMD-EPYC-CPU.patch @@ -0,0 +1,111 @@ +From 1fb1babff46264380702607ac92fc95d8adb90c1 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:19:02 +0100 +Subject: [PATCH 12/14] i386: Enable TOPOEXT feature on AMD EPYC CPU + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-10-ehabkost@redhat.com> +Patchwork-id: 81531 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 09/11] i386: Enable TOPOEXT feature on AMD EPYC CPU +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Enable TOPOEXT feature on EPYC CPU. This is required to support +hyperthreading on VM guests. Also extend xlevel to 0x8000001E. + +Disable topoext on PC_COMPAT_2_12 and keep xlevel 0x8000000a. + +Signed-off-by: Babu Moger +Message-Id: <1529443919-67509-3-git-send-email-babu.moger@amd.com> +[ehabkost: Added EPYC-IBPB.xlevel to PC_COMPAT_2_12] +Signed-off-by: Eduardo Habkost +(cherry picked from commit e00516475c270dcb6705753da96063f95699abf2) +[ehabkost: moved compat code to PC_RHEL75_COMPAT] +[ehabkost: fixed typo on EPYC-IBPB compat entry] +Signed-off-by: Eduardo Habkost +--- +Changes v2 -> v3: +* Removed duplicate TYPE_X86_CPU.legacy-cache=on entry from + PC_RHEL7_5_COMPAT (caught by Laurent Vivier) + +Changes v1 -> v2: +* Fixed typo on EPYC-IBPB compat entry + +Signed-off-by: Danilo C. L. de Paula +--- + include/hw/i386/pc.h | 12 ++++++++++++ + target/i386/cpu.c | 10 ++++++---- + 2 files changed, 18 insertions(+), 4 deletions(-) + +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index 5aebf6e..c33ddbb 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -983,6 +983,18 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .driver = TYPE_X86_CPU,\ + .property = "legacy-cache",\ + .value = "on",\ ++ },{ /* PC_RHEL7_5_COMPAT from PC_COMPAT_2_12 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "topoext",\ ++ .value = "off",\ ++ },{ /* PC_RHEL7_5_COMPAT from PC_COMPAT_2_12 */ \ ++ .driver = "EPYC-" TYPE_X86_CPU,\ ++ .property = "xlevel",\ ++ .value = stringify(0x8000000a),\ ++ },{ /* PC_RHEL7_5_COMPAT from PC_COMPAT_2_12 */ \ ++ .driver = "EPYC-IBPB-" TYPE_X86_CPU,\ ++ .property = "xlevel",\ ++ .value = stringify(0x8000000a),\ + }, + + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index cd0667f..fe2b2e8 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -2562,7 +2562,8 @@ static X86CPUDefinition builtin_x86_defs[] = { + .features[FEAT_8000_0001_ECX] = + CPUID_EXT3_OSVW | CPUID_EXT3_3DNOWPREFETCH | + CPUID_EXT3_MISALIGNSSE | CPUID_EXT3_SSE4A | CPUID_EXT3_ABM | +- CPUID_EXT3_CR8LEG | CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM, ++ CPUID_EXT3_CR8LEG | CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM | ++ CPUID_EXT3_TOPOEXT, + .features[FEAT_7_0_EBX] = + CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_BMI1 | CPUID_7_0_EBX_AVX2 | + CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_BMI2 | CPUID_7_0_EBX_RDSEED | +@@ -2577,7 +2578,7 @@ static X86CPUDefinition builtin_x86_defs[] = { + CPUID_XSAVE_XGETBV1, + .features[FEAT_6_EAX] = + CPUID_6_EAX_ARAT, +- .xlevel = 0x8000000A, ++ .xlevel = 0x8000001E, + .model_id = "AMD EPYC Processor", + .cache_info = &epyc_cache_info, + }, +@@ -2607,7 +2608,8 @@ static X86CPUDefinition builtin_x86_defs[] = { + .features[FEAT_8000_0001_ECX] = + CPUID_EXT3_OSVW | CPUID_EXT3_3DNOWPREFETCH | + CPUID_EXT3_MISALIGNSSE | CPUID_EXT3_SSE4A | CPUID_EXT3_ABM | +- CPUID_EXT3_CR8LEG | CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM, ++ CPUID_EXT3_CR8LEG | CPUID_EXT3_SVM | CPUID_EXT3_LAHF_LM | ++ CPUID_EXT3_TOPOEXT, + .features[FEAT_8000_0008_EBX] = + CPUID_8000_0008_EBX_IBPB, + .features[FEAT_7_0_EBX] = +@@ -2624,7 +2626,7 @@ static X86CPUDefinition builtin_x86_defs[] = { + CPUID_XSAVE_XGETBV1, + .features[FEAT_6_EAX] = + CPUID_6_EAX_ARAT, +- .xlevel = 0x8000000A, ++ .xlevel = 0x8000001E, + .model_id = "AMD EPYC Processor (with IBPB)", + .cache_info = &epyc_cache_info, + }, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Fix-arch_query_cpu_model_expansion-leak.patch b/SOURCES/kvm-i386-Fix-arch_query_cpu_model_expansion-leak.patch new file mode 100644 index 0000000..bb90d34 --- /dev/null +++ b/SOURCES/kvm-i386-Fix-arch_query_cpu_model_expansion-leak.patch @@ -0,0 +1,87 @@ +From 03f6fac8ac03d391fdbd7353ffd7c6eb1bd30bea Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Fri, 31 Aug 2018 13:59:22 +0100 +Subject: [PATCH 2/3] i386: Fix arch_query_cpu_model_expansion() leak + +RH-Author: Markus Armbruster +Message-id: <20180831135922.6073-3-armbru@redhat.com> +Patchwork-id: 81980 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 2/2] i386: Fix arch_query_cpu_model_expansion() leak +Bugzilla: 1615717 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Miroslav Rezanina + +From: Eduardo Habkost + +Reported by Coverity: + +Error: RESOURCE_LEAK (CWE-772): [#def439] +qemu-2.12.0/target/i386/cpu.c:3179: alloc_fn: Storage is returned from allocation function "qdict_new". +qemu-2.12.0/qobject/qdict.c:34:5: alloc_fn: Storage is returned from allocation function "g_malloc0". +qemu-2.12.0/qobject/qdict.c:34:5: var_assign: Assigning: "qdict" = "g_malloc0(4120UL)". +qemu-2.12.0/qobject/qdict.c:37:5: return_alloc: Returning allocated memory "qdict". +qemu-2.12.0/target/i386/cpu.c:3179: var_assign: Assigning: "props" = storage returned from "qdict_new()". +qemu-2.12.0/target/i386/cpu.c:3217: leaked_storage: Variable "props" going out of scope leaks the storage it points to. + +This was introduced by commit b8097deb359b ("i386: Improve +query-cpu-model-expansion full mode"). + +The leak is only theoretical: if ret->model->props is set to +props, the qapi_free_CpuModelExpansionInfo() call will free props +too in case of errors. The only way for this to not happen is if +we enter the default branch of the switch statement, which would +never happen because all CpuModelExpansionType values are being +handled. + +It's still worth to change this to make the allocation logic +easier to follow and make the Coverity error go away. To make +everything simpler, initialize ret->model and ret->model->props +earlier in the function. + +While at it, remove redundant check for !prop because prop is +always initialized at the beginning of the function. + +Fixes: b8097deb359bbbd92592b9670adfe9e245b2d0bd +Signed-off-by: Eduardo Habkost +Message-Id: <20180816183509.8231-1-ehabkost@redhat.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit e38bf612477fca62b205ebd909b1372a7e45a8c0) +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 9 +++------ + 1 file changed, 3 insertions(+), 6 deletions(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index caab4e2..605d0fa 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -3727,6 +3727,9 @@ arch_query_cpu_model_expansion(CpuModelExpansionType type, + } + + props = qdict_new(); ++ ret->model = g_new0(CpuModelInfo, 1); ++ ret->model->props = QOBJECT(props); ++ ret->model->has_props = true; + + switch (type) { + case CPU_MODEL_EXPANSION_TYPE_STATIC: +@@ -3747,15 +3750,9 @@ arch_query_cpu_model_expansion(CpuModelExpansionType type, + goto out; + } + +- if (!props) { +- props = qdict_new(); +- } + x86_cpu_to_dict(xc, props); + +- ret->model = g_new0(CpuModelInfo, 1); + ret->model->name = g_strdup(base_name); +- ret->model->props = QOBJECT(props); +- ret->model->has_props = true; + + out: + object_unref(OBJECT(xc)); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Fix-up-the-Node-id-for-CPUID_8000_001E.patch b/SOURCES/kvm-i386-Fix-up-the-Node-id-for-CPUID_8000_001E.patch new file mode 100644 index 0000000..46890c4 --- /dev/null +++ b/SOURCES/kvm-i386-Fix-up-the-Node-id-for-CPUID_8000_001E.patch @@ -0,0 +1,88 @@ +From b2f6405ed56e629f9999bead454a2d019bf77dc3 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:19:01 +0100 +Subject: [PATCH 11/14] i386: Fix up the Node id for CPUID_8000_001E + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-9-ehabkost@redhat.com> +Patchwork-id: 81530 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 08/11] i386: Fix up the Node id for CPUID_8000_001E +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +This is part of topoext support. To keep the compatibility, it is better +we support all the combination of nr_cores and nr_threads currently +supported. By allowing more nr_cores and nr_threads, we might end up with +more nodes than we can actually support with the real hardware. We need to +fix up the node id to make this work. We can achieve this by shifting the +socket_id bits left to address more nodes. + +Signed-off-by: Babu Moger +Message-Id: <1529443919-67509-2-git-send-email-babu.moger@amd.com> +Reviewed-by: Eduardo Habkost +Signed-off-by: Eduardo Habkost +(cherry picked from commit 631be32155dbafa1fe886f2488127956c9120ba6) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 26 +++++++++++++++++++++++++- + 1 file changed, 25 insertions(+), 1 deletion(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 3ed1e47..cd0667f 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -19,6 +19,7 @@ + + #include "qemu/osdep.h" + #include "qemu/cutils.h" ++#include "qemu/bitops.h" + + #include "cpu.h" + #include "exec/exec-all.h" +@@ -474,6 +475,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu, + uint32_t *ecx, uint32_t *edx) + { + struct core_topology topo = {0}; ++ unsigned long nodes; ++ int shift; + + build_core_topology(cs->nr_cores, cpu->core_id, &topo); + *eax = cpu->apic_id; +@@ -506,7 +509,28 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu, + * 2 Socket id + * 1:0 Node id + */ +- *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) | topo.node_id; ++ if (topo.num_nodes <= 4) { ++ *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) | ++ topo.node_id; ++ } else { ++ /* ++ * Node id fix up. Actual hardware supports up to 4 nodes. But with ++ * more than 32 cores, we may end up with more than 4 nodes. ++ * Node id is a combination of socket id and node id. Only requirement ++ * here is that this number should be unique accross the system. ++ * Shift the socket id to accommodate more nodes. We dont expect both ++ * socket id and node id to be big number at the same time. This is not ++ * an ideal config but we need to to support it. Max nodes we can have ++ * is 32 (255/8) with 8 cores per node and 255 max cores. We only need ++ * 5 bits for nodes. Find the left most set bit to represent the total ++ * number of nodes. find_last_bit returns last set bit(0 based). Left ++ * shift(+1) the socket id to represent all the nodes. ++ */ ++ nodes = topo.num_nodes - 1; ++ shift = find_last_bit(&nodes, 8); ++ *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) | ++ topo.node_id; ++ } + *edx = 0; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Helpers-to-encode-cache-information-consistentl.patch b/SOURCES/kvm-i386-Helpers-to-encode-cache-information-consistentl.patch new file mode 100644 index 0000000..8d77049 --- /dev/null +++ b/SOURCES/kvm-i386-Helpers-to-encode-cache-information-consistentl.patch @@ -0,0 +1,687 @@ +From 8959789a6c29612ba41f966c8ab6382dd944701e Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:18:54 +0100 +Subject: [PATCH 04/14] i386: Helpers to encode cache information consistently + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-2-ehabkost@redhat.com> +Patchwork-id: 81525 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 01/11] i386: Helpers to encode cache information consistently +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +Instead of having a collection of macros that need to be used in +complex expressions to build CPUID data, define a CPUCacheInfo +struct that can hold information about a given cache. Helper +functions will take a CPUCacheInfo struct as input to encode +CPUID leaves for a cache. + +This will help us ensure consistency between cache information +CPUID leaves, and make the existing inconsistencies in CPUID info +more visible. + +Signed-off-by: Eduardo Habkost +Signed-off-by: Babu Moger +Tested-by: Geoffrey McRae +Message-Id: <20180510204148.11687-2-babu.moger@amd.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit 7e3482f824809e1f6ffeb5bb8103ba27a7d1a52a) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 495 ++++++++++++++++++++++++++++++++++++++++-------------- + target/i386/cpu.h | 53 ++++++ + 2 files changed, 424 insertions(+), 124 deletions(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index a9db495..6c57b2f 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -56,33 +56,240 @@ + + #include "disas/capstone.h" + ++/* Helpers for building CPUID[2] descriptors: */ ++ ++struct CPUID2CacheDescriptorInfo { ++ enum CacheType type; ++ int level; ++ int size; ++ int line_size; ++ int associativity; ++}; + +-/* Cache topology CPUID constants: */ ++#define KiB 1024 ++#define MiB (1024 * 1024) + +-/* CPUID Leaf 2 Descriptors */ ++/* ++ * Known CPUID 2 cache descriptors. ++ * From Intel SDM Volume 2A, CPUID instruction ++ */ ++struct CPUID2CacheDescriptorInfo cpuid2_cache_descriptors[] = { ++ [0x06] = { .level = 1, .type = ICACHE, .size = 8 * KiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x08] = { .level = 1, .type = ICACHE, .size = 16 * KiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x09] = { .level = 1, .type = ICACHE, .size = 32 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x0A] = { .level = 1, .type = DCACHE, .size = 8 * KiB, ++ .associativity = 2, .line_size = 32, }, ++ [0x0C] = { .level = 1, .type = DCACHE, .size = 16 * KiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x0D] = { .level = 1, .type = DCACHE, .size = 16 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x0E] = { .level = 1, .type = DCACHE, .size = 24 * KiB, ++ .associativity = 6, .line_size = 64, }, ++ [0x1D] = { .level = 2, .type = UNIFIED_CACHE, .size = 128 * KiB, ++ .associativity = 2, .line_size = 64, }, ++ [0x21] = { .level = 2, .type = UNIFIED_CACHE, .size = 256 * KiB, ++ .associativity = 8, .line_size = 64, }, ++ /* lines per sector is not supported cpuid2_cache_descriptor(), ++ * so descriptors 0x22, 0x23 are not included ++ */ ++ [0x24] = { .level = 2, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 16, .line_size = 64, }, ++ /* lines per sector is not supported cpuid2_cache_descriptor(), ++ * so descriptors 0x25, 0x20 are not included ++ */ ++ [0x2C] = { .level = 1, .type = DCACHE, .size = 32 * KiB, ++ .associativity = 8, .line_size = 64, }, ++ [0x30] = { .level = 1, .type = ICACHE, .size = 32 * KiB, ++ .associativity = 8, .line_size = 64, }, ++ [0x41] = { .level = 2, .type = UNIFIED_CACHE, .size = 128 * KiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x42] = { .level = 2, .type = UNIFIED_CACHE, .size = 256 * KiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x43] = { .level = 2, .type = UNIFIED_CACHE, .size = 512 * KiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x44] = { .level = 2, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x45] = { .level = 2, .type = UNIFIED_CACHE, .size = 2 * MiB, ++ .associativity = 4, .line_size = 32, }, ++ [0x46] = { .level = 3, .type = UNIFIED_CACHE, .size = 4 * MiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x47] = { .level = 3, .type = UNIFIED_CACHE, .size = 8 * MiB, ++ .associativity = 8, .line_size = 64, }, ++ [0x48] = { .level = 2, .type = UNIFIED_CACHE, .size = 3 * MiB, ++ .associativity = 12, .line_size = 64, }, ++ /* Descriptor 0x49 depends on CPU family/model, so it is not included */ ++ [0x4A] = { .level = 3, .type = UNIFIED_CACHE, .size = 6 * MiB, ++ .associativity = 12, .line_size = 64, }, ++ [0x4B] = { .level = 3, .type = UNIFIED_CACHE, .size = 8 * MiB, ++ .associativity = 16, .line_size = 64, }, ++ [0x4C] = { .level = 3, .type = UNIFIED_CACHE, .size = 12 * MiB, ++ .associativity = 12, .line_size = 64, }, ++ [0x4D] = { .level = 3, .type = UNIFIED_CACHE, .size = 16 * MiB, ++ .associativity = 16, .line_size = 64, }, ++ [0x4E] = { .level = 2, .type = UNIFIED_CACHE, .size = 6 * MiB, ++ .associativity = 24, .line_size = 64, }, ++ [0x60] = { .level = 1, .type = DCACHE, .size = 16 * KiB, ++ .associativity = 8, .line_size = 64, }, ++ [0x66] = { .level = 1, .type = DCACHE, .size = 8 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x67] = { .level = 1, .type = DCACHE, .size = 16 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x68] = { .level = 1, .type = DCACHE, .size = 32 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x78] = { .level = 2, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 4, .line_size = 64, }, ++ /* lines per sector is not supported cpuid2_cache_descriptor(), ++ * so descriptors 0x79, 0x7A, 0x7B, 0x7C are not included. ++ */ ++ [0x7D] = { .level = 2, .type = UNIFIED_CACHE, .size = 2 * MiB, ++ .associativity = 8, .line_size = 64, }, ++ [0x7F] = { .level = 2, .type = UNIFIED_CACHE, .size = 512 * KiB, ++ .associativity = 2, .line_size = 64, }, ++ [0x80] = { .level = 2, .type = UNIFIED_CACHE, .size = 512 * KiB, ++ .associativity = 8, .line_size = 64, }, ++ [0x82] = { .level = 2, .type = UNIFIED_CACHE, .size = 256 * KiB, ++ .associativity = 8, .line_size = 32, }, ++ [0x83] = { .level = 2, .type = UNIFIED_CACHE, .size = 512 * KiB, ++ .associativity = 8, .line_size = 32, }, ++ [0x84] = { .level = 2, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 8, .line_size = 32, }, ++ [0x85] = { .level = 2, .type = UNIFIED_CACHE, .size = 2 * MiB, ++ .associativity = 8, .line_size = 32, }, ++ [0x86] = { .level = 2, .type = UNIFIED_CACHE, .size = 512 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0x87] = { .level = 2, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 8, .line_size = 64, }, ++ [0xD0] = { .level = 3, .type = UNIFIED_CACHE, .size = 512 * KiB, ++ .associativity = 4, .line_size = 64, }, ++ [0xD1] = { .level = 3, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 4, .line_size = 64, }, ++ [0xD2] = { .level = 3, .type = UNIFIED_CACHE, .size = 2 * MiB, ++ .associativity = 4, .line_size = 64, }, ++ [0xD6] = { .level = 3, .type = UNIFIED_CACHE, .size = 1 * MiB, ++ .associativity = 8, .line_size = 64, }, ++ [0xD7] = { .level = 3, .type = UNIFIED_CACHE, .size = 2 * MiB, ++ .associativity = 8, .line_size = 64, }, ++ [0xD8] = { .level = 3, .type = UNIFIED_CACHE, .size = 4 * MiB, ++ .associativity = 8, .line_size = 64, }, ++ [0xDC] = { .level = 3, .type = UNIFIED_CACHE, .size = 1.5 * MiB, ++ .associativity = 12, .line_size = 64, }, ++ [0xDD] = { .level = 3, .type = UNIFIED_CACHE, .size = 3 * MiB, ++ .associativity = 12, .line_size = 64, }, ++ [0xDE] = { .level = 3, .type = UNIFIED_CACHE, .size = 6 * MiB, ++ .associativity = 12, .line_size = 64, }, ++ [0xE2] = { .level = 3, .type = UNIFIED_CACHE, .size = 2 * MiB, ++ .associativity = 16, .line_size = 64, }, ++ [0xE3] = { .level = 3, .type = UNIFIED_CACHE, .size = 4 * MiB, ++ .associativity = 16, .line_size = 64, }, ++ [0xE4] = { .level = 3, .type = UNIFIED_CACHE, .size = 8 * MiB, ++ .associativity = 16, .line_size = 64, }, ++ [0xEA] = { .level = 3, .type = UNIFIED_CACHE, .size = 12 * MiB, ++ .associativity = 24, .line_size = 64, }, ++ [0xEB] = { .level = 3, .type = UNIFIED_CACHE, .size = 18 * MiB, ++ .associativity = 24, .line_size = 64, }, ++ [0xEC] = { .level = 3, .type = UNIFIED_CACHE, .size = 24 * MiB, ++ .associativity = 24, .line_size = 64, }, ++}; ++ ++/* ++ * "CPUID leaf 2 does not report cache descriptor information, ++ * use CPUID leaf 4 to query cache parameters" ++ */ ++#define CACHE_DESCRIPTOR_UNAVAILABLE 0xFF + +-#define CPUID_2_L1D_32KB_8WAY_64B 0x2c +-#define CPUID_2_L1I_32KB_8WAY_64B 0x30 +-#define CPUID_2_L2_2MB_8WAY_64B 0x7d +-#define CPUID_2_L3_16MB_16WAY_64B 0x4d ++/* ++ * Return a CPUID 2 cache descriptor for a given cache. ++ * If no known descriptor is found, return CACHE_DESCRIPTOR_UNAVAILABLE ++ */ ++static uint8_t cpuid2_cache_descriptor(CPUCacheInfo *cache) ++{ ++ int i; ++ ++ assert(cache->size > 0); ++ assert(cache->level > 0); ++ assert(cache->line_size > 0); ++ assert(cache->associativity > 0); ++ for (i = 0; i < ARRAY_SIZE(cpuid2_cache_descriptors); i++) { ++ struct CPUID2CacheDescriptorInfo *d = &cpuid2_cache_descriptors[i]; ++ if (d->level == cache->level && d->type == cache->type && ++ d->size == cache->size && d->line_size == cache->line_size && ++ d->associativity == cache->associativity) { ++ return i; ++ } ++ } + ++ return CACHE_DESCRIPTOR_UNAVAILABLE; ++} + + /* CPUID Leaf 4 constants: */ + + /* EAX: */ +-#define CPUID_4_TYPE_DCACHE 1 +-#define CPUID_4_TYPE_ICACHE 2 +-#define CPUID_4_TYPE_UNIFIED 3 ++#define CACHE_TYPE_D 1 ++#define CACHE_TYPE_I 2 ++#define CACHE_TYPE_UNIFIED 3 + +-#define CPUID_4_LEVEL(l) ((l) << 5) ++#define CACHE_LEVEL(l) (l << 5) + +-#define CPUID_4_SELF_INIT_LEVEL (1 << 8) +-#define CPUID_4_FULLY_ASSOC (1 << 9) ++#define CACHE_SELF_INIT_LEVEL (1 << 8) + + /* EDX: */ +-#define CPUID_4_NO_INVD_SHARING (1 << 0) +-#define CPUID_4_INCLUSIVE (1 << 1) +-#define CPUID_4_COMPLEX_IDX (1 << 2) ++#define CACHE_NO_INVD_SHARING (1 << 0) ++#define CACHE_INCLUSIVE (1 << 1) ++#define CACHE_COMPLEX_IDX (1 << 2) ++ ++/* Encode CacheType for CPUID[4].EAX */ ++#define CACHE_TYPE(t) (((t) == DCACHE) ? CACHE_TYPE_D : \ ++ ((t) == ICACHE) ? CACHE_TYPE_I : \ ++ ((t) == UNIFIED_CACHE) ? CACHE_TYPE_UNIFIED : \ ++ 0 /* Invalid value */) ++ ++ ++/* Encode cache info for CPUID[4] */ ++static void encode_cache_cpuid4(CPUCacheInfo *cache, ++ int num_apic_ids, int num_cores, ++ uint32_t *eax, uint32_t *ebx, ++ uint32_t *ecx, uint32_t *edx) ++{ ++ assert(cache->size == cache->line_size * cache->associativity * ++ cache->partitions * cache->sets); ++ ++ assert(num_apic_ids > 0); ++ *eax = CACHE_TYPE(cache->type) | ++ CACHE_LEVEL(cache->level) | ++ (cache->self_init ? CACHE_SELF_INIT_LEVEL : 0) | ++ ((num_cores - 1) << 26) | ++ ((num_apic_ids - 1) << 14); ++ ++ assert(cache->line_size > 0); ++ assert(cache->partitions > 0); ++ assert(cache->associativity > 0); ++ /* We don't implement fully-associative caches */ ++ assert(cache->associativity < cache->sets); ++ *ebx = (cache->line_size - 1) | ++ ((cache->partitions - 1) << 12) | ++ ((cache->associativity - 1) << 22); ++ ++ assert(cache->sets > 0); ++ *ecx = cache->sets - 1; ++ ++ *edx = (cache->no_invd_sharing ? CACHE_NO_INVD_SHARING : 0) | ++ (cache->inclusive ? CACHE_INCLUSIVE : 0) | ++ (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0); ++} ++ ++/* Encode cache info for CPUID[0x80000005].ECX or CPUID[0x80000005].EDX */ ++static uint32_t encode_cache_cpuid80000005(CPUCacheInfo *cache) ++{ ++ assert(cache->size % 1024 == 0); ++ assert(cache->lines_per_tag > 0); ++ assert(cache->associativity > 0); ++ assert(cache->line_size > 0); ++ return ((cache->size / 1024) << 24) | (cache->associativity << 16) | ++ (cache->lines_per_tag << 8) | (cache->line_size); ++} + + #define ASSOC_FULL 0xFF + +@@ -100,57 +307,140 @@ + a == ASSOC_FULL ? 0xF : \ + 0 /* invalid value */) + ++/* ++ * Encode cache info for CPUID[0x80000006].ECX and CPUID[0x80000006].EDX ++ * @l3 can be NULL. ++ */ ++static void encode_cache_cpuid80000006(CPUCacheInfo *l2, ++ CPUCacheInfo *l3, ++ uint32_t *ecx, uint32_t *edx) ++{ ++ assert(l2->size % 1024 == 0); ++ assert(l2->associativity > 0); ++ assert(l2->lines_per_tag > 0); ++ assert(l2->line_size > 0); ++ *ecx = ((l2->size / 1024) << 16) | ++ (AMD_ENC_ASSOC(l2->associativity) << 12) | ++ (l2->lines_per_tag << 8) | (l2->line_size); ++ ++ if (l3) { ++ assert(l3->size % (512 * 1024) == 0); ++ assert(l3->associativity > 0); ++ assert(l3->lines_per_tag > 0); ++ assert(l3->line_size > 0); ++ *edx = ((l3->size / (512 * 1024)) << 18) | ++ (AMD_ENC_ASSOC(l3->associativity) << 12) | ++ (l3->lines_per_tag << 8) | (l3->line_size); ++ } else { ++ *edx = 0; ++ } ++} + + /* Definitions of the hardcoded cache entries we expose: */ + + /* L1 data cache: */ +-#define L1D_LINE_SIZE 64 +-#define L1D_ASSOCIATIVITY 8 +-#define L1D_SETS 64 +-#define L1D_PARTITIONS 1 +-/* Size = LINE_SIZE*ASSOCIATIVITY*SETS*PARTITIONS = 32KiB */ +-#define L1D_DESCRIPTOR CPUID_2_L1D_32KB_8WAY_64B ++static CPUCacheInfo l1d_cache = { ++ .type = DCACHE, ++ .level = 1, ++ .size = 32 * KiB, ++ .self_init = 1, ++ .line_size = 64, ++ .associativity = 8, ++ .sets = 64, ++ .partitions = 1, ++ .no_invd_sharing = true, ++}; ++ + /*FIXME: CPUID leaf 0x80000005 is inconsistent with leaves 2 & 4 */ +-#define L1D_LINES_PER_TAG 1 +-#define L1D_SIZE_KB_AMD 64 +-#define L1D_ASSOCIATIVITY_AMD 2 ++static CPUCacheInfo l1d_cache_amd = { ++ .type = DCACHE, ++ .level = 1, ++ .size = 64 * KiB, ++ .self_init = 1, ++ .line_size = 64, ++ .associativity = 2, ++ .sets = 512, ++ .partitions = 1, ++ .lines_per_tag = 1, ++ .no_invd_sharing = true, ++}; + + /* L1 instruction cache: */ +-#define L1I_LINE_SIZE 64 +-#define L1I_ASSOCIATIVITY 8 +-#define L1I_SETS 64 +-#define L1I_PARTITIONS 1 +-/* Size = LINE_SIZE*ASSOCIATIVITY*SETS*PARTITIONS = 32KiB */ +-#define L1I_DESCRIPTOR CPUID_2_L1I_32KB_8WAY_64B ++static CPUCacheInfo l1i_cache = { ++ .type = ICACHE, ++ .level = 1, ++ .size = 32 * KiB, ++ .self_init = 1, ++ .line_size = 64, ++ .associativity = 8, ++ .sets = 64, ++ .partitions = 1, ++ .no_invd_sharing = true, ++}; ++ + /*FIXME: CPUID leaf 0x80000005 is inconsistent with leaves 2 & 4 */ +-#define L1I_LINES_PER_TAG 1 +-#define L1I_SIZE_KB_AMD 64 +-#define L1I_ASSOCIATIVITY_AMD 2 ++static CPUCacheInfo l1i_cache_amd = { ++ .type = ICACHE, ++ .level = 1, ++ .size = 64 * KiB, ++ .self_init = 1, ++ .line_size = 64, ++ .associativity = 2, ++ .sets = 512, ++ .partitions = 1, ++ .lines_per_tag = 1, ++ .no_invd_sharing = true, ++}; + + /* Level 2 unified cache: */ +-#define L2_LINE_SIZE 64 +-#define L2_ASSOCIATIVITY 16 +-#define L2_SETS 4096 +-#define L2_PARTITIONS 1 +-/* Size = LINE_SIZE*ASSOCIATIVITY*SETS*PARTITIONS = 4MiB */ ++static CPUCacheInfo l2_cache = { ++ .type = UNIFIED_CACHE, ++ .level = 2, ++ .size = 4 * MiB, ++ .self_init = 1, ++ .line_size = 64, ++ .associativity = 16, ++ .sets = 4096, ++ .partitions = 1, ++ .no_invd_sharing = true, ++}; ++ + /*FIXME: CPUID leaf 2 descriptor is inconsistent with CPUID leaf 4 */ +-#define L2_DESCRIPTOR CPUID_2_L2_2MB_8WAY_64B ++static CPUCacheInfo l2_cache_cpuid2 = { ++ .type = UNIFIED_CACHE, ++ .level = 2, ++ .size = 2 * MiB, ++ .line_size = 64, ++ .associativity = 8, ++}; ++ ++ + /*FIXME: CPUID leaf 0x80000006 is inconsistent with leaves 2 & 4 */ +-#define L2_LINES_PER_TAG 1 +-#define L2_SIZE_KB_AMD 512 ++static CPUCacheInfo l2_cache_amd = { ++ .type = UNIFIED_CACHE, ++ .level = 2, ++ .size = 512 * KiB, ++ .line_size = 64, ++ .lines_per_tag = 1, ++ .associativity = 16, ++ .sets = 512, ++ .partitions = 1, ++}; + + /* Level 3 unified cache: */ +-#define L3_SIZE_KB 0 /* disabled */ +-#define L3_ASSOCIATIVITY 0 /* disabled */ +-#define L3_LINES_PER_TAG 0 /* disabled */ +-#define L3_LINE_SIZE 0 /* disabled */ +-#define L3_N_LINE_SIZE 64 +-#define L3_N_ASSOCIATIVITY 16 +-#define L3_N_SETS 16384 +-#define L3_N_PARTITIONS 1 +-#define L3_N_DESCRIPTOR CPUID_2_L3_16MB_16WAY_64B +-#define L3_N_LINES_PER_TAG 1 +-#define L3_N_SIZE_KB_AMD 16384 ++static CPUCacheInfo l3_cache = { ++ .type = UNIFIED_CACHE, ++ .level = 3, ++ .size = 16 * MiB, ++ .line_size = 64, ++ .associativity = 16, ++ .sets = 16384, ++ .partitions = 1, ++ .lines_per_tag = 1, ++ .self_init = true, ++ .inclusive = true, ++ .complex_indexing = true, ++}; + + /* TLB definitions: */ + +@@ -3327,85 +3617,53 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + if (!cpu->enable_l3_cache) { + *ecx = 0; + } else { +- *ecx = L3_N_DESCRIPTOR; ++ *ecx = cpuid2_cache_descriptor(&l3_cache); + } +- *edx = (L1D_DESCRIPTOR << 16) | \ +- (L1I_DESCRIPTOR << 8) | \ +- (L2_DESCRIPTOR); ++ *edx = (cpuid2_cache_descriptor(&l1d_cache) << 16) | ++ (cpuid2_cache_descriptor(&l1i_cache) << 8) | ++ (cpuid2_cache_descriptor(&l2_cache_cpuid2)); + break; + case 4: + /* cache info: needed for Core compatibility */ + if (cpu->cache_info_passthrough) { + host_cpuid(index, count, eax, ebx, ecx, edx); ++ /* QEMU gives out its own APIC IDs, never pass down bits 31..26. */ + *eax &= ~0xFC000000; ++ if ((*eax & 31) && cs->nr_cores > 1) { ++ *eax |= (cs->nr_cores - 1) << 26; ++ } + } else { + *eax = 0; + switch (count) { + case 0: /* L1 dcache info */ +- *eax |= CPUID_4_TYPE_DCACHE | \ +- CPUID_4_LEVEL(1) | \ +- CPUID_4_SELF_INIT_LEVEL; +- *ebx = (L1D_LINE_SIZE - 1) | \ +- ((L1D_PARTITIONS - 1) << 12) | \ +- ((L1D_ASSOCIATIVITY - 1) << 22); +- *ecx = L1D_SETS - 1; +- *edx = CPUID_4_NO_INVD_SHARING; ++ encode_cache_cpuid4(&l1d_cache, ++ 1, cs->nr_cores, ++ eax, ebx, ecx, edx); + break; + case 1: /* L1 icache info */ +- *eax |= CPUID_4_TYPE_ICACHE | \ +- CPUID_4_LEVEL(1) | \ +- CPUID_4_SELF_INIT_LEVEL; +- *ebx = (L1I_LINE_SIZE - 1) | \ +- ((L1I_PARTITIONS - 1) << 12) | \ +- ((L1I_ASSOCIATIVITY - 1) << 22); +- *ecx = L1I_SETS - 1; +- *edx = CPUID_4_NO_INVD_SHARING; ++ encode_cache_cpuid4(&l1i_cache, ++ 1, cs->nr_cores, ++ eax, ebx, ecx, edx); + break; + case 2: /* L2 cache info */ +- *eax |= CPUID_4_TYPE_UNIFIED | \ +- CPUID_4_LEVEL(2) | \ +- CPUID_4_SELF_INIT_LEVEL; +- if (cs->nr_threads > 1) { +- *eax |= (cs->nr_threads - 1) << 14; +- } +- *ebx = (L2_LINE_SIZE - 1) | \ +- ((L2_PARTITIONS - 1) << 12) | \ +- ((L2_ASSOCIATIVITY - 1) << 22); +- *ecx = L2_SETS - 1; +- *edx = CPUID_4_NO_INVD_SHARING; ++ encode_cache_cpuid4(&l2_cache, ++ cs->nr_threads, cs->nr_cores, ++ eax, ebx, ecx, edx); + break; + case 3: /* L3 cache info */ +- if (!cpu->enable_l3_cache) { +- *eax = 0; +- *ebx = 0; +- *ecx = 0; +- *edx = 0; ++ pkg_offset = apicid_pkg_offset(cs->nr_cores, cs->nr_threads); ++ if (cpu->enable_l3_cache) { ++ encode_cache_cpuid4(&l3_cache, ++ (1 << pkg_offset), cs->nr_cores, ++ eax, ebx, ecx, edx); + break; + } +- *eax |= CPUID_4_TYPE_UNIFIED | \ +- CPUID_4_LEVEL(3) | \ +- CPUID_4_SELF_INIT_LEVEL; +- pkg_offset = apicid_pkg_offset(cs->nr_cores, cs->nr_threads); +- *eax |= ((1 << pkg_offset) - 1) << 14; +- *ebx = (L3_N_LINE_SIZE - 1) | \ +- ((L3_N_PARTITIONS - 1) << 12) | \ +- ((L3_N_ASSOCIATIVITY - 1) << 22); +- *ecx = L3_N_SETS - 1; +- *edx = CPUID_4_INCLUSIVE | CPUID_4_COMPLEX_IDX; +- break; ++ /* fall through */ + default: /* end of info */ +- *eax = 0; +- *ebx = 0; +- *ecx = 0; +- *edx = 0; ++ *eax = *ebx = *ecx = *edx = 0; + break; + } + } +- +- /* QEMU gives out its own APIC IDs, never pass down bits 31..26. */ +- if ((*eax & 31) && cs->nr_cores > 1) { +- *eax |= (cs->nr_cores - 1) << 26; +- } + break; + case 5: + /* mwait info: needed for Core compatibility */ +@@ -3609,10 +3867,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + (L1_ITLB_2M_ASSOC << 8) | (L1_ITLB_2M_ENTRIES); + *ebx = (L1_DTLB_4K_ASSOC << 24) | (L1_DTLB_4K_ENTRIES << 16) | \ + (L1_ITLB_4K_ASSOC << 8) | (L1_ITLB_4K_ENTRIES); +- *ecx = (L1D_SIZE_KB_AMD << 24) | (L1D_ASSOCIATIVITY_AMD << 16) | \ +- (L1D_LINES_PER_TAG << 8) | (L1D_LINE_SIZE); +- *edx = (L1I_SIZE_KB_AMD << 24) | (L1I_ASSOCIATIVITY_AMD << 16) | \ +- (L1I_LINES_PER_TAG << 8) | (L1I_LINE_SIZE); ++ *ecx = encode_cache_cpuid80000005(&l1d_cache_amd); ++ *edx = encode_cache_cpuid80000005(&l1i_cache_amd); + break; + case 0x80000006: + /* cache info (L2 cache) */ +@@ -3628,18 +3884,9 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + (L2_DTLB_4K_ENTRIES << 16) | \ + (AMD_ENC_ASSOC(L2_ITLB_4K_ASSOC) << 12) | \ + (L2_ITLB_4K_ENTRIES); +- *ecx = (L2_SIZE_KB_AMD << 16) | \ +- (AMD_ENC_ASSOC(L2_ASSOCIATIVITY) << 12) | \ +- (L2_LINES_PER_TAG << 8) | (L2_LINE_SIZE); +- if (!cpu->enable_l3_cache) { +- *edx = ((L3_SIZE_KB / 512) << 18) | \ +- (AMD_ENC_ASSOC(L3_ASSOCIATIVITY) << 12) | \ +- (L3_LINES_PER_TAG << 8) | (L3_LINE_SIZE); +- } else { +- *edx = ((L3_N_SIZE_KB_AMD / 512) << 18) | \ +- (AMD_ENC_ASSOC(L3_N_ASSOCIATIVITY) << 12) | \ +- (L3_N_LINES_PER_TAG << 8) | (L3_N_LINE_SIZE); +- } ++ encode_cache_cpuid80000006(&l2_cache_amd, ++ cpu->enable_l3_cache ? &l3_cache : NULL, ++ ecx, edx); + break; + case 0x80000007: + *eax = 0; +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index 1b219fa..fa03e2c 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -1044,6 +1044,59 @@ typedef enum TPRAccess { + TPR_ACCESS_WRITE, + } TPRAccess; + ++/* Cache information data structures: */ ++ ++enum CacheType { ++ DCACHE, ++ ICACHE, ++ UNIFIED_CACHE ++}; ++ ++typedef struct CPUCacheInfo { ++ enum CacheType type; ++ uint8_t level; ++ /* Size in bytes */ ++ uint32_t size; ++ /* Line size, in bytes */ ++ uint16_t line_size; ++ /* ++ * Associativity. ++ * Note: representation of fully-associative caches is not implemented ++ */ ++ uint8_t associativity; ++ /* Physical line partitions. CPUID[0x8000001D].EBX, CPUID[4].EBX */ ++ uint8_t partitions; ++ /* Number of sets. CPUID[0x8000001D].ECX, CPUID[4].ECX */ ++ uint32_t sets; ++ /* ++ * Lines per tag. ++ * AMD-specific: CPUID[0x80000005], CPUID[0x80000006]. ++ * (Is this synonym to @partitions?) ++ */ ++ uint8_t lines_per_tag; ++ ++ /* Self-initializing cache */ ++ bool self_init; ++ /* ++ * WBINVD/INVD is not guaranteed to act upon lower level caches of ++ * non-originating threads sharing this cache. ++ * CPUID[4].EDX[bit 0], CPUID[0x8000001D].EDX[bit 0] ++ */ ++ bool no_invd_sharing; ++ /* ++ * Cache is inclusive of lower cache levels. ++ * CPUID[4].EDX[bit 1], CPUID[0x8000001D].EDX[bit 1]. ++ */ ++ bool inclusive; ++ /* ++ * A complex function is used to index the cache, potentially using all ++ * address bits. CPUID[4].EDX[bit 2]. ++ */ ++ bool complex_indexing; ++} CPUCacheInfo; ++ ++ ++ + typedef struct CPUX86State { + /* standard registers */ + target_ulong regs[CPU_NB_REGS]; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Initialize-cache-information-for-EPYC-family-pr.patch b/SOURCES/kvm-i386-Initialize-cache-information-for-EPYC-family-pr.patch new file mode 100644 index 0000000..396184f --- /dev/null +++ b/SOURCES/kvm-i386-Initialize-cache-information-for-EPYC-family-pr.patch @@ -0,0 +1,110 @@ +From 15a2a3aa1c4eb2c3abac569a120714425b64864d Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:18:56 +0100 +Subject: [PATCH 06/14] i386: Initialize cache information for EPYC family + processors + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-4-ehabkost@redhat.com> +Patchwork-id: 81527 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 03/11] i386: Initialize cache information for EPYC family processors +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Initialize pre-determined cache information for EPYC processors. + +Signed-off-by: Babu Moger +Tested-by: Geoffrey McRae +Message-Id: <20180510204148.11687-5-babu.moger@amd.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit fe52acd2a054b97765963a42037f2f886545e30c) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 52 insertions(+) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 50af741..bd0abc2 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -1108,6 +1108,56 @@ struct X86CPUDefinition { + CPUCaches *cache_info; + }; + ++static CPUCaches epyc_cache_info = { ++ .l1d_cache = { ++ .type = DCACHE, ++ .level = 1, ++ .size = 32 * KiB, ++ .line_size = 64, ++ .associativity = 8, ++ .partitions = 1, ++ .sets = 64, ++ .lines_per_tag = 1, ++ .self_init = 1, ++ .no_invd_sharing = true, ++ }, ++ .l1i_cache = { ++ .type = ICACHE, ++ .level = 1, ++ .size = 64 * KiB, ++ .line_size = 64, ++ .associativity = 4, ++ .partitions = 1, ++ .sets = 256, ++ .lines_per_tag = 1, ++ .self_init = 1, ++ .no_invd_sharing = true, ++ }, ++ .l2_cache = { ++ .type = UNIFIED_CACHE, ++ .level = 2, ++ .size = 512 * KiB, ++ .line_size = 64, ++ .associativity = 8, ++ .partitions = 1, ++ .sets = 1024, ++ .lines_per_tag = 1, ++ }, ++ .l3_cache = { ++ .type = UNIFIED_CACHE, ++ .level = 3, ++ .size = 8 * MiB, ++ .line_size = 64, ++ .associativity = 16, ++ .partitions = 1, ++ .sets = 8192, ++ .lines_per_tag = 1, ++ .self_init = true, ++ .inclusive = true, ++ .complex_indexing = true, ++ }, ++}; ++ + static X86CPUDefinition builtin_x86_defs[] = { + { + /* qemu64 is the default CPU model for all *-rhel7.* machine-types. +@@ -2327,6 +2377,7 @@ static X86CPUDefinition builtin_x86_defs[] = { + CPUID_6_EAX_ARAT, + .xlevel = 0x8000000A, + .model_id = "AMD EPYC Processor", ++ .cache_info = &epyc_cache_info, + }, + { + .name = "EPYC-IBPB", +@@ -2373,6 +2424,7 @@ static X86CPUDefinition builtin_x86_defs[] = { + CPUID_6_EAX_ARAT, + .xlevel = 0x8000000A, + .model_id = "AMD EPYC Processor (with IBPB)", ++ .cache_info = &epyc_cache_info, + }, + }; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Populate-AMD-Processor-Cache-Information-for-cp.patch b/SOURCES/kvm-i386-Populate-AMD-Processor-Cache-Information-for-cp.patch new file mode 100644 index 0000000..882fe56 --- /dev/null +++ b/SOURCES/kvm-i386-Populate-AMD-Processor-Cache-Information-for-cp.patch @@ -0,0 +1,212 @@ +From 9308fcbb870779d4d52f0497dbf68d2651b8895b Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:18:59 +0100 +Subject: [PATCH 09/14] i386: Populate AMD Processor Cache Information for + cpuid 0x8000001D + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-7-ehabkost@redhat.com> +Patchwork-id: 81529 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 06/11] i386: Populate AMD Processor Cache Information for cpuid 0x8000001D +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Add information for cpuid 0x8000001D leaf. Populate cache topology information +for different cache types (Data Cache, Instruction Cache, L2 and L3) supported +by 0x8000001D leaf. Please refer to the Processor Programming Reference (PPR) +for AMD Family 17h Model for more details. + +Signed-off-by: Babu Moger +Message-Id: <1527176614-26271-3-git-send-email-babu.moger@amd.com> +Reviewed-by: Eduardo Habkost +Signed-off-by: Eduardo Habkost +(cherry picked from commit 8f4202fb1080f86958782b1fca0bf0279f67d136) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 117 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ + target/i386/kvm.c | 29 ++++++++++++-- + 2 files changed, 143 insertions(+), 3 deletions(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index f98a964..d2474d7 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -337,6 +337,99 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2, + } + + /* ++ * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E ++ * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3. ++ * Define the constants to build the cpu topology. Right now, TOPOEXT ++ * feature is enabled only on EPYC. So, these constants are based on ++ * EPYC supported configurations. We may need to handle the cases if ++ * these values change in future. ++ */ ++/* Maximum core complexes in a node */ ++#define MAX_CCX 2 ++/* Maximum cores in a core complex */ ++#define MAX_CORES_IN_CCX 4 ++/* Maximum cores in a node */ ++#define MAX_CORES_IN_NODE 8 ++/* Maximum nodes in a socket */ ++#define MAX_NODES_PER_SOCKET 4 ++ ++/* ++ * Figure out the number of nodes required to build this config. ++ * Max cores in a node is 8 ++ */ ++static int nodes_in_socket(int nr_cores) ++{ ++ int nodes; ++ ++ nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE); ++ ++ /* Hardware does not support config with 3 nodes, return 4 in that case */ ++ return (nodes == 3) ? 4 : nodes; ++} ++ ++/* ++ * Decide the number of cores in a core complex with the given nr_cores using ++ * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and ++ * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible ++ * L3 cache is shared across all cores in a core complex. So, this will also ++ * tell us how many cores are sharing the L3 cache. ++ */ ++static int cores_in_core_complex(int nr_cores) ++{ ++ int nodes; ++ ++ /* Check if we can fit all the cores in one core complex */ ++ if (nr_cores <= MAX_CORES_IN_CCX) { ++ return nr_cores; ++ } ++ /* Get the number of nodes required to build this config */ ++ nodes = nodes_in_socket(nr_cores); ++ ++ /* ++ * Divide the cores accros all the core complexes ++ * Return rounded up value ++ */ ++ return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX); ++} ++ ++/* Encode cache info for CPUID[8000001D] */ ++static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs, ++ uint32_t *eax, uint32_t *ebx, ++ uint32_t *ecx, uint32_t *edx) ++{ ++ uint32_t l3_cores; ++ assert(cache->size == cache->line_size * cache->associativity * ++ cache->partitions * cache->sets); ++ ++ *eax = CACHE_TYPE(cache->type) | CACHE_LEVEL(cache->level) | ++ (cache->self_init ? CACHE_SELF_INIT_LEVEL : 0); ++ ++ /* L3 is shared among multiple cores */ ++ if (cache->level == 3) { ++ l3_cores = cores_in_core_complex(cs->nr_cores); ++ *eax |= ((l3_cores * cs->nr_threads) - 1) << 14; ++ } else { ++ *eax |= ((cs->nr_threads - 1) << 14); ++ } ++ ++ assert(cache->line_size > 0); ++ assert(cache->partitions > 0); ++ assert(cache->associativity > 0); ++ /* We don't implement fully-associative caches */ ++ assert(cache->associativity < cache->sets); ++ *ebx = (cache->line_size - 1) | ++ ((cache->partitions - 1) << 12) | ++ ((cache->associativity - 1) << 22); ++ ++ assert(cache->sets > 0); ++ *ecx = cache->sets - 1; ++ ++ *edx = (cache->no_invd_sharing ? CACHE_NO_INVD_SHARING : 0) | ++ (cache->inclusive ? CACHE_INCLUSIVE : 0) | ++ (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0); ++} ++ ++/* + * Definitions of the hardcoded cache entries we expose: + * These are legacy cache values. If there is a need to change any + * of these values please use builtin_x86_defs +@@ -3988,6 +4081,30 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, + *edx = 0; + } + break; ++ case 0x8000001D: ++ *eax = 0; ++ switch (count) { ++ case 0: /* L1 dcache info */ ++ encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs, ++ eax, ebx, ecx, edx); ++ break; ++ case 1: /* L1 icache info */ ++ encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs, ++ eax, ebx, ecx, edx); ++ break; ++ case 2: /* L2 cache info */ ++ encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs, ++ eax, ebx, ecx, edx); ++ break; ++ case 3: /* L3 cache info */ ++ encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs, ++ eax, ebx, ecx, edx); ++ break; ++ default: /* end of info */ ++ *eax = *ebx = *ecx = *edx = 0; ++ break; ++ } ++ break; + case 0xC0000000: + *eax = env->cpuid_xlevel2; + *ebx = 0; +diff --git a/target/i386/kvm.c b/target/i386/kvm.c +index 6c49954..6e66f9c 100644 +--- a/target/i386/kvm.c ++++ b/target/i386/kvm.c +@@ -967,9 +967,32 @@ int kvm_arch_init_vcpu(CPUState *cs) + } + c = &cpuid_data.entries[cpuid_i++]; + +- c->function = i; +- c->flags = 0; +- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx); ++ switch (i) { ++ case 0x8000001d: ++ /* Query for all AMD cache information leaves */ ++ for (j = 0; ; j++) { ++ c->function = i; ++ c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX; ++ c->index = j; ++ cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx); ++ ++ if (c->eax == 0) { ++ break; ++ } ++ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) { ++ fprintf(stderr, "cpuid_data is full, no space for " ++ "cpuid(eax:0x%x,ecx:0x%x)\n", i, j); ++ abort(); ++ } ++ c = &cpuid_data.entries[cpuid_i++]; ++ } ++ break; ++ default: ++ c->function = i; ++ c->flags = 0; ++ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx); ++ break; ++ } + } + + /* Call Centaur's CPUID instructions they are supported. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-Remove-generic-SMT-thread-check.patch b/SOURCES/kvm-i386-Remove-generic-SMT-thread-check.patch new file mode 100644 index 0000000..f439589 --- /dev/null +++ b/SOURCES/kvm-i386-Remove-generic-SMT-thread-check.patch @@ -0,0 +1,69 @@ +From e6f541bd5eea74185e34e41e628d52cfc5ab7b6a Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 26 Jul 2018 17:19:03 +0100 +Subject: [PATCH 13/14] i386: Remove generic SMT thread check + +RH-Author: Eduardo Habkost +Message-id: <20180726171904.27418-11-ehabkost@redhat.com> +Patchwork-id: 81532 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH v2 10/11] i386: Remove generic SMT thread check +Bugzilla: 1597739 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Laurent Vivier +RH-Acked-by: Igor Mammedov + +From: Babu Moger + +Remove generic non-intel check while validating hyperthreading support. +Certain AMD CPUs can support hyperthreading now. + +CPU family with TOPOEXT feature can support hyperthreading now. + +Signed-off-by: Babu Moger +Tested-by: Geoffrey McRae +Reviewed-by: Eduardo Habkost +Message-Id: <1529443919-67509-4-git-send-email-babu.moger@amd.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit 6b2942f966d5e54c37d305c80f5f98d504c2bc55) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 17 +++++++++++------ + 1 file changed, 11 insertions(+), 6 deletions(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index fe2b2e8..8647109 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -4952,17 +4952,22 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp) + + qemu_init_vcpu(cs); + +- /* Only Intel CPUs support hyperthreading. Even though QEMU fixes this +- * issue by adjusting CPUID_0000_0001_EBX and CPUID_8000_0008_ECX +- * based on inputs (sockets,cores,threads), it is still better to gives ++ /* ++ * Most Intel and certain AMD CPUs support hyperthreading. Even though QEMU ++ * fixes this issue by adjusting CPUID_0000_0001_EBX and CPUID_8000_0008_ECX ++ * based on inputs (sockets,cores,threads), it is still better to give + * users a warning. + * + * NOTE: the following code has to follow qemu_init_vcpu(). Otherwise + * cs->nr_threads hasn't be populated yet and the checking is incorrect. + */ +- if (!IS_INTEL_CPU(env) && cs->nr_threads > 1 && !ht_warned) { +- error_report("AMD CPU doesn't support hyperthreading. Please configure" +- " -smp options properly."); ++ if (IS_AMD_CPU(env) && ++ !(env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_TOPOEXT) && ++ cs->nr_threads > 1 && !ht_warned) { ++ error_report("This family of AMD CPU doesn't support " ++ "hyperthreading(%d). Please configure -smp " ++ "options properly or try enabling topoext feature.", ++ cs->nr_threads); + ht_warned = true; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-define-the-AMD-virt-ssbd-CPUID-feature-bit-CVE-.patch b/SOURCES/kvm-i386-define-the-AMD-virt-ssbd-CPUID-feature-bit-CVE-.patch new file mode 100644 index 0000000..06b9349 --- /dev/null +++ b/SOURCES/kvm-i386-define-the-AMD-virt-ssbd-CPUID-feature-bit-CVE-.patch @@ -0,0 +1,54 @@ +From 7085d8ac9d907fbfddb2c2c432f239939aa2aa2f Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Wed, 13 Jun 2018 18:08:12 +0200 +Subject: [PATCH 153/268] i386: define the AMD 'virt-ssbd' CPUID feature bit + (CVE-2018-3639) +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Habkost +Message-id: <20180613180812.28169-3-ehabkost@redhat.com> +Patchwork-id: 80676 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/2] i386: define the AMD 'virt-ssbd' CPUID feature bit (CVE-2018-3639) +Bugzilla: 1574216 +RH-Acked-by: Igor Mammedov +RH-Acked-by: Miroslav Rezanina +RH-Acked-by: Laurent Vivier + +From: Konrad Rzeszutek Wilk + +AMD Zen expose the Intel equivalant to Speculative Store Bypass Disable +via the 0x80000008_EBX[25] CPUID feature bit. + +This needs to be exposed to guest OS to allow them to protect +against CVE-2018-3639. + +Signed-off-by: Konrad Rzeszutek Wilk +Reviewed-by: Daniel P. Berrangé +Signed-off-by: Daniel P. Berrangé +Message-Id: <20180521215424.13520-3-berrange@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit 403503b162ffc33fb64cfefdf7b880acf41772cd) +Signed-off-by: Eduardo Habkost +Signed-off-by: Miroslav Rezanina +--- + target/i386/cpu.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 8647109..caab4e2 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -1033,7 +1033,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { + "ibpb", NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, +- NULL, NULL, NULL, NULL, ++ NULL, "virt-ssbd", NULL, NULL, + NULL, NULL, NULL, NULL, + }, + .cpuid_eax = 0x80000008, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-define-the-ssbd-CPUID-feature-bit-CVE-2018-3639.patch b/SOURCES/kvm-i386-define-the-ssbd-CPUID-feature-bit-CVE-2018-3639.patch new file mode 100644 index 0000000..5f9e1e2 --- /dev/null +++ b/SOURCES/kvm-i386-define-the-ssbd-CPUID-feature-bit-CVE-2018-3639.patch @@ -0,0 +1,65 @@ +From 8c311241e277756db896fdab983c6250ffde5fc0 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Sat, 13 Oct 2018 03:32:30 +0100 +Subject: [PATCH 17/17] i386: define the 'ssbd' CPUID feature bit + (CVE-2018-3639) +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Habkost +Message-id: <20181013033230.14687-2-ehabkost@redhat.com> +Patchwork-id: 82685 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] i386: define the 'ssbd' CPUID feature bit (CVE-2018-3639) +Bugzilla: 1633928 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Igor Mammedov +RH-Acked-by: Daniel P. Berrange + +From: Daniel P. Berrangé + +New microcode introduces the "Speculative Store Bypass Disable" +CPUID feature bit. This needs to be exposed to guest OS to allow +them to protect against CVE-2018-3639. + +Signed-off-by: Daniel P. Berrangé +Reviewed-by: Konrad Rzeszutek Wilk +Signed-off-by: Konrad Rzeszutek Wilk +Message-Id: <20180521215424.13520-2-berrange@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit d19d1f965904a533998739698020ff4ee8a103da) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 2 +- + target/i386/cpu.h | 1 + + 2 files changed, 2 insertions(+), 1 deletion(-) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index 0215b20..228935f 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -1008,7 +1008,7 @@ static FeatureWordInfo feature_word_info[FEATURE_WORDS] = { + NULL, NULL, NULL, NULL, + NULL, NULL, NULL, NULL, + NULL, NULL, "spec-ctrl", NULL, +- NULL, NULL, NULL, NULL, ++ NULL, NULL, NULL, "ssbd", + }, + .cpuid_eax = 7, + .cpuid_needs_ecx = true, .cpuid_ecx = 0, +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index c47db96..4a3ef4b 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -685,6 +685,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS]; + #define CPUID_7_0_EDX_AVX512_4VNNIW (1U << 2) /* AVX512 Neural Network Instructions */ + #define CPUID_7_0_EDX_AVX512_4FMAPS (1U << 3) /* AVX512 Multiply Accumulation Single Precision */ + #define CPUID_7_0_EDX_SPEC_CTRL (1U << 26) /* Speculation Control */ ++#define CPUID_7_0_EDX_SPEC_CTRL_SSBD (1U << 31) /* Speculative Store Bypass Disable */ + + #define KVM_HINTS_DEDICATED (1U << 0) + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-i386-do-not-migrate-MSR_SMI_COUNT-on-machine-types-2.patch b/SOURCES/kvm-i386-do-not-migrate-MSR_SMI_COUNT-on-machine-types-2.patch new file mode 100644 index 0000000..95b8e7d --- /dev/null +++ b/SOURCES/kvm-i386-do-not-migrate-MSR_SMI_COUNT-on-machine-types-2.patch @@ -0,0 +1,86 @@ +From d625effaaebcc744531947209db5a4ec5aca0b0a Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Fri, 14 Dec 2018 18:20:55 +0000 +Subject: [PATCH 3/5] i386: do not migrate MSR_SMI_COUNT on machine types <2.12 + +RH-Author: Dr. David Alan Gilbert +Message-id: <20181214182056.20233-2-dgilbert@redhat.com> +Patchwork-id: 83520 +O-Subject: [RHEL8 qemu-kvm PATCH 1/2] i386: do not migrate MSR_SMI_COUNT on machine types <2.12 +Bugzilla: 1659565 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Paolo Bonzini + +From: Paolo Bonzini + +MSR_SMI_COUNT started being migrated in QEMU 2.12. Do not migrate it +on older machine types, or the subsection causes a load failure for +guests that use SMM. + +Signed-off-by: Paolo Bonzini +(cherry picked from commit 990e0be2603511560168e1ad61f68294d951c39e) +Signed-off-by: Danilo C. L. de Paula +--- + include/hw/i386/pc.h | 4 ++++ + target/i386/cpu.c | 2 ++ + target/i386/cpu.h | 1 + + target/i386/machine.c | 2 +- + 4 files changed, 8 insertions(+), 1 deletion(-) + +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index c29176d..dd473ca 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -311,6 +311,10 @@ bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *); + #define PC_COMPAT_2_11 \ + HW_COMPAT_2_11 \ + {\ ++ .driver = TYPE_X86_CPU,\ ++ .property = "x-migrate-smi-count",\ ++ .value = "off",\ ++ },{\ + .driver = "Skylake-Server" "-" TYPE_X86_CPU,\ + .property = "clflushopt",\ + .value = "off",\ +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index c37cd1e..c979feb 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -5398,6 +5398,8 @@ static Property x86_cpu_properties[] = { + false), + DEFINE_PROP_BOOL("vmware-cpuid-freq", X86CPU, vmware_cpuid_freq, true), + DEFINE_PROP_BOOL("tcg-cpuid", X86CPU, expose_tcg, true), ++ DEFINE_PROP_BOOL("x-migrate-smi-count", X86CPU, migrate_smi_count, ++ true), + /* + * lecacy_cache defaults to true unless the CPU model provides its + * own cache information (see x86_cpu_load_def()). +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index 58d5430..fb6caf4 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -1369,6 +1369,7 @@ struct X86CPU { + bool expose_kvm; + bool expose_tcg; + bool migratable; ++ bool migrate_smi_count; + bool max_features; /* Enable all supported features automatically */ + uint32_t apic_id; + +diff --git a/target/i386/machine.c b/target/i386/machine.c +index 5e9a19b..2a85c91 100644 +--- a/target/i386/machine.c ++++ b/target/i386/machine.c +@@ -400,7 +400,7 @@ static bool msr_smi_count_needed(void *opaque) + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + +- return env->msr_smi_count != 0; ++ return cpu->migrate_smi_count && env->msr_smi_count != 0; + } + + static const VMStateDescription vmstate_msr_smi_count = { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-include-Add-IEC-binary-prefixes-in-qemu-units.h.patch b/SOURCES/kvm-include-Add-IEC-binary-prefixes-in-qemu-units.h.patch new file mode 100644 index 0000000..1415a13 --- /dev/null +++ b/SOURCES/kvm-include-Add-IEC-binary-prefixes-in-qemu-units.h.patch @@ -0,0 +1,62 @@ +From 5ab34ef15de3097ec4bcdb7ed369b8810a4e2d90 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:29 +0000 +Subject: [PATCH 04/15] include: Add IEC binary prefixes in "qemu/units.h" +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-5-kwolf@redhat.com> +Patchwork-id: 83287 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 04/15] include: Add IEC binary prefixes in "qemu/units.h" +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Philippe Mathieu-Daudé + +Loosely based on 076b35b5a56. + +Suggested-by: Stefan Weil +Signed-off-by: Philippe Mathieu-Daudé +Message-Id: <20180625124238.25339-2-f4bug@amsat.org> +Signed-off-by: Paolo Bonzini +(cherry picked from commit 7ecdc94c40f4958a66893c0eac423c6a80f376d4) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/units.h | 20 ++++++++++++++++++++ + 1 file changed, 20 insertions(+) + create mode 100644 include/qemu/units.h + +diff --git a/include/qemu/units.h b/include/qemu/units.h +new file mode 100644 +index 0000000..692db3f +--- /dev/null ++++ b/include/qemu/units.h +@@ -0,0 +1,20 @@ ++/* ++ * IEC binary prefixes definitions ++ * ++ * Copyright (C) 2015 Nikunj A Dadhania, IBM Corporation ++ * Copyright (C) 2018 Philippe Mathieu-Daudé ++ * ++ * SPDX-License-Identifier: GPL-2.0-or-later ++ */ ++ ++#ifndef QEMU_UNITS_H ++#define QEMU_UNITS_H ++ ++#define KiB (INT64_C(1) << 10) ++#define MiB (INT64_C(1) << 20) ++#define GiB (INT64_C(1) << 30) ++#define TiB (INT64_C(1) << 40) ++#define PiB (INT64_C(1) << 50) ++#define EiB (INT64_C(1) << 60) ++ ++#endif +-- +1.8.3.1 + diff --git a/SOURCES/kvm-include-Add-a-lookup-table-of-sizes.patch b/SOURCES/kvm-include-Add-a-lookup-table-of-sizes.patch new file mode 100644 index 0000000..004d31d --- /dev/null +++ b/SOURCES/kvm-include-Add-a-lookup-table-of-sizes.patch @@ -0,0 +1,113 @@ +From ac9ce8475684e9ed0670a7ac798e57ca2c971e54 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:31 +0000 +Subject: [PATCH 06/15] include: Add a lookup table of sizes + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-7-kwolf@redhat.com> +Patchwork-id: 83294 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 06/15] include: Add a lookup table of sizes +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +Adding a lookup table for the powers of two, with the appropriate size +prefixes. This is needed when a size has to be stringified, in which +case something like '(1 * KiB)' would become a literal '(1 * (1L << 10))' +string. Powers of two are used very often for sizes, so such a table +will also make it easier and more intuitive to write them. + +This table is generatred using the following AWK script: + +BEGIN { + suffix="KMGTPE"; + for(i=10; i<64; i++) { + val=2**i; + s=substr(suffix, int(i/10), 1); + n=2**(i%10); + pad=21-int(log(n)/log(10)); + printf("#define S_%d%siB %*d\n", n, s, pad, val); + } +} + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 540b8492618ebbe98e7462bd7d31361b8cb10a05) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/units.h | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 55 insertions(+) + +diff --git a/include/qemu/units.h b/include/qemu/units.h +index 692db3f..68a7758 100644 +--- a/include/qemu/units.h ++++ b/include/qemu/units.h +@@ -17,4 +17,59 @@ + #define PiB (INT64_C(1) << 50) + #define EiB (INT64_C(1) << 60) + ++#define S_1KiB 1024 ++#define S_2KiB 2048 ++#define S_4KiB 4096 ++#define S_8KiB 8192 ++#define S_16KiB 16384 ++#define S_32KiB 32768 ++#define S_64KiB 65536 ++#define S_128KiB 131072 ++#define S_256KiB 262144 ++#define S_512KiB 524288 ++#define S_1MiB 1048576 ++#define S_2MiB 2097152 ++#define S_4MiB 4194304 ++#define S_8MiB 8388608 ++#define S_16MiB 16777216 ++#define S_32MiB 33554432 ++#define S_64MiB 67108864 ++#define S_128MiB 134217728 ++#define S_256MiB 268435456 ++#define S_512MiB 536870912 ++#define S_1GiB 1073741824 ++#define S_2GiB 2147483648 ++#define S_4GiB 4294967296 ++#define S_8GiB 8589934592 ++#define S_16GiB 17179869184 ++#define S_32GiB 34359738368 ++#define S_64GiB 68719476736 ++#define S_128GiB 137438953472 ++#define S_256GiB 274877906944 ++#define S_512GiB 549755813888 ++#define S_1TiB 1099511627776 ++#define S_2TiB 2199023255552 ++#define S_4TiB 4398046511104 ++#define S_8TiB 8796093022208 ++#define S_16TiB 17592186044416 ++#define S_32TiB 35184372088832 ++#define S_64TiB 70368744177664 ++#define S_128TiB 140737488355328 ++#define S_256TiB 281474976710656 ++#define S_512TiB 562949953421312 ++#define S_1PiB 1125899906842624 ++#define S_2PiB 2251799813685248 ++#define S_4PiB 4503599627370496 ++#define S_8PiB 9007199254740992 ++#define S_16PiB 18014398509481984 ++#define S_32PiB 36028797018963968 ++#define S_64PiB 72057594037927936 ++#define S_128PiB 144115188075855872 ++#define S_256PiB 288230376151711744 ++#define S_512PiB 576460752303423488 ++#define S_1EiB 1152921504606846976 ++#define S_2EiB 2305843009213693952 ++#define S_4EiB 4611686018427387904 ++#define S_8EiB 9223372036854775808 ++ + #endif +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-add-iommu-lock.patch b/SOURCES/kvm-intel-iommu-add-iommu-lock.patch new file mode 100644 index 0000000..cb5cad4 --- /dev/null +++ b/SOURCES/kvm-intel-iommu-add-iommu-lock.patch @@ -0,0 +1,252 @@ +From a4d88508d1d4f8995d15c1ed822104e46c7b9624 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:40 +0100 +Subject: [PATCH 10/17] intel-iommu: add iommu lock + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-4-peterx@redhat.com> +Patchwork-id: 82675 +O-Subject: [RHEL-8 qemu-kvm PATCH 3/9] intel-iommu: add iommu lock +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +SECURITY IMPLICATION: this patch fixes a potential race when multiple +threads access the IOMMU IOTLB cache. + +Add a per-iommu big lock to protect IOMMU status. Currently the only +thing to be protected is the IOTLB/context cache, since that can be +accessed even without BQL, e.g., in IO dataplane. + +Note that we don't need to protect device page tables since that's fully +controlled by the guest kernel. However there is still possibility that +malicious drivers will program the device to not obey the rule. In that +case QEMU can't really do anything useful, instead the guest itself will +be responsible for all uncertainties. + +CC: QEMU Stable +Reported-by: Fam Zheng +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 1d9efa73e12ddf361ea997c2d532cc4afa6674d1) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 56 ++++++++++++++++++++++++++++++++++++------- + include/hw/i386/intel_iommu.h | 6 +++++ + 2 files changed, 53 insertions(+), 9 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index 3df9045..8d4069d 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -128,6 +128,16 @@ static uint64_t vtd_set_clear_mask_quad(IntelIOMMUState *s, hwaddr addr, + return new_val; + } + ++static inline void vtd_iommu_lock(IntelIOMMUState *s) ++{ ++ qemu_mutex_lock(&s->iommu_lock); ++} ++ ++static inline void vtd_iommu_unlock(IntelIOMMUState *s) ++{ ++ qemu_mutex_unlock(&s->iommu_lock); ++} ++ + /* GHashTable functions */ + static gboolean vtd_uint64_equal(gconstpointer v1, gconstpointer v2) + { +@@ -172,9 +182,9 @@ static gboolean vtd_hash_remove_by_page(gpointer key, gpointer value, + } + + /* Reset all the gen of VTDAddressSpace to zero and set the gen of +- * IntelIOMMUState to 1. ++ * IntelIOMMUState to 1. Must be called with IOMMU lock held. + */ +-static void vtd_reset_context_cache(IntelIOMMUState *s) ++static void vtd_reset_context_cache_locked(IntelIOMMUState *s) + { + VTDAddressSpace *vtd_as; + VTDBus *vtd_bus; +@@ -197,12 +207,20 @@ static void vtd_reset_context_cache(IntelIOMMUState *s) + s->context_cache_gen = 1; + } + +-static void vtd_reset_iotlb(IntelIOMMUState *s) ++/* Must be called with IOMMU lock held. */ ++static void vtd_reset_iotlb_locked(IntelIOMMUState *s) + { + assert(s->iotlb); + g_hash_table_remove_all(s->iotlb); + } + ++static void vtd_reset_iotlb(IntelIOMMUState *s) ++{ ++ vtd_iommu_lock(s); ++ vtd_reset_iotlb_locked(s); ++ vtd_iommu_unlock(s); ++} ++ + static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id, + uint32_t level) + { +@@ -215,6 +233,7 @@ static uint64_t vtd_get_iotlb_gfn(hwaddr addr, uint32_t level) + return (addr & vtd_slpt_level_page_mask(level)) >> VTD_PAGE_SHIFT_4K; + } + ++/* Must be called with IOMMU lock held */ + static VTDIOTLBEntry *vtd_lookup_iotlb(IntelIOMMUState *s, uint16_t source_id, + hwaddr addr) + { +@@ -235,6 +254,7 @@ out: + return entry; + } + ++/* Must be with IOMMU lock held */ + static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id, + uint16_t domain_id, hwaddr addr, uint64_t slpte, + uint8_t access_flags, uint32_t level) +@@ -246,7 +266,7 @@ static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id, + trace_vtd_iotlb_page_update(source_id, addr, slpte, domain_id); + if (g_hash_table_size(s->iotlb) >= VTD_IOTLB_MAX_SIZE) { + trace_vtd_iotlb_reset("iotlb exceeds size limit"); +- vtd_reset_iotlb(s); ++ vtd_reset_iotlb_locked(s); + } + + entry->gfn = gfn; +@@ -1106,7 +1126,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, + IntelIOMMUState *s = vtd_as->iommu_state; + VTDContextEntry ce; + uint8_t bus_num = pci_bus_num(bus); +- VTDContextCacheEntry *cc_entry = &vtd_as->context_cache_entry; ++ VTDContextCacheEntry *cc_entry; + uint64_t slpte, page_mask; + uint32_t level; + uint16_t source_id = vtd_make_source_id(bus_num, devfn); +@@ -1123,6 +1143,10 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, + */ + assert(!vtd_is_interrupt_addr(addr)); + ++ vtd_iommu_lock(s); ++ ++ cc_entry = &vtd_as->context_cache_entry; ++ + /* Try to fetch slpte form IOTLB */ + iotlb_entry = vtd_lookup_iotlb(s, source_id, addr); + if (iotlb_entry) { +@@ -1182,7 +1206,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, + * IOMMU region can be swapped back. + */ + vtd_pt_enable_fast_path(s, source_id); +- ++ vtd_iommu_unlock(s); + return true; + } + +@@ -1203,6 +1227,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, + vtd_update_iotlb(s, source_id, VTD_CONTEXT_ENTRY_DID(ce.hi), addr, slpte, + access_flags, level); + out: ++ vtd_iommu_unlock(s); + entry->iova = addr & page_mask; + entry->translated_addr = vtd_get_slpte_addr(slpte, s->aw_bits) & page_mask; + entry->addr_mask = ~page_mask; +@@ -1210,6 +1235,7 @@ out: + return true; + + error: ++ vtd_iommu_unlock(s); + entry->iova = 0; + entry->translated_addr = 0; + entry->addr_mask = 0; +@@ -1258,10 +1284,13 @@ static void vtd_iommu_replay_all(IntelIOMMUState *s) + static void vtd_context_global_invalidate(IntelIOMMUState *s) + { + trace_vtd_inv_desc_cc_global(); ++ /* Protects context cache */ ++ vtd_iommu_lock(s); + s->context_cache_gen++; + if (s->context_cache_gen == VTD_CONTEXT_CACHE_GEN_MAX) { +- vtd_reset_context_cache(s); ++ vtd_reset_context_cache_locked(s); + } ++ vtd_iommu_unlock(s); + vtd_switch_address_space_all(s); + /* + * From VT-d spec 6.5.2.1, a global context entry invalidation +@@ -1313,7 +1342,9 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s, + if (vtd_as && ((devfn_it & mask) == (devfn & mask))) { + trace_vtd_inv_desc_cc_device(bus_n, VTD_PCI_SLOT(devfn_it), + VTD_PCI_FUNC(devfn_it)); ++ vtd_iommu_lock(s); + vtd_as->context_cache_entry.context_cache_gen = 0; ++ vtd_iommu_unlock(s); + /* + * Do switch address space when needed, in case if the + * device passthrough bit is switched. +@@ -1377,8 +1408,10 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id) + + trace_vtd_inv_desc_iotlb_domain(domain_id); + ++ vtd_iommu_lock(s); + g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_domain, + &domain_id); ++ vtd_iommu_unlock(s); + + QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { + if (!vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), +@@ -1426,7 +1459,9 @@ static void vtd_iotlb_page_invalidate(IntelIOMMUState *s, uint16_t domain_id, + info.domain_id = domain_id; + info.addr = addr; + info.mask = ~((1 << am) - 1); ++ vtd_iommu_lock(s); + g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_page, &info); ++ vtd_iommu_unlock(s); + vtd_iotlb_page_invalidate_notify(s, domain_id, addr, am); + } + +@@ -2929,8 +2964,10 @@ static void vtd_init(IntelIOMMUState *s) + s->cap |= VTD_CAP_CM; + } + +- vtd_reset_context_cache(s); +- vtd_reset_iotlb(s); ++ vtd_iommu_lock(s); ++ vtd_reset_context_cache_locked(s); ++ vtd_reset_iotlb_locked(s); ++ vtd_iommu_unlock(s); + + /* Define registers with default values and bit semantics */ + vtd_define_long(s, DMAR_VER_REG, 0x10UL, 0, 0); +@@ -3070,6 +3107,7 @@ static void vtd_realize(DeviceState *dev, Error **errp) + } + + QLIST_INIT(&s->vtd_as_with_notifiers); ++ qemu_mutex_init(&s->iommu_lock); + memset(s->vtd_as_by_bus_num, 0, sizeof(s->vtd_as_by_bus_num)); + memory_region_init_io(&s->csrmem, OBJECT(s), &vtd_mem_ops, s, + "intel_iommu", DMAR_REG_SIZE); +diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h +index 032e33b..016e74b 100644 +--- a/include/hw/i386/intel_iommu.h ++++ b/include/hw/i386/intel_iommu.h +@@ -300,6 +300,12 @@ struct IntelIOMMUState { + OnOffAuto intr_eim; /* Toggle for EIM cabability */ + bool buggy_eim; /* Force buggy EIM unless eim=off */ + uint8_t aw_bits; /* Host/IOVA address width (in bits) */ ++ ++ /* ++ * Protects IOMMU states in general. Currently it protects the ++ * per-IOMMU IOTLB cache, and context entry cache in VTDAddressSpace. ++ */ ++ QemuMutex iommu_lock; + }; + + /* Find the VTD Address space associated with the given bus pointer, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-introduce-vtd_page_walk_info.patch b/SOURCES/kvm-intel-iommu-introduce-vtd_page_walk_info.patch new file mode 100644 index 0000000..bb871b8 --- /dev/null +++ b/SOURCES/kvm-intel-iommu-introduce-vtd_page_walk_info.patch @@ -0,0 +1,206 @@ +From 22bf46f7576cc12a301d34ca743a6661afcb61ac Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:42 +0100 +Subject: [PATCH 12/17] intel-iommu: introduce vtd_page_walk_info + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-6-peterx@redhat.com> +Patchwork-id: 82678 +O-Subject: [RHEL-8 qemu-kvm PATCH 5/9] intel-iommu: introduce vtd_page_walk_info +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +During the recursive page walking of IOVA page tables, some stack +variables are constant variables and never changed during the whole page +walking procedure. Isolate them into a struct so that we don't need to +pass those contants down the stack every time and multiple times. + +CC: QEMU Stable +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit fe215b0cbb8c1f4b4af0a64aa5c02042080dd537) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 84 +++++++++++++++++++++++++++++++-------------------- + 1 file changed, 52 insertions(+), 32 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index 38ccc74..e247269 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -748,9 +748,27 @@ static int vtd_iova_to_slpte(VTDContextEntry *ce, uint64_t iova, bool is_write, + + typedef int (*vtd_page_walk_hook)(IOMMUTLBEntry *entry, void *private); + ++/** ++ * Constant information used during page walking ++ * ++ * @hook_fn: hook func to be called when detected page ++ * @private: private data to be passed into hook func ++ * @notify_unmap: whether we should notify invalid entries ++ * @aw: maximum address width ++ */ ++typedef struct { ++ vtd_page_walk_hook hook_fn; ++ void *private; ++ bool notify_unmap; ++ uint8_t aw; ++} vtd_page_walk_info; ++ + static int vtd_page_walk_one(IOMMUTLBEntry *entry, int level, +- vtd_page_walk_hook hook_fn, void *private) ++ vtd_page_walk_info *info) + { ++ vtd_page_walk_hook hook_fn = info->hook_fn; ++ void *private = info->private; ++ + assert(hook_fn); + trace_vtd_page_walk_one(level, entry->iova, entry->translated_addr, + entry->addr_mask, entry->perm); +@@ -763,17 +781,13 @@ static int vtd_page_walk_one(IOMMUTLBEntry *entry, int level, + * @addr: base GPA addr to start the walk + * @start: IOVA range start address + * @end: IOVA range end address (start <= addr < end) +- * @hook_fn: hook func to be called when detected page +- * @private: private data to be passed into hook func + * @read: whether parent level has read permission + * @write: whether parent level has write permission +- * @notify_unmap: whether we should notify invalid entries +- * @aw: maximum address width ++ * @info: constant information for the page walk + */ + static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, +- uint64_t end, vtd_page_walk_hook hook_fn, +- void *private, uint32_t level, bool read, +- bool write, bool notify_unmap, uint8_t aw) ++ uint64_t end, uint32_t level, bool read, ++ bool write, vtd_page_walk_info *info) + { + bool read_cur, write_cur, entry_valid; + uint32_t offset; +@@ -823,24 +837,24 @@ static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, + + if (vtd_is_last_slpte(slpte, level)) { + /* NOTE: this is only meaningful if entry_valid == true */ +- entry.translated_addr = vtd_get_slpte_addr(slpte, aw); +- if (!entry_valid && !notify_unmap) { ++ entry.translated_addr = vtd_get_slpte_addr(slpte, info->aw); ++ if (!entry_valid && !info->notify_unmap) { + trace_vtd_page_walk_skip_perm(iova, iova_next); + goto next; + } +- ret = vtd_page_walk_one(&entry, level, hook_fn, private); ++ ret = vtd_page_walk_one(&entry, level, info); + if (ret < 0) { + return ret; + } + } else { + if (!entry_valid) { +- if (notify_unmap) { ++ if (info->notify_unmap) { + /* + * The whole entry is invalid; unmap it all. + * Translated address is meaningless, zero it. + */ + entry.translated_addr = 0x0; +- ret = vtd_page_walk_one(&entry, level, hook_fn, private); ++ ret = vtd_page_walk_one(&entry, level, info); + if (ret < 0) { + return ret; + } +@@ -849,10 +863,9 @@ static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, + } + goto next; + } +- ret = vtd_page_walk_level(vtd_get_slpte_addr(slpte, aw), iova, +- MIN(iova_next, end), hook_fn, private, +- level - 1, read_cur, write_cur, +- notify_unmap, aw); ++ ret = vtd_page_walk_level(vtd_get_slpte_addr(slpte, info->aw), ++ iova, MIN(iova_next, end), level - 1, ++ read_cur, write_cur, info); + if (ret < 0) { + return ret; + } +@@ -871,28 +884,24 @@ next: + * @ce: context entry to walk upon + * @start: IOVA address to start the walk + * @end: IOVA range end address (start <= addr < end) +- * @hook_fn: the hook that to be called for each detected area +- * @private: private data for the hook function +- * @aw: maximum address width ++ * @info: page walking information struct + */ + static int vtd_page_walk(VTDContextEntry *ce, uint64_t start, uint64_t end, +- vtd_page_walk_hook hook_fn, void *private, +- bool notify_unmap, uint8_t aw) ++ vtd_page_walk_info *info) + { + dma_addr_t addr = vtd_ce_get_slpt_base(ce); + uint32_t level = vtd_ce_get_level(ce); + +- if (!vtd_iova_range_check(start, ce, aw)) { ++ if (!vtd_iova_range_check(start, ce, info->aw)) { + return -VTD_FR_ADDR_BEYOND_MGAW; + } + +- if (!vtd_iova_range_check(end, ce, aw)) { ++ if (!vtd_iova_range_check(end, ce, info->aw)) { + /* Fix end so that it reaches the maximum */ +- end = vtd_iova_limit(ce, aw); ++ end = vtd_iova_limit(ce, info->aw); + } + +- return vtd_page_walk_level(addr, start, end, hook_fn, private, +- level, true, true, notify_unmap, aw); ++ return vtd_page_walk_level(addr, start, end, level, true, true, info); + } + + /* Map a device to its corresponding domain (context-entry) */ +@@ -1449,14 +1458,19 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + vtd_as->devfn, &ce); + if (!ret && domain_id == VTD_CONTEXT_ENTRY_DID(ce.hi)) { + if (vtd_as_has_map_notifier(vtd_as)) { ++ vtd_page_walk_info info = { ++ .hook_fn = vtd_page_invalidate_notify_hook, ++ .private = (void *)&vtd_as->iommu, ++ .notify_unmap = true, ++ .aw = s->aw_bits, ++ }; ++ + /* + * As long as we have MAP notifications registered in + * any of our IOMMU notifiers, we need to sync the + * shadow page table. + */ +- vtd_page_walk(&ce, addr, addr + size, +- vtd_page_invalidate_notify_hook, +- (void *)&vtd_as->iommu, true, s->aw_bits); ++ vtd_page_walk(&ce, addr, addr + size, &info); + } else { + /* + * For UNMAP-only notifiers, we don't need to walk the +@@ -2924,8 +2938,14 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) + ce.hi, ce.lo); + if (vtd_as_has_map_notifier(vtd_as)) { + /* This is required only for MAP typed notifiers */ +- vtd_page_walk(&ce, 0, ~0ULL, vtd_replay_hook, (void *)n, false, +- s->aw_bits); ++ vtd_page_walk_info info = { ++ .hook_fn = vtd_replay_hook, ++ .private = (void *)n, ++ .notify_unmap = false, ++ .aw = s->aw_bits, ++ }; ++ ++ vtd_page_walk(&ce, 0, ~0ULL, &info); + } + } else { + trace_vtd_replay_ce_invalid(bus_n, PCI_SLOT(vtd_as->devfn), +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-only-do-page-walk-for-MAP-notifiers.patch b/SOURCES/kvm-intel-iommu-only-do-page-walk-for-MAP-notifiers.patch new file mode 100644 index 0000000..ffc193d --- /dev/null +++ b/SOURCES/kvm-intel-iommu-only-do-page-walk-for-MAP-notifiers.patch @@ -0,0 +1,127 @@ +From 3ce00f8e07a2f9e1b3839b367821121511301c9f Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:41 +0100 +Subject: [PATCH 11/17] intel-iommu: only do page walk for MAP notifiers + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-5-peterx@redhat.com> +Patchwork-id: 82677 +O-Subject: [RHEL-8 qemu-kvm PATCH 4/9] intel-iommu: only do page walk for MAP notifiers +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +For UNMAP-only IOMMU notifiers, we don't need to walk the page tables. +Fasten that procedure by skipping the page table walk. That should +boost performance for UNMAP-only notifiers like vhost. + +CC: QEMU Stable +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 4f8a62a933a79094e44bc1b16b63bb23e62d67b4) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 44 ++++++++++++++++++++++++++++++++++++++----- + include/hw/i386/intel_iommu.h | 2 ++ + 2 files changed, 41 insertions(+), 5 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index 8d4069d..38ccc74 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -138,6 +138,12 @@ static inline void vtd_iommu_unlock(IntelIOMMUState *s) + qemu_mutex_unlock(&s->iommu_lock); + } + ++/* Whether the address space needs to notify new mappings */ ++static inline gboolean vtd_as_has_map_notifier(VTDAddressSpace *as) ++{ ++ return as->notifier_flags & IOMMU_NOTIFIER_MAP; ++} ++ + /* GHashTable functions */ + static gboolean vtd_uint64_equal(gconstpointer v1, gconstpointer v2) + { +@@ -1436,14 +1442,36 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + VTDAddressSpace *vtd_as; + VTDContextEntry ce; + int ret; ++ hwaddr size = (1 << am) * VTD_PAGE_SIZE; + + QLIST_FOREACH(vtd_as, &(s->vtd_as_with_notifiers), next) { + ret = vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), + vtd_as->devfn, &ce); + if (!ret && domain_id == VTD_CONTEXT_ENTRY_DID(ce.hi)) { +- vtd_page_walk(&ce, addr, addr + (1 << am) * VTD_PAGE_SIZE, +- vtd_page_invalidate_notify_hook, +- (void *)&vtd_as->iommu, true, s->aw_bits); ++ if (vtd_as_has_map_notifier(vtd_as)) { ++ /* ++ * As long as we have MAP notifications registered in ++ * any of our IOMMU notifiers, we need to sync the ++ * shadow page table. ++ */ ++ vtd_page_walk(&ce, addr, addr + size, ++ vtd_page_invalidate_notify_hook, ++ (void *)&vtd_as->iommu, true, s->aw_bits); ++ } else { ++ /* ++ * For UNMAP-only notifiers, we don't need to walk the ++ * page tables. We just deliver the PSI down to ++ * invalidate caches. ++ */ ++ IOMMUTLBEntry entry = { ++ .target_as = &address_space_memory, ++ .iova = addr, ++ .translated_addr = 0, ++ .addr_mask = size - 1, ++ .perm = IOMMU_NONE, ++ }; ++ memory_region_notify_iommu(&vtd_as->iommu, entry); ++ } + } + } + } +@@ -2383,6 +2411,9 @@ static void vtd_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu, + exit(1); + } + ++ /* Update per-address-space notifier flags */ ++ vtd_as->notifier_flags = new; ++ + if (old == IOMMU_NOTIFIER_NONE) { + QLIST_INSERT_HEAD(&s->vtd_as_with_notifiers, vtd_as, next); + } else if (new == IOMMU_NOTIFIER_NONE) { +@@ -2891,8 +2922,11 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) + PCI_FUNC(vtd_as->devfn), + VTD_CONTEXT_ENTRY_DID(ce.hi), + ce.hi, ce.lo); +- vtd_page_walk(&ce, 0, ~0ULL, vtd_replay_hook, (void *)n, false, +- s->aw_bits); ++ if (vtd_as_has_map_notifier(vtd_as)) { ++ /* This is required only for MAP typed notifiers */ ++ vtd_page_walk(&ce, 0, ~0ULL, vtd_replay_hook, (void *)n, false, ++ s->aw_bits); ++ } + } else { + trace_vtd_replay_ce_invalid(bus_n, PCI_SLOT(vtd_as->devfn), + PCI_FUNC(vtd_as->devfn)); +diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h +index 016e74b..156f35e 100644 +--- a/include/hw/i386/intel_iommu.h ++++ b/include/hw/i386/intel_iommu.h +@@ -93,6 +93,8 @@ struct VTDAddressSpace { + IntelIOMMUState *iommu_state; + VTDContextCacheEntry context_cache_entry; + QLIST_ENTRY(VTDAddressSpace) next; ++ /* Superset of notifier flags that this address space has */ ++ IOMMUNotifierFlag notifier_flags; + }; + + struct VTDBus { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-pass-in-address-space-when-page-walk.patch b/SOURCES/kvm-intel-iommu-pass-in-address-space-when-page-walk.patch new file mode 100644 index 0000000..b8634df --- /dev/null +++ b/SOURCES/kvm-intel-iommu-pass-in-address-space-when-page-walk.patch @@ -0,0 +1,63 @@ +From 898b7073aa58dbd144b929cf28a297908b8f45f2 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:43 +0100 +Subject: [PATCH 13/17] intel-iommu: pass in address space when page walk + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-7-peterx@redhat.com> +Patchwork-id: 82679 +O-Subject: [RHEL-8 qemu-kvm PATCH 6/9] intel-iommu: pass in address space when page walk +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +We pass in the VTDAddressSpace too. It'll be used in the follow up +patches. + +CC: QEMU Stable +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 2f764fa87d2a81812b313dd6d998e10126292653) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index e247269..a882894 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -754,9 +754,11 @@ typedef int (*vtd_page_walk_hook)(IOMMUTLBEntry *entry, void *private); + * @hook_fn: hook func to be called when detected page + * @private: private data to be passed into hook func + * @notify_unmap: whether we should notify invalid entries ++ * @as: VT-d address space of the device + * @aw: maximum address width + */ + typedef struct { ++ VTDAddressSpace *as; + vtd_page_walk_hook hook_fn; + void *private; + bool notify_unmap; +@@ -1463,6 +1465,7 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + .private = (void *)&vtd_as->iommu, + .notify_unmap = true, + .aw = s->aw_bits, ++ .as = vtd_as, + }; + + /* +@@ -2943,6 +2946,7 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) + .private = (void *)n, + .notify_unmap = false, + .aw = s->aw_bits, ++ .as = vtd_as, + }; + + vtd_page_walk(&ce, 0, ~0ULL, &info); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-remove-IntelIOMMUNotifierNode.patch b/SOURCES/kvm-intel-iommu-remove-IntelIOMMUNotifierNode.patch new file mode 100644 index 0000000..8f2a145 --- /dev/null +++ b/SOURCES/kvm-intel-iommu-remove-IntelIOMMUNotifierNode.patch @@ -0,0 +1,182 @@ +From 98fcc1d8ee1fb973cb2b63ed6cad756fa42ffce6 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:39 +0100 +Subject: [PATCH 09/17] intel-iommu: remove IntelIOMMUNotifierNode + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-3-peterx@redhat.com> +Patchwork-id: 82676 +O-Subject: [RHEL-8 qemu-kvm PATCH 2/9] intel-iommu: remove IntelIOMMUNotifierNode +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +That is not really necessary. Removing that node struct and put the +list entry directly into VTDAddressSpace. It simplfies the code a lot. +Since at it, rename the old notifiers_list into vtd_as_with_notifiers. + +CC: QEMU Stable +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit b4a4ba0d68f50f218ee3957b6638dbee32a5eeef) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 41 +++++++++++------------------------------ + include/hw/i386/intel_iommu.h | 9 ++------- + 2 files changed, 13 insertions(+), 37 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index b359efd..3df9045 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -1248,10 +1248,10 @@ static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s) + + static void vtd_iommu_replay_all(IntelIOMMUState *s) + { +- IntelIOMMUNotifierNode *node; ++ VTDAddressSpace *vtd_as; + +- QLIST_FOREACH(node, &s->notifiers_list, next) { +- memory_region_iommu_replay_all(&node->vtd_as->iommu); ++ QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { ++ memory_region_iommu_replay_all(&vtd_as->iommu); + } + } + +@@ -1372,7 +1372,6 @@ static void vtd_iotlb_global_invalidate(IntelIOMMUState *s) + + static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id) + { +- IntelIOMMUNotifierNode *node; + VTDContextEntry ce; + VTDAddressSpace *vtd_as; + +@@ -1381,8 +1380,7 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id) + g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_domain, + &domain_id); + +- QLIST_FOREACH(node, &s->notifiers_list, next) { +- vtd_as = node->vtd_as; ++ QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { + if (!vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), + vtd_as->devfn, &ce) && + domain_id == VTD_CONTEXT_ENTRY_DID(ce.hi)) { +@@ -1402,12 +1400,11 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + uint16_t domain_id, hwaddr addr, + uint8_t am) + { +- IntelIOMMUNotifierNode *node; ++ VTDAddressSpace *vtd_as; + VTDContextEntry ce; + int ret; + +- QLIST_FOREACH(node, &(s->notifiers_list), next) { +- VTDAddressSpace *vtd_as = node->vtd_as; ++ QLIST_FOREACH(vtd_as, &(s->vtd_as_with_notifiers), next) { + ret = vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), + vtd_as->devfn, &ce); + if (!ret && domain_id == VTD_CONTEXT_ENTRY_DID(ce.hi)) { +@@ -2344,8 +2341,6 @@ static void vtd_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu, + { + VTDAddressSpace *vtd_as = container_of(iommu, VTDAddressSpace, iommu); + IntelIOMMUState *s = vtd_as->iommu_state; +- IntelIOMMUNotifierNode *node = NULL; +- IntelIOMMUNotifierNode *next_node = NULL; + + if (!s->caching_mode && new & IOMMU_NOTIFIER_MAP) { + error_report("We need to set caching-mode=1 for intel-iommu to enable " +@@ -2354,21 +2349,9 @@ static void vtd_iommu_notify_flag_changed(IOMMUMemoryRegion *iommu, + } + + if (old == IOMMU_NOTIFIER_NONE) { +- node = g_malloc0(sizeof(*node)); +- node->vtd_as = vtd_as; +- QLIST_INSERT_HEAD(&s->notifiers_list, node, next); +- return; +- } +- +- /* update notifier node with new flags */ +- QLIST_FOREACH_SAFE(node, &s->notifiers_list, next, next_node) { +- if (node->vtd_as == vtd_as) { +- if (new == IOMMU_NOTIFIER_NONE) { +- QLIST_REMOVE(node, next); +- g_free(node); +- } +- return; +- } ++ QLIST_INSERT_HEAD(&s->vtd_as_with_notifiers, vtd_as, next); ++ } else if (new == IOMMU_NOTIFIER_NONE) { ++ QLIST_REMOVE(vtd_as, next); + } + } + +@@ -2838,12 +2821,10 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) + + static void vtd_address_space_unmap_all(IntelIOMMUState *s) + { +- IntelIOMMUNotifierNode *node; + VTDAddressSpace *vtd_as; + IOMMUNotifier *n; + +- QLIST_FOREACH(node, &s->notifiers_list, next) { +- vtd_as = node->vtd_as; ++ QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { + IOMMU_NOTIFIER_FOREACH(n, &vtd_as->iommu) { + vtd_address_space_unmap(vtd_as, n); + } +@@ -3088,7 +3069,7 @@ static void vtd_realize(DeviceState *dev, Error **errp) + return; + } + +- QLIST_INIT(&s->notifiers_list); ++ QLIST_INIT(&s->vtd_as_with_notifiers); + memset(s->vtd_as_by_bus_num, 0, sizeof(s->vtd_as_by_bus_num)); + memory_region_init_io(&s->csrmem, OBJECT(s), &vtd_mem_ops, s, + "intel_iommu", DMAR_REG_SIZE); +diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h +index 45ec891..032e33b 100644 +--- a/include/hw/i386/intel_iommu.h ++++ b/include/hw/i386/intel_iommu.h +@@ -67,7 +67,6 @@ typedef union VTD_IR_TableEntry VTD_IR_TableEntry; + typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress; + typedef struct VTDIrq VTDIrq; + typedef struct VTD_MSIMessage VTD_MSIMessage; +-typedef struct IntelIOMMUNotifierNode IntelIOMMUNotifierNode; + + /* Context-Entry */ + struct VTDContextEntry { +@@ -93,6 +92,7 @@ struct VTDAddressSpace { + MemoryRegion iommu_ir; /* Interrupt region: 0xfeeXXXXX */ + IntelIOMMUState *iommu_state; + VTDContextCacheEntry context_cache_entry; ++ QLIST_ENTRY(VTDAddressSpace) next; + }; + + struct VTDBus { +@@ -253,11 +253,6 @@ struct VTD_MSIMessage { + /* When IR is enabled, all MSI/MSI-X data bits should be zero */ + #define VTD_IR_MSI_DATA (0) + +-struct IntelIOMMUNotifierNode { +- VTDAddressSpace *vtd_as; +- QLIST_ENTRY(IntelIOMMUNotifierNode) next; +-}; +- + /* The iommu (DMAR) device state struct */ + struct IntelIOMMUState { + X86IOMMUState x86_iommu; +@@ -295,7 +290,7 @@ struct IntelIOMMUState { + GHashTable *vtd_as_by_busptr; /* VTDBus objects indexed by PCIBus* reference */ + VTDBus *vtd_as_by_bus_num[VTD_PCI_BUS_MAX]; /* VTDBus objects indexed by bus number */ + /* list of registered notifiers */ +- QLIST_HEAD(, IntelIOMMUNotifierNode) notifiers_list; ++ QLIST_HEAD(, VTDAddressSpace) vtd_as_with_notifiers; + + /* interrupt remapping */ + bool intr_enabled; /* Whether guest enabled IR */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-replace-more-vtd_err_-traces.patch b/SOURCES/kvm-intel-iommu-replace-more-vtd_err_-traces.patch new file mode 100644 index 0000000..e21bcbf --- /dev/null +++ b/SOURCES/kvm-intel-iommu-replace-more-vtd_err_-traces.patch @@ -0,0 +1,215 @@ +From f83115d485ef395b751b8d25b56917cf7630d63d Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:34 +0000 +Subject: [PATCH 06/35] intel-iommu: replace more vtd_err_* traces +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-4-peterx@redhat.com> +Patchwork-id: 82962 +O-Subject: [RHEL-8 qemu-kvm PATCH 3/7] intel-iommu: replace more vtd_err_* traces +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Laurent Vivier + +Bugzilla: 1625173 + +Replace all the trace_vtd_err_*() hooks with the new error_report_once() +since they are similar to trace_vtd_err() - dumping the first error +would be mostly enough, then we have them on by default too. + +Signed-off-by: Peter Xu +Message-Id: <20180815095328.32414-4-peterx@redhat.com> +[Use "%x" instead of "%" PRIx16 to print uint16_t, whitespace tidied up] +Signed-off-by: Markus Armbruster +(cherry picked from commit 4e4abd111a2af0179a4467368d695958844bf113) +Signed-off-by: Peter Xu + +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 64 ++++++++++++++++++++++++++++++++++++--------------- + hw/i386/trace-events | 12 ---------- + 2 files changed, 46 insertions(+), 30 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index ab11cc4..aab86e9 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -705,7 +705,8 @@ static int vtd_iova_to_slpte(VTDContextEntry *ce, uint64_t iova, bool is_write, + uint64_t access_right_check; + + if (!vtd_iova_range_check(iova, ce, aw_bits)) { +- trace_vtd_err_dmar_iova_overflow(iova); ++ error_report_once("%s: detected IOVA overflow (iova=0x%" PRIx64 ")", ++ __func__, iova); + return -VTD_FR_ADDR_BEYOND_MGAW; + } + +@@ -717,7 +718,8 @@ static int vtd_iova_to_slpte(VTDContextEntry *ce, uint64_t iova, bool is_write, + slpte = vtd_get_slpte(addr, offset); + + if (slpte == (uint64_t)-1) { +- trace_vtd_err_dmar_slpte_read_error(iova, level); ++ error_report_once("%s: detected read error on DMAR slpte " ++ "(iova=0x%" PRIx64 ")", __func__, iova); + if (level == vtd_ce_get_level(ce)) { + /* Invalid programming of context-entry */ + return -VTD_FR_CONTEXT_ENTRY_INV; +@@ -728,11 +730,17 @@ static int vtd_iova_to_slpte(VTDContextEntry *ce, uint64_t iova, bool is_write, + *reads = (*reads) && (slpte & VTD_SL_R); + *writes = (*writes) && (slpte & VTD_SL_W); + if (!(slpte & access_right_check)) { +- trace_vtd_err_dmar_slpte_perm_error(iova, level, slpte, is_write); ++ error_report_once("%s: detected slpte permission error " ++ "(iova=0x%" PRIx64 ", level=0x%" PRIx32 ", " ++ "slpte=0x%" PRIx64 ", write=%d)", __func__, ++ iova, level, slpte, is_write); + return is_write ? -VTD_FR_WRITE : -VTD_FR_READ; + } + if (vtd_slpte_nonzero_rsvd(slpte, level)) { +- trace_vtd_err_dmar_slpte_resv_error(iova, level, slpte); ++ error_report_once("%s: detected splte reserve non-zero " ++ "iova=0x%" PRIx64 ", level=0x%" PRIx32 ++ "slpte=0x%" PRIx64 ")", __func__, iova, ++ level, slpte); + return -VTD_FR_PAGING_ENTRY_RSVD; + } + +@@ -1697,7 +1705,10 @@ static void vtd_handle_gcmd_qie(IntelIOMMUState *s, bool en) + /* Ok - report back to driver */ + vtd_set_clear_mask_long(s, DMAR_GSTS_REG, VTD_GSTS_QIES, 0); + } else { +- trace_vtd_err_qi_disable(s->iq_head, s->iq_tail, s->iq_last_desc_type); ++ error_report_once("%s: detected improper state when disable QI " ++ "(head=0x%x, tail=0x%x, last_type=%d)", ++ __func__, ++ s->iq_head, s->iq_tail, s->iq_last_desc_type); + } + } + } +@@ -2094,7 +2105,9 @@ static void vtd_fetch_inv_desc(IntelIOMMUState *s) + + if (s->iq_tail >= s->iq_size) { + /* Detects an invalid Tail pointer */ +- trace_vtd_err_qi_tail(s->iq_tail, s->iq_size); ++ error_report_once("%s: detected invalid QI tail " ++ "(tail=0x%x, size=0x%x)", ++ __func__, s->iq_tail, s->iq_size); + vtd_handle_inv_queue_error(s); + return; + } +@@ -2507,10 +2520,12 @@ static IOMMUTLBEntry vtd_iommu_translate(IOMMUMemoryRegion *iommu, hwaddr addr, + iotlb.iova, iotlb.translated_addr, + iotlb.addr_mask); + } else { +- trace_vtd_err_dmar_translate(pci_bus_num(vtd_as->bus), +- VTD_PCI_SLOT(vtd_as->devfn), +- VTD_PCI_FUNC(vtd_as->devfn), +- iotlb.iova); ++ error_report_once("%s: detected translation failure " ++ "(dev=%02x:%02x:%02x, iova=0x%" PRIx64 ")", ++ __func__, pci_bus_num(vtd_as->bus), ++ VTD_PCI_SLOT(vtd_as->devfn), ++ VTD_PCI_FUNC(vtd_as->devfn), ++ iotlb.iova); + } + + return iotlb; +@@ -2626,15 +2641,19 @@ static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index, + le64_to_cpu(entry->data[0])); + + if (!entry->irte.present) { +- trace_vtd_err_irte(index, le64_to_cpu(entry->data[1]), +- le64_to_cpu(entry->data[0])); ++ error_report_once("%s: detected non-present IRTE " ++ "(index=%u, high=0x%" PRIx64 ", low=0x%" PRIx64 ")", ++ __func__, index, le64_to_cpu(entry->data[1]), ++ le64_to_cpu(entry->data[0])); + return -VTD_FR_IR_ENTRY_P; + } + + if (entry->irte.__reserved_0 || entry->irte.__reserved_1 || + entry->irte.__reserved_2) { +- trace_vtd_err_irte(index, le64_to_cpu(entry->data[1]), +- le64_to_cpu(entry->data[0])); ++ error_report_once("%s: detected non-zero reserved IRTE " ++ "(index=%u, high=0x%" PRIx64 ", low=0x%" PRIx64 ")", ++ __func__, index, le64_to_cpu(entry->data[1]), ++ le64_to_cpu(entry->data[0])); + return -VTD_FR_IR_IRTE_RSVD; + } + +@@ -2648,7 +2667,9 @@ static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index, + case VTD_SVT_ALL: + mask = vtd_svt_mask[entry->irte.sid_q]; + if ((source_id & mask) != (sid & mask)) { +- trace_vtd_err_irte_sid(index, sid, source_id); ++ error_report_once("%s: invalid IRTE SID " ++ "(index=%u, sid=%u, source_id=%u)", ++ __func__, index, sid, source_id); + return -VTD_FR_IR_SID_ERR; + } + break; +@@ -2658,13 +2679,17 @@ static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index, + bus_min = source_id & 0xff; + bus = sid >> 8; + if (bus > bus_max || bus < bus_min) { +- trace_vtd_err_irte_sid_bus(index, bus, bus_min, bus_max); ++ error_report_once("%s: invalid SVT_BUS " ++ "(index=%u, bus=%u, min=%u, max=%u)", ++ __func__, index, bus, bus_min, bus_max); + return -VTD_FR_IR_SID_ERR; + } + break; + + default: +- trace_vtd_err_irte_svt(index, entry->irte.sid_vtype); ++ error_report_once("%s: detected invalid IRTE SVT " ++ "(index=%u, type=%d)", __func__, ++ index, entry->irte.sid_vtype); + /* Take this as verification failure. */ + return -VTD_FR_IR_SID_ERR; + break; +@@ -2786,7 +2811,10 @@ static int vtd_interrupt_remap_msi(IntelIOMMUState *iommu, + if (addr.addr.sub_valid) { + trace_vtd_ir_remap_type("MSI"); + if (origin->data & VTD_IR_MSI_DATA_RESERVED) { +- trace_vtd_err_ir_msi_invalid(sid, origin->address, origin->data); ++ error_report_once("%s: invalid IR MSI " ++ "(sid=%u, address=0x%" PRIx64 ++ ", data=0x%" PRIx32 ")", ++ __func__, sid, origin->address, origin->data); + return -VTD_FR_IR_REQ_RSVD; + } + } else { +diff --git a/hw/i386/trace-events b/hw/i386/trace-events +index 922431b..9e6fc4d 100644 +--- a/hw/i386/trace-events ++++ b/hw/i386/trace-events +@@ -69,19 +69,7 @@ vtd_ir_remap_msi_req(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PR + vtd_fsts_ppf(bool set) "FSTS PPF bit set to %d" + vtd_fsts_clear_ip(void) "" + vtd_frr_new(int index, uint64_t hi, uint64_t lo) "index %d high 0x%"PRIx64" low 0x%"PRIx64 +-vtd_err_dmar_iova_overflow(uint64_t iova) "iova 0x%"PRIx64 +-vtd_err_dmar_slpte_read_error(uint64_t iova, int level) "iova 0x%"PRIx64" level %d" +-vtd_err_dmar_slpte_perm_error(uint64_t iova, int level, uint64_t slpte, bool is_write) "iova 0x%"PRIx64" level %d slpte 0x%"PRIx64" write %d" +-vtd_err_dmar_slpte_resv_error(uint64_t iova, int level, uint64_t slpte) "iova 0x%"PRIx64" level %d slpte 0x%"PRIx64 +-vtd_err_dmar_translate(uint8_t bus, uint8_t slot, uint8_t func, uint64_t iova) "dev %02x:%02x.%02x iova 0x%"PRIx64 + vtd_warn_invalid_qi_tail(uint16_t tail) "tail 0x%"PRIx16 +-vtd_err_qi_disable(uint16_t head, uint16_t tail, int type) "head 0x%"PRIx16" tail 0x%"PRIx16" last_desc_type %d" +-vtd_err_qi_tail(uint16_t tail, uint16_t size) "tail 0x%"PRIx16" size 0x%"PRIx16 +-vtd_err_irte(int index, uint64_t lo, uint64_t hi) "index %d low 0x%"PRIx64" high 0x%"PRIx64 +-vtd_err_irte_sid(int index, uint16_t req, uint16_t target) "index %d SVT_ALL sid 0x%"PRIx16" (should be: 0x%"PRIx16")" +-vtd_err_irte_sid_bus(int index, uint8_t bus, uint8_t min, uint8_t max) "index %d SVT_BUS bus 0x%"PRIx8" (should be: 0x%"PRIx8"-0x%"PRIx8")" +-vtd_err_irte_svt(int index, int type) "index %d SVT type %d" +-vtd_err_ir_msi_invalid(uint16_t sid, uint64_t addr, uint64_t data) "sid 0x%"PRIx16" addr 0x%"PRIx64" data 0x%"PRIx64 + vtd_warn_ir_vector(uint16_t sid, int index, int vec, int target) "sid 0x%"PRIx16" index %d vec %d (should be: %d)" + vtd_warn_ir_trigger(uint16_t sid, int index, int trig, int target) "sid 0x%"PRIx16" index %d trigger %d (should be: %d)" + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-rework-the-page-walk-logic.patch b/SOURCES/kvm-intel-iommu-rework-the-page-walk-logic.patch new file mode 100644 index 0000000..1a4c943 --- /dev/null +++ b/SOURCES/kvm-intel-iommu-rework-the-page-walk-logic.patch @@ -0,0 +1,401 @@ +From 560b10746036972ee8be6ff5c252ab2fc56de0e4 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:46 +0100 +Subject: [PATCH 16/17] intel-iommu: rework the page walk logic + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-10-peterx@redhat.com> +Patchwork-id: 82682 +O-Subject: [RHEL-8 qemu-kvm PATCH 9/9] intel-iommu: rework the page walk logic +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +This patch fixes a potential small window that the DMA page table might +be incomplete or invalid when the guest sends domain/context +invalidations to a device. This can cause random DMA errors for +assigned devices. + +This is a major change to the VT-d shadow page walking logic. It +includes but is not limited to: + +- For each VTDAddressSpace, now we maintain what IOVA ranges we have + mapped and what we have not. With that information, now we only send + MAP or UNMAP when necessary. Say, we don't send MAP notifies if we + know we have already mapped the range, meanwhile we don't send UNMAP + notifies if we know we never mapped the range at all. + +- Introduce vtd_sync_shadow_page_table[_range] APIs so that we can call + in any places to resync the shadow page table for a device. + +- When we receive domain/context invalidation, we should not really run + the replay logic, instead we use the new sync shadow page table API to + resync the whole shadow page table without unmapping the whole + region. After this change, we'll only do the page walk once for each + domain invalidations (before this, it can be multiple, depending on + number of notifiers per address space). + +While at it, the page walking logic is also refactored to be simpler. + +CC: QEMU Stable +Reported-by: Jintack Lim +Tested-by: Jintack Lim +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 63b88968f139b6a77f2f81e6f1eedf70c0170a85) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 213 ++++++++++++++++++++++++++++++------------ + hw/i386/trace-events | 3 +- + include/hw/i386/intel_iommu.h | 2 + + 3 files changed, 159 insertions(+), 59 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index 61bb3d3..b5a09b7 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -769,10 +769,77 @@ typedef struct { + + static int vtd_page_walk_one(IOMMUTLBEntry *entry, vtd_page_walk_info *info) + { ++ VTDAddressSpace *as = info->as; + vtd_page_walk_hook hook_fn = info->hook_fn; + void *private = info->private; ++ DMAMap target = { ++ .iova = entry->iova, ++ .size = entry->addr_mask, ++ .translated_addr = entry->translated_addr, ++ .perm = entry->perm, ++ }; ++ DMAMap *mapped = iova_tree_find(as->iova_tree, &target); ++ ++ if (entry->perm == IOMMU_NONE && !info->notify_unmap) { ++ trace_vtd_page_walk_one_skip_unmap(entry->iova, entry->addr_mask); ++ return 0; ++ } + + assert(hook_fn); ++ ++ /* Update local IOVA mapped ranges */ ++ if (entry->perm) { ++ if (mapped) { ++ /* If it's exactly the same translation, skip */ ++ if (!memcmp(mapped, &target, sizeof(target))) { ++ trace_vtd_page_walk_one_skip_map(entry->iova, entry->addr_mask, ++ entry->translated_addr); ++ return 0; ++ } else { ++ /* ++ * Translation changed. Normally this should not ++ * happen, but it can happen when with buggy guest ++ * OSes. Note that there will be a small window that ++ * we don't have map at all. But that's the best ++ * effort we can do. The ideal way to emulate this is ++ * atomically modify the PTE to follow what has ++ * changed, but we can't. One example is that vfio ++ * driver only has VFIO_IOMMU_[UN]MAP_DMA but no ++ * interface to modify a mapping (meanwhile it seems ++ * meaningless to even provide one). Anyway, let's ++ * mark this as a TODO in case one day we'll have ++ * a better solution. ++ */ ++ IOMMUAccessFlags cache_perm = entry->perm; ++ int ret; ++ ++ /* Emulate an UNMAP */ ++ entry->perm = IOMMU_NONE; ++ trace_vtd_page_walk_one(info->domain_id, ++ entry->iova, ++ entry->translated_addr, ++ entry->addr_mask, ++ entry->perm); ++ ret = hook_fn(entry, private); ++ if (ret) { ++ return ret; ++ } ++ /* Drop any existing mapping */ ++ iova_tree_remove(as->iova_tree, &target); ++ /* Recover the correct permission */ ++ entry->perm = cache_perm; ++ } ++ } ++ iova_tree_insert(as->iova_tree, &target); ++ } else { ++ if (!mapped) { ++ /* Skip since we didn't map this range at all */ ++ trace_vtd_page_walk_one_skip_unmap(entry->iova, entry->addr_mask); ++ return 0; ++ } ++ iova_tree_remove(as->iova_tree, &target); ++ } ++ + trace_vtd_page_walk_one(info->domain_id, entry->iova, + entry->translated_addr, entry->addr_mask, + entry->perm); +@@ -834,45 +901,34 @@ static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, + */ + entry_valid = read_cur | write_cur; + +- entry.target_as = &address_space_memory; +- entry.iova = iova & subpage_mask; +- entry.perm = IOMMU_ACCESS_FLAG(read_cur, write_cur); +- entry.addr_mask = ~subpage_mask; +- +- if (vtd_is_last_slpte(slpte, level)) { +- /* NOTE: this is only meaningful if entry_valid == true */ +- entry.translated_addr = vtd_get_slpte_addr(slpte, info->aw); +- if (!entry_valid && !info->notify_unmap) { +- trace_vtd_page_walk_skip_perm(iova, iova_next); +- goto next; +- } +- ret = vtd_page_walk_one(&entry, info); +- if (ret < 0) { +- return ret; +- } +- } else { +- if (!entry_valid) { +- if (info->notify_unmap) { +- /* +- * The whole entry is invalid; unmap it all. +- * Translated address is meaningless, zero it. +- */ +- entry.translated_addr = 0x0; +- ret = vtd_page_walk_one(&entry, info); +- if (ret < 0) { +- return ret; +- } +- } else { +- trace_vtd_page_walk_skip_perm(iova, iova_next); +- } +- goto next; +- } ++ if (!vtd_is_last_slpte(slpte, level) && entry_valid) { ++ /* ++ * This is a valid PDE (or even bigger than PDE). We need ++ * to walk one further level. ++ */ + ret = vtd_page_walk_level(vtd_get_slpte_addr(slpte, info->aw), + iova, MIN(iova_next, end), level - 1, + read_cur, write_cur, info); +- if (ret < 0) { +- return ret; +- } ++ } else { ++ /* ++ * This means we are either: ++ * ++ * (1) the real page entry (either 4K page, or huge page) ++ * (2) the whole range is invalid ++ * ++ * In either case, we send an IOTLB notification down. ++ */ ++ entry.target_as = &address_space_memory; ++ entry.iova = iova & subpage_mask; ++ entry.perm = IOMMU_ACCESS_FLAG(read_cur, write_cur); ++ entry.addr_mask = ~subpage_mask; ++ /* NOTE: this is only meaningful if entry_valid == true */ ++ entry.translated_addr = vtd_get_slpte_addr(slpte, info->aw); ++ ret = vtd_page_walk_one(&entry, info); ++ } ++ ++ if (ret < 0) { ++ return ret; + } + + next: +@@ -964,6 +1020,58 @@ static int vtd_dev_to_context_entry(IntelIOMMUState *s, uint8_t bus_num, + return 0; + } + ++static int vtd_sync_shadow_page_hook(IOMMUTLBEntry *entry, ++ void *private) ++{ ++ memory_region_notify_iommu((IOMMUMemoryRegion *)private, *entry); ++ return 0; ++} ++ ++/* If context entry is NULL, we'll try to fetch it on our own. */ ++static int vtd_sync_shadow_page_table_range(VTDAddressSpace *vtd_as, ++ VTDContextEntry *ce, ++ hwaddr addr, hwaddr size) ++{ ++ IntelIOMMUState *s = vtd_as->iommu_state; ++ vtd_page_walk_info info = { ++ .hook_fn = vtd_sync_shadow_page_hook, ++ .private = (void *)&vtd_as->iommu, ++ .notify_unmap = true, ++ .aw = s->aw_bits, ++ .as = vtd_as, ++ }; ++ VTDContextEntry ce_cache; ++ int ret; ++ ++ if (ce) { ++ /* If the caller provided context entry, use it */ ++ ce_cache = *ce; ++ } else { ++ /* If the caller didn't provide ce, try to fetch */ ++ ret = vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), ++ vtd_as->devfn, &ce_cache); ++ if (ret) { ++ /* ++ * This should not really happen, but in case it happens, ++ * we just skip the sync for this time. After all we even ++ * don't have the root table pointer! ++ */ ++ trace_vtd_err("Detected invalid context entry when " ++ "trying to sync shadow page table"); ++ return 0; ++ } ++ } ++ ++ info.domain_id = VTD_CONTEXT_ENTRY_DID(ce_cache.hi); ++ ++ return vtd_page_walk(&ce_cache, addr, addr + size, &info); ++} ++ ++static int vtd_sync_shadow_page_table(VTDAddressSpace *vtd_as) ++{ ++ return vtd_sync_shadow_page_table_range(vtd_as, NULL, 0, UINT64_MAX); ++} ++ + /* + * Fetch translation type for specific device. Returns <0 if error + * happens, otherwise return the shifted type to check against +@@ -1296,7 +1404,7 @@ static void vtd_iommu_replay_all(IntelIOMMUState *s) + VTDAddressSpace *vtd_as; + + QLIST_FOREACH(vtd_as, &s->vtd_as_with_notifiers, next) { +- memory_region_iommu_replay_all(&vtd_as->iommu); ++ vtd_sync_shadow_page_table(vtd_as); + } + } + +@@ -1371,14 +1479,13 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s, + vtd_switch_address_space(vtd_as); + /* + * So a device is moving out of (or moving into) a +- * domain, a replay() suites here to notify all the +- * IOMMU_NOTIFIER_MAP registers about this change. ++ * domain, resync the shadow page table. + * This won't bring bad even if we have no such + * notifier registered - the IOMMU notification + * framework will skip MAP notifications if that + * happened. + */ +- memory_region_iommu_replay_all(&vtd_as->iommu); ++ vtd_sync_shadow_page_table(vtd_as); + } + } + } +@@ -1436,18 +1543,11 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id) + if (!vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), + vtd_as->devfn, &ce) && + domain_id == VTD_CONTEXT_ENTRY_DID(ce.hi)) { +- memory_region_iommu_replay_all(&vtd_as->iommu); ++ vtd_sync_shadow_page_table(vtd_as); + } + } + } + +-static int vtd_page_invalidate_notify_hook(IOMMUTLBEntry *entry, +- void *private) +-{ +- memory_region_notify_iommu((IOMMUMemoryRegion *)private, *entry); +- return 0; +-} +- + static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + uint16_t domain_id, hwaddr addr, + uint8_t am) +@@ -1462,21 +1562,12 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + vtd_as->devfn, &ce); + if (!ret && domain_id == VTD_CONTEXT_ENTRY_DID(ce.hi)) { + if (vtd_as_has_map_notifier(vtd_as)) { +- vtd_page_walk_info info = { +- .hook_fn = vtd_page_invalidate_notify_hook, +- .private = (void *)&vtd_as->iommu, +- .notify_unmap = true, +- .aw = s->aw_bits, +- .as = vtd_as, +- .domain_id = domain_id, +- }; +- + /* + * As long as we have MAP notifications registered in + * any of our IOMMU notifiers, we need to sync the + * shadow page table. + */ +- vtd_page_walk(&ce, addr, addr + size, &info); ++ vtd_sync_shadow_page_table_range(vtd_as, &ce, addr, size); + } else { + /* + * For UNMAP-only notifiers, we don't need to walk the +@@ -2806,6 +2897,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) + vtd_dev_as->devfn = (uint8_t)devfn; + vtd_dev_as->iommu_state = s; + vtd_dev_as->context_cache_entry.context_cache_gen = 0; ++ vtd_dev_as->iova_tree = iova_tree_new(); + + /* + * Memory region relationships looks like (Address range shows +@@ -2858,6 +2950,7 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) + hwaddr start = n->start; + hwaddr end = n->end; + IntelIOMMUState *s = as->iommu_state; ++ DMAMap map; + + /* + * Note: all the codes in this function has a assumption that IOVA +@@ -2902,6 +2995,10 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) + VTD_PCI_FUNC(as->devfn), + entry.iova, size); + ++ map.iova = entry.iova; ++ map.size = entry.addr_mask; ++ iova_tree_remove(as->iova_tree, &map); ++ + memory_region_notify_one(n, &entry); + } + +diff --git a/hw/i386/trace-events b/hw/i386/trace-events +index ca23ba9..e14d06e 100644 +--- a/hw/i386/trace-events ++++ b/hw/i386/trace-events +@@ -40,8 +40,9 @@ vtd_replay_ce_valid(uint8_t bus, uint8_t dev, uint8_t fn, uint16_t domain, uint6 + vtd_replay_ce_invalid(uint8_t bus, uint8_t dev, uint8_t fn) "replay invalid context device %02"PRIx8":%02"PRIx8".%02"PRIx8 + vtd_page_walk_level(uint64_t addr, uint32_t level, uint64_t start, uint64_t end) "walk (base=0x%"PRIx64", level=%"PRIu32") iova range 0x%"PRIx64" - 0x%"PRIx64 + vtd_page_walk_one(uint16_t domain, uint64_t iova, uint64_t gpa, uint64_t mask, int perm) "domain 0x%"PRIu16" iova 0x%"PRIx64" -> gpa 0x%"PRIx64" mask 0x%"PRIx64" perm %d" ++vtd_page_walk_one_skip_map(uint64_t iova, uint64_t mask, uint64_t translated) "iova 0x%"PRIx64" mask 0x%"PRIx64" translated 0x%"PRIx64 ++vtd_page_walk_one_skip_unmap(uint64_t iova, uint64_t mask) "iova 0x%"PRIx64" mask 0x%"PRIx64 + vtd_page_walk_skip_read(uint64_t iova, uint64_t next) "Page walk skip iova 0x%"PRIx64" - 0x%"PRIx64" due to unable to read" +-vtd_page_walk_skip_perm(uint64_t iova, uint64_t next) "Page walk skip iova 0x%"PRIx64" - 0x%"PRIx64" due to perm empty" + vtd_page_walk_skip_reserve(uint64_t iova, uint64_t next) "Page walk skip iova 0x%"PRIx64" - 0x%"PRIx64" due to rsrv set" + vtd_switch_address_space(uint8_t bus, uint8_t slot, uint8_t fn, bool on) "Device %02x:%02x.%x switching address space (iommu enabled=%d)" + vtd_as_unmap_whole(uint8_t bus, uint8_t slot, uint8_t fn, uint64_t iova, uint64_t size) "Device %02x:%02x.%x start 0x%"PRIx64" size 0x%"PRIx64 +diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h +index 156f35e..fbfedcb 100644 +--- a/include/hw/i386/intel_iommu.h ++++ b/include/hw/i386/intel_iommu.h +@@ -27,6 +27,7 @@ + #include "hw/i386/ioapic.h" + #include "hw/pci/msi.h" + #include "hw/sysbus.h" ++#include "qemu/iova-tree.h" + + #define TYPE_INTEL_IOMMU_DEVICE "intel-iommu" + #define INTEL_IOMMU_DEVICE(obj) \ +@@ -95,6 +96,7 @@ struct VTDAddressSpace { + QLIST_ENTRY(VTDAddressSpace) next; + /* Superset of notifier flags that this address space has */ + IOMMUNotifierFlag notifier_flags; ++ IOVATree *iova_tree; /* Traces mapped IOVA ranges */ + }; + + struct VTDBus { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-send-PSI-always-even-if-across-PDEs.patch b/SOURCES/kvm-intel-iommu-send-PSI-always-even-if-across-PDEs.patch new file mode 100644 index 0000000..1f4cec9 --- /dev/null +++ b/SOURCES/kvm-intel-iommu-send-PSI-always-even-if-across-PDEs.patch @@ -0,0 +1,131 @@ +From a09020ea2e2e645b95ed603e075938d413f1114f Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:38 +0100 +Subject: [PATCH 08/17] intel-iommu: send PSI always even if across PDEs + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-2-peterx@redhat.com> +Patchwork-id: 82674 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/9] intel-iommu: send PSI always even if across PDEs +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +SECURITY IMPLICATION: without this patch, any guest with both assigned +device and a vIOMMU might encounter stale IO page mappings even if guest +has already unmapped the page, which may lead to guest memory +corruption. The stale mappings will only be limited to the guest's own +memory range, so it should not affect the host memory or other guests on +the host. + +During IOVA page table walking, there is a special case when the PSI +covers one whole PDE (Page Directory Entry, which contains 512 Page +Table Entries) or more. In the past, we skip that entry and we don't +notify the IOMMU notifiers. This is not correct. We should send UNMAP +notification to registered UNMAP notifiers in this case. + +For UNMAP only notifiers, this might cause IOTLBs cached in the devices +even if they were already invalid. For MAP/UNMAP notifiers like +vfio-pci, this will cause stale page mappings. + +This special case doesn't trigger often, but it is very easy to be +triggered by nested device assignments, since in that case we'll +possibly map the whole L2 guest RAM region into the device's IOVA +address space (several GBs at least), which is far bigger than normal +kernel driver usages of the device (tens of MBs normally). + +Without this patch applied to L1 QEMU, nested device assignment to L2 +guests will dump some errors like: + +qemu-system-x86_64: VFIO_MAP_DMA: -17 +qemu-system-x86_64: vfio_dma_map(0x557305420c30, 0xad000, 0x1000, + 0x7f89a920d000) = -17 (File exists) + +CC: QEMU Stable +Acked-by: Jason Wang +[peterx: rewrite the commit message] +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 36d2d52bdb45f5b753a61fdaf0fe7891f1f5b61d) +Signed-off-by: Peter Xu + +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 42 ++++++++++++++++++++++++++++++------------ + 1 file changed, 30 insertions(+), 12 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index fb31de9..b359efd 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -722,6 +722,15 @@ static int vtd_iova_to_slpte(VTDContextEntry *ce, uint64_t iova, bool is_write, + + typedef int (*vtd_page_walk_hook)(IOMMUTLBEntry *entry, void *private); + ++static int vtd_page_walk_one(IOMMUTLBEntry *entry, int level, ++ vtd_page_walk_hook hook_fn, void *private) ++{ ++ assert(hook_fn); ++ trace_vtd_page_walk_one(level, entry->iova, entry->translated_addr, ++ entry->addr_mask, entry->perm); ++ return hook_fn(entry, private); ++} ++ + /** + * vtd_page_walk_level - walk over specific level for IOVA range + * +@@ -781,28 +790,37 @@ static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, + */ + entry_valid = read_cur | write_cur; + ++ entry.target_as = &address_space_memory; ++ entry.iova = iova & subpage_mask; ++ entry.perm = IOMMU_ACCESS_FLAG(read_cur, write_cur); ++ entry.addr_mask = ~subpage_mask; ++ + if (vtd_is_last_slpte(slpte, level)) { +- entry.target_as = &address_space_memory; +- entry.iova = iova & subpage_mask; + /* NOTE: this is only meaningful if entry_valid == true */ + entry.translated_addr = vtd_get_slpte_addr(slpte, aw); +- entry.addr_mask = ~subpage_mask; +- entry.perm = IOMMU_ACCESS_FLAG(read_cur, write_cur); + if (!entry_valid && !notify_unmap) { + trace_vtd_page_walk_skip_perm(iova, iova_next); + goto next; + } +- trace_vtd_page_walk_one(level, entry.iova, entry.translated_addr, +- entry.addr_mask, entry.perm); +- if (hook_fn) { +- ret = hook_fn(&entry, private); +- if (ret < 0) { +- return ret; +- } ++ ret = vtd_page_walk_one(&entry, level, hook_fn, private); ++ if (ret < 0) { ++ return ret; + } + } else { + if (!entry_valid) { +- trace_vtd_page_walk_skip_perm(iova, iova_next); ++ if (notify_unmap) { ++ /* ++ * The whole entry is invalid; unmap it all. ++ * Translated address is meaningless, zero it. ++ */ ++ entry.translated_addr = 0x0; ++ ret = vtd_page_walk_one(&entry, level, hook_fn, private); ++ if (ret < 0) { ++ return ret; ++ } ++ } else { ++ trace_vtd_page_walk_skip_perm(iova, iova_next); ++ } + goto next; + } + ret = vtd_page_walk_level(vtd_get_slpte_addr(slpte, aw), iova, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-start-to-use-error_report_once.patch b/SOURCES/kvm-intel-iommu-start-to-use-error_report_once.patch new file mode 100644 index 0000000..c950b1d --- /dev/null +++ b/SOURCES/kvm-intel-iommu-start-to-use-error_report_once.patch @@ -0,0 +1,239 @@ +From df15f85e59fe41c2663242feb7b5213047a9f456 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:33 +0000 +Subject: [PATCH 05/35] intel-iommu: start to use error_report_once +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-3-peterx@redhat.com> +Patchwork-id: 82961 +O-Subject: [RHEL-8 qemu-kvm PATCH 2/7] intel-iommu: start to use error_report_once +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Laurent Vivier + +Bugzilla: 1625173 + +Replace existing trace_vtd_err() with error_report_once() then stderr +will capture something if any of the error happens, meanwhile we don't +suffer from any DDOS. Then remove the trace point. Since at it, +provide more information where proper (now we can pass parameters into +the report function). + +Signed-off-by: Peter Xu +Message-Id: <20180815095328.32414-3-peterx@redhat.com> +Reviewed-by: Philippe Mathieu-Daudé +[Two format strings fixed, whitespace tidied up] +Signed-off-by: Markus Armbruster +(cherry picked from commit 1376211f77bdcd84dc4acb877690f7399d8cf58a) +Signed-off-by: Peter Xu + +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 65 +++++++++++++++++++++++++++++---------------------- + hw/i386/trace-events | 1 - + 2 files changed, 37 insertions(+), 29 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index b5a09b7..ab11cc4 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -311,14 +311,14 @@ static void vtd_generate_fault_event(IntelIOMMUState *s, uint32_t pre_fsts) + { + if (pre_fsts & VTD_FSTS_PPF || pre_fsts & VTD_FSTS_PFO || + pre_fsts & VTD_FSTS_IQE) { +- trace_vtd_err("There are previous interrupt conditions " +- "to be serviced by software, fault event " +- "is not generated."); ++ error_report_once("There are previous interrupt conditions " ++ "to be serviced by software, fault event " ++ "is not generated"); + return; + } + vtd_set_clear_mask_long(s, DMAR_FECTL_REG, 0, VTD_FECTL_IP); + if (vtd_get_long_raw(s, DMAR_FECTL_REG) & VTD_FECTL_IM) { +- trace_vtd_err("Interrupt Mask set, irq is not generated."); ++ error_report_once("Interrupt Mask set, irq is not generated"); + } else { + vtd_generate_interrupt(s, DMAR_FEADDR_REG, DMAR_FEDATA_REG); + vtd_set_clear_mask_long(s, DMAR_FECTL_REG, VTD_FECTL_IP, 0); +@@ -426,20 +426,20 @@ static void vtd_report_dmar_fault(IntelIOMMUState *s, uint16_t source_id, + trace_vtd_dmar_fault(source_id, fault, addr, is_write); + + if (fsts_reg & VTD_FSTS_PFO) { +- trace_vtd_err("New fault is not recorded due to " +- "Primary Fault Overflow."); ++ error_report_once("New fault is not recorded due to " ++ "Primary Fault Overflow"); + return; + } + + if (vtd_try_collapse_fault(s, source_id)) { +- trace_vtd_err("New fault is not recorded due to " +- "compression of faults."); ++ error_report_once("New fault is not recorded due to " ++ "compression of faults"); + return; + } + + if (vtd_is_frcd_set(s, s->next_frcd_reg)) { +- trace_vtd_err("Next Fault Recording Reg is used, " +- "new fault is not recorded, set PFO field."); ++ error_report_once("Next Fault Recording Reg is used, " ++ "new fault is not recorded, set PFO field"); + vtd_set_clear_mask_long(s, DMAR_FSTS_REG, 0, VTD_FSTS_PFO); + return; + } +@@ -447,8 +447,8 @@ static void vtd_report_dmar_fault(IntelIOMMUState *s, uint16_t source_id, + vtd_record_frcd(s, s->next_frcd_reg, source_id, addr, fault, is_write); + + if (fsts_reg & VTD_FSTS_PPF) { +- trace_vtd_err("There are pending faults already, " +- "fault event is not generated."); ++ error_report_once("There are pending faults already, " ++ "fault event is not generated"); + vtd_set_frcd_and_update_ppf(s, s->next_frcd_reg); + s->next_frcd_reg++; + if (s->next_frcd_reg == DMAR_FRCD_REG_NR) { +@@ -1056,8 +1056,10 @@ static int vtd_sync_shadow_page_table_range(VTDAddressSpace *vtd_as, + * we just skip the sync for this time. After all we even + * don't have the root table pointer! + */ +- trace_vtd_err("Detected invalid context entry when " +- "trying to sync shadow page table"); ++ error_report_once("%s: invalid context entry for bus 0x%x" ++ " devfn 0x%x", ++ __func__, pci_bus_num(vtd_as->bus), ++ vtd_as->devfn); + return 0; + } + } +@@ -1514,7 +1516,8 @@ static uint64_t vtd_context_cache_invalidate(IntelIOMMUState *s, uint64_t val) + break; + + default: +- trace_vtd_err("Context cache invalidate type error."); ++ error_report_once("%s: invalid context: 0x%" PRIx64, ++ __func__, val); + caig = 0; + } + return caig; +@@ -1634,7 +1637,8 @@ static uint64_t vtd_iotlb_flush(IntelIOMMUState *s, uint64_t val) + am = VTD_IVA_AM(addr); + addr = VTD_IVA_ADDR(addr); + if (am > VTD_MAMV) { +- trace_vtd_err("IOTLB PSI flush: address mask overflow."); ++ error_report_once("%s: address mask overflow: 0x%" PRIx64, ++ __func__, vtd_get_quad_raw(s, DMAR_IVA_REG)); + iaig = 0; + break; + } +@@ -1643,7 +1647,8 @@ static uint64_t vtd_iotlb_flush(IntelIOMMUState *s, uint64_t val) + break; + + default: +- trace_vtd_err("IOTLB flush: invalid granularity."); ++ error_report_once("%s: invalid granularity: 0x%" PRIx64, ++ __func__, val); + iaig = 0; + } + return iaig; +@@ -1793,8 +1798,8 @@ static void vtd_handle_ccmd_write(IntelIOMMUState *s) + /* Context-cache invalidation request */ + if (val & VTD_CCMD_ICC) { + if (s->qi_enabled) { +- trace_vtd_err("Queued Invalidation enabled, " +- "should not use register-based invalidation"); ++ error_report_once("Queued Invalidation enabled, " ++ "should not use register-based invalidation"); + return; + } + ret = vtd_context_cache_invalidate(s, val); +@@ -1814,8 +1819,8 @@ static void vtd_handle_iotlb_write(IntelIOMMUState *s) + /* IOTLB invalidation request */ + if (val & VTD_TLB_IVT) { + if (s->qi_enabled) { +- trace_vtd_err("Queued Invalidation enabled, " +- "should not use register-based invalidation."); ++ error_report_once("Queued Invalidation enabled, " ++ "should not use register-based invalidation"); + return; + } + ret = vtd_iotlb_flush(s, val); +@@ -1833,7 +1838,7 @@ static bool vtd_get_inv_desc(dma_addr_t base_addr, uint32_t offset, + dma_addr_t addr = base_addr + offset * sizeof(*inv_desc); + if (dma_memory_read(&address_space_memory, addr, inv_desc, + sizeof(*inv_desc))) { +- trace_vtd_err("Read INV DESC failed."); ++ error_report_once("Read INV DESC failed"); + inv_desc->lo = 0; + inv_desc->hi = 0; + return false; +@@ -2188,7 +2193,8 @@ static uint64_t vtd_mem_read(void *opaque, hwaddr addr, unsigned size) + trace_vtd_reg_read(addr, size); + + if (addr + size > DMAR_REG_SIZE) { +- trace_vtd_err("Read MMIO over range."); ++ error_report_once("%s: MMIO over range: addr=0x%" PRIx64 ++ " size=0x%u", __func__, addr, size); + return (uint64_t)-1; + } + +@@ -2239,7 +2245,8 @@ static void vtd_mem_write(void *opaque, hwaddr addr, + trace_vtd_reg_write(addr, size, val); + + if (addr + size > DMAR_REG_SIZE) { +- trace_vtd_err("Write MMIO over range."); ++ error_report_once("%s: MMIO over range: addr=0x%" PRIx64 ++ " size=0x%u", __func__, addr, size); + return; + } + +@@ -2610,7 +2617,8 @@ static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index, + addr = iommu->intr_root + index * sizeof(*entry); + if (dma_memory_read(&address_space_memory, addr, entry, + sizeof(*entry))) { +- trace_vtd_err("Memory read failed for IRTE."); ++ error_report_once("%s: read failed: ind=0x%x addr=0x%" PRIx64, ++ __func__, index, addr); + return -VTD_FR_IR_ROOT_INVAL; + } + +@@ -2742,14 +2750,15 @@ static int vtd_interrupt_remap_msi(IntelIOMMUState *iommu, + } + + if (origin->address & VTD_MSI_ADDR_HI_MASK) { +- trace_vtd_err("MSI address high 32 bits non-zero when " +- "Interrupt Remapping enabled."); ++ error_report_once("%s: MSI address high 32 bits non-zero detected: " ++ "address=0x%" PRIx64, __func__, origin->address); + return -VTD_FR_IR_REQ_RSVD; + } + + addr.data = origin->address & VTD_MSI_ADDR_LO_MASK; + if (addr.addr.__head != 0xfee) { +- trace_vtd_err("MSI addr low 32 bit invalid."); ++ error_report_once("%s: MSI address low 32 bit invalid: 0x%" PRIx32, ++ __func__, addr.data); + return -VTD_FR_IR_REQ_RSVD; + } + +diff --git a/hw/i386/trace-events b/hw/i386/trace-events +index e14d06e..922431b 100644 +--- a/hw/i386/trace-events ++++ b/hw/i386/trace-events +@@ -69,7 +69,6 @@ vtd_ir_remap_msi_req(uint64_t addr, uint64_t data) "addr 0x%"PRIx64" data 0x%"PR + vtd_fsts_ppf(bool set) "FSTS PPF bit set to %d" + vtd_fsts_clear_ip(void) "" + vtd_frr_new(int index, uint64_t hi, uint64_t lo) "index %d high 0x%"PRIx64" low 0x%"PRIx64 +-vtd_err(const char *str) "%s" + vtd_err_dmar_iova_overflow(uint64_t iova) "iova 0x%"PRIx64 + vtd_err_dmar_slpte_read_error(uint64_t iova, int level) "iova 0x%"PRIx64" level %d" + vtd_err_dmar_slpte_perm_error(uint64_t iova, int level, uint64_t slpte, bool is_write) "iova 0x%"PRIx64" level %d slpte 0x%"PRIx64" write %d" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel-iommu-trace-domain-id-during-page-walk.patch b/SOURCES/kvm-intel-iommu-trace-domain-id-during-page-walk.patch new file mode 100644 index 0000000..ce8a094 --- /dev/null +++ b/SOURCES/kvm-intel-iommu-trace-domain-id-during-page-walk.patch @@ -0,0 +1,119 @@ +From c856dbfb24dcca457dfe8daadfe5a657f60e79f7 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:44 +0100 +Subject: [PATCH 14/17] intel-iommu: trace domain id during page walk + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-8-peterx@redhat.com> +Patchwork-id: 82680 +O-Subject: [RHEL-8 qemu-kvm PATCH 7/9] intel-iommu: trace domain id during page walk +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +This patch only modifies the trace points. + +Previously we were tracing page walk levels. They are redundant since +we have page mask (size) already. Now we trace something much more +useful which is the domain ID of the page walking. That can be very +useful when we trace more than one devices on the same system, so that +we can know which map is for which domain. + +CC: QEMU Stable +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit d118c06ebbee2d23ddf873cae4a809311aa61310) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 16 ++++++++++------ + hw/i386/trace-events | 2 +- + 2 files changed, 11 insertions(+), 7 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index a882894..61bb3d3 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -756,6 +756,7 @@ typedef int (*vtd_page_walk_hook)(IOMMUTLBEntry *entry, void *private); + * @notify_unmap: whether we should notify invalid entries + * @as: VT-d address space of the device + * @aw: maximum address width ++ * @domain: domain ID of the page walk + */ + typedef struct { + VTDAddressSpace *as; +@@ -763,17 +764,18 @@ typedef struct { + void *private; + bool notify_unmap; + uint8_t aw; ++ uint16_t domain_id; + } vtd_page_walk_info; + +-static int vtd_page_walk_one(IOMMUTLBEntry *entry, int level, +- vtd_page_walk_info *info) ++static int vtd_page_walk_one(IOMMUTLBEntry *entry, vtd_page_walk_info *info) + { + vtd_page_walk_hook hook_fn = info->hook_fn; + void *private = info->private; + + assert(hook_fn); +- trace_vtd_page_walk_one(level, entry->iova, entry->translated_addr, +- entry->addr_mask, entry->perm); ++ trace_vtd_page_walk_one(info->domain_id, entry->iova, ++ entry->translated_addr, entry->addr_mask, ++ entry->perm); + return hook_fn(entry, private); + } + +@@ -844,7 +846,7 @@ static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, + trace_vtd_page_walk_skip_perm(iova, iova_next); + goto next; + } +- ret = vtd_page_walk_one(&entry, level, info); ++ ret = vtd_page_walk_one(&entry, info); + if (ret < 0) { + return ret; + } +@@ -856,7 +858,7 @@ static int vtd_page_walk_level(dma_addr_t addr, uint64_t start, + * Translated address is meaningless, zero it. + */ + entry.translated_addr = 0x0; +- ret = vtd_page_walk_one(&entry, level, info); ++ ret = vtd_page_walk_one(&entry, info); + if (ret < 0) { + return ret; + } +@@ -1466,6 +1468,7 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s, + .notify_unmap = true, + .aw = s->aw_bits, + .as = vtd_as, ++ .domain_id = domain_id, + }; + + /* +@@ -2947,6 +2950,7 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) + .notify_unmap = false, + .aw = s->aw_bits, + .as = vtd_as, ++ .domain_id = VTD_CONTEXT_ENTRY_DID(ce.hi), + }; + + vtd_page_walk(&ce, 0, ~0ULL, &info); +diff --git a/hw/i386/trace-events b/hw/i386/trace-events +index 22d4464..ca23ba9 100644 +--- a/hw/i386/trace-events ++++ b/hw/i386/trace-events +@@ -39,7 +39,7 @@ vtd_fault_disabled(void) "Fault processing disabled for context entry" + vtd_replay_ce_valid(uint8_t bus, uint8_t dev, uint8_t fn, uint16_t domain, uint64_t hi, uint64_t lo) "replay valid context device %02"PRIx8":%02"PRIx8".%02"PRIx8" domain 0x%"PRIx16" hi 0x%"PRIx64" lo 0x%"PRIx64 + vtd_replay_ce_invalid(uint8_t bus, uint8_t dev, uint8_t fn) "replay invalid context device %02"PRIx8":%02"PRIx8".%02"PRIx8 + vtd_page_walk_level(uint64_t addr, uint32_t level, uint64_t start, uint64_t end) "walk (base=0x%"PRIx64", level=%"PRIu32") iova range 0x%"PRIx64" - 0x%"PRIx64 +-vtd_page_walk_one(uint32_t level, uint64_t iova, uint64_t gpa, uint64_t mask, int perm) "detected page level 0x%"PRIx32" iova 0x%"PRIx64" -> gpa 0x%"PRIx64" mask 0x%"PRIx64" perm %d" ++vtd_page_walk_one(uint16_t domain, uint64_t iova, uint64_t gpa, uint64_t mask, int perm) "domain 0x%"PRIu16" iova 0x%"PRIx64" -> gpa 0x%"PRIx64" mask 0x%"PRIx64" perm %d" + vtd_page_walk_skip_read(uint64_t iova, uint64_t next) "Page walk skip iova 0x%"PRIx64" - 0x%"PRIx64" due to unable to read" + vtd_page_walk_skip_perm(uint64_t iova, uint64_t next) "Page walk skip iova 0x%"PRIx64" - 0x%"PRIx64" due to perm empty" + vtd_page_walk_skip_reserve(uint64_t iova, uint64_t next) "Page walk skip iova 0x%"PRIx64" - 0x%"PRIx64" due to rsrv set" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel_iommu-better-handling-of-dmar-state-switch.patch b/SOURCES/kvm-intel_iommu-better-handling-of-dmar-state-switch.patch new file mode 100644 index 0000000..915ffa0 --- /dev/null +++ b/SOURCES/kvm-intel_iommu-better-handling-of-dmar-state-switch.patch @@ -0,0 +1,149 @@ +From 467b5e3127faf168fba6ce61a602f82d34a699c7 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:36 +0000 +Subject: [PATCH 08/35] intel_iommu: better handling of dmar state switch + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-6-peterx@redhat.com> +Patchwork-id: 82964 +O-Subject: [RHEL-8 qemu-kvm PATCH 5/7] intel_iommu: better handling of dmar state switch +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Laurent Vivier + +Bugzilla: 1625173 + +QEMU is not handling the global DMAR switch well, especially when from +"on" to "off". + +Let's first take the example of system reset. + +Assuming that a guest has IOMMU enabled. When it reboots, we will drop +all the existing DMAR mappings to handle the system reset, however we'll +still keep the existing memory layouts which has the IOMMU memory region +enabled. So after the reboot and before the kernel reloads again, there +will be no mapping at all for the host device. That's problematic since +any software (for example, SeaBIOS) that runs earlier than the kernel +after the reboot will assume the IOMMU is disabled, so any DMA from the +software will fail. + +For example, a guest that boots on an assigned NVMe device might fail to +find the boot device after a system reboot/reset and we'll be able to +observe SeaBIOS errors if we capture the debugging log: + + WARNING - Timeout at nvme_wait:144! + +Meanwhile, we should see DMAR errors on the host of that NVMe device. +It's the DMA fault that caused a NVMe driver timeout. + +The correct fix should be that we do proper switching of device DMA +address spaces when system resets, which will setup correct memory +regions and notify the backend of the devices. This might not affect +much on non-assigned devices since QEMU VT-d emulation will assume a +default passthrough mapping if DMAR is not enabled in the GCMD +register (please refer to vtd_iommu_translate). However that's required +for an assigned devices, since that'll rebuild the correct GPA to HPA +mapping that is needed for any DMA operation during guest bootstrap. + +Besides the system reset, we have some other places that might change +the global DMAR status and we'd better do the same thing there. For +example, when we change the state of GCMD register, or the DMAR root +pointer. Do the same refresh for all these places. For these two +places we'll also need to explicitly invalidate the context entry cache +and iotlb cache. + +Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1625173 +CC: QEMU Stable +Reported-by: Cong Li +Signed-off-by: Peter Xu +-- +v2: +- do the same for GCMD write, or root pointer update [Alex] +- test is carried out by me this time, by observing the + vtd_switch_address_space tracepoint after system reboot +v3: +- rewrite commit message as suggested by Alex +Signed-off-by: Peter Xu +Reviewed-by: Eric Auger +Reviewed-by: Jason Wang +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 2cc9ddccebcaa48b3debfc279a83761fcbb7616c) +Signed-off-by: Peter Xu + +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 21 ++++++++++++++------- + 1 file changed, 14 insertions(+), 7 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index 48d0ba3..a6e87a9 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -37,6 +37,8 @@ + #include "kvm_i386.h" + #include "trace.h" + ++static void vtd_address_space_refresh_all(IntelIOMMUState *s); ++ + static void vtd_define_quad(IntelIOMMUState *s, hwaddr addr, uint64_t val, + uint64_t wmask, uint64_t w1cmask) + { +@@ -1436,7 +1438,7 @@ static void vtd_context_global_invalidate(IntelIOMMUState *s) + vtd_reset_context_cache_locked(s); + } + vtd_iommu_unlock(s); +- vtd_switch_address_space_all(s); ++ vtd_address_space_refresh_all(s); + /* + * From VT-d spec 6.5.2.1, a global context entry invalidation + * should be followed by a IOTLB global invalidation, so we should +@@ -1727,6 +1729,8 @@ static void vtd_handle_gcmd_srtp(IntelIOMMUState *s) + vtd_root_table_setup(s); + /* Ok - report back to driver */ + vtd_set_clear_mask_long(s, DMAR_GSTS_REG, 0, VTD_GSTS_RTPS); ++ vtd_reset_caches(s); ++ vtd_address_space_refresh_all(s); + } + + /* Set Interrupt Remap Table Pointer */ +@@ -1759,7 +1763,8 @@ static void vtd_handle_gcmd_te(IntelIOMMUState *s, bool en) + vtd_set_clear_mask_long(s, DMAR_GSTS_REG, VTD_GSTS_TES, 0); + } + +- vtd_switch_address_space_all(s); ++ vtd_reset_caches(s); ++ vtd_address_space_refresh_all(s); + } + + /* Handle Interrupt Remap Enable/Disable */ +@@ -3059,6 +3064,12 @@ static void vtd_address_space_unmap_all(IntelIOMMUState *s) + } + } + ++static void vtd_address_space_refresh_all(IntelIOMMUState *s) ++{ ++ vtd_address_space_unmap_all(s); ++ vtd_switch_address_space_all(s); ++} ++ + static int vtd_replay_hook(IOMMUTLBEntry *entry, void *private) + { + memory_region_notify_one((IOMMUNotifier *)private, entry); +@@ -3231,11 +3242,7 @@ static void vtd_reset(DeviceState *dev) + IntelIOMMUState *s = INTEL_IOMMU_DEVICE(dev); + + vtd_init(s); +- +- /* +- * When device reset, throw away all mappings and external caches +- */ +- vtd_address_space_unmap_all(s); ++ vtd_address_space_refresh_all(s); + } + + static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel_iommu-handle-invalid-ce-for-shadow-sync.patch b/SOURCES/kvm-intel_iommu-handle-invalid-ce-for-shadow-sync.patch new file mode 100644 index 0000000..28680e8 --- /dev/null +++ b/SOURCES/kvm-intel_iommu-handle-invalid-ce-for-shadow-sync.patch @@ -0,0 +1,88 @@ +From f7e1caa3302f705abe196b8a021cfa52750749ff Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:38 +0000 +Subject: [PATCH 10/35] intel_iommu: handle invalid ce for shadow sync + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-8-peterx@redhat.com> +Patchwork-id: 82966 +O-Subject: [RHEL-8 qemu-kvm PATCH 7/7] intel_iommu: handle invalid ce for shadow sync +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Laurent Vivier + +Bugzilla: 1629616 + +We should handle VTD_FR_CONTEXT_ENTRY_P properly when synchronizing +shadow page tables. Having invalid context entry there is perfectly +valid when we move a device out of an existing domain. When that +happens, instead of posting an error we invalidate the whole region. + +Without this patch, QEMU will crash if we do these steps: + +(1) start QEMU with VT-d IOMMU and two 10G NICs (ixgbe) +(2) bind the NICs with vfio-pci in the guest +(3) start testpmd with the NICs applied +(4) stop testpmd +(5) rebind the NIC back to ixgbe kernel driver + +The patch should fix it. + +Reported-by: Pei Zhang +Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1627272 +Signed-off-by: Peter Xu +Reviewed-by: Eric Auger +Reviewed-by: Maxime Coquelin +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit c28b535d083d0a263d38d9ceeada83cdae8c64f0) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 17 +++++++++++++++++ + 1 file changed, 17 insertions(+) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index c95128d..12af410 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -38,6 +38,7 @@ + #include "trace.h" + + static void vtd_address_space_refresh_all(IntelIOMMUState *s); ++static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n); + + static void vtd_define_quad(IntelIOMMUState *s, hwaddr addr, uint64_t val, + uint64_t wmask, uint64_t w1cmask) +@@ -1066,11 +1067,27 @@ static int vtd_sync_shadow_page_table(VTDAddressSpace *vtd_as) + { + int ret; + VTDContextEntry ce; ++ IOMMUNotifier *n; + + ret = vtd_dev_to_context_entry(vtd_as->iommu_state, + pci_bus_num(vtd_as->bus), + vtd_as->devfn, &ce); + if (ret) { ++ if (ret == -VTD_FR_CONTEXT_ENTRY_P) { ++ /* ++ * It's a valid scenario to have a context entry that is ++ * not present. For example, when a device is removed ++ * from an existing domain then the context entry will be ++ * zeroed by the guest before it was put into another ++ * domain. When this happens, instead of synchronizing ++ * the shadow pages we should invalidate all existing ++ * mappings and notify the backends. ++ */ ++ IOMMU_NOTIFIER_FOREACH(n, &vtd_as->iommu) { ++ vtd_address_space_unmap(vtd_as, n); ++ } ++ ret = 0; ++ } + return ret; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel_iommu-introduce-vtd_reset_caches.patch b/SOURCES/kvm-intel_iommu-introduce-vtd_reset_caches.patch new file mode 100644 index 0000000..1df892f --- /dev/null +++ b/SOURCES/kvm-intel_iommu-introduce-vtd_reset_caches.patch @@ -0,0 +1,65 @@ +From c79d36500fdf74bbc87b2475b3a95648ee2dff8a Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:35 +0000 +Subject: [PATCH 07/35] intel_iommu: introduce vtd_reset_caches() + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-5-peterx@redhat.com> +Patchwork-id: 82963 +O-Subject: [RHEL-8 qemu-kvm PATCH 4/7] intel_iommu: introduce vtd_reset_caches() +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Laurent Vivier + +Bugzilla: 1625173 + +Provide the function and use it in vtd_init(). Used to reset both +context entry cache and iotlb cache for the whole IOMMU unit. + +Signed-off-by: Peter Xu +Reviewed-by: Eric Auger +Reviewed-by: Jason Wang +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 06aba4ca52fd2c8718b8ba486f22f0aa7c99ed55) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 13 +++++++++---- + 1 file changed, 9 insertions(+), 4 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index aab86e9..48d0ba3 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -227,6 +227,14 @@ static void vtd_reset_iotlb(IntelIOMMUState *s) + vtd_iommu_unlock(s); + } + ++static void vtd_reset_caches(IntelIOMMUState *s) ++{ ++ vtd_iommu_lock(s); ++ vtd_reset_iotlb_locked(s); ++ vtd_reset_context_cache_locked(s); ++ vtd_iommu_unlock(s); ++} ++ + static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id, + uint32_t level) + { +@@ -3160,10 +3168,7 @@ static void vtd_init(IntelIOMMUState *s) + s->cap |= VTD_CAP_CM; + } + +- vtd_iommu_lock(s); +- vtd_reset_context_cache_locked(s); +- vtd_reset_iotlb_locked(s); +- vtd_iommu_unlock(s); ++ vtd_reset_caches(s); + + /* Define registers with default values and bit semantics */ + vtd_define_long(s, DMAR_VER_REG, 0x10UL, 0, 0); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-intel_iommu-move-ce-fetching-out-when-sync-shadow.patch b/SOURCES/kvm-intel_iommu-move-ce-fetching-out-when-sync-shadow.patch new file mode 100644 index 0000000..07a7501 --- /dev/null +++ b/SOURCES/kvm-intel_iommu-move-ce-fetching-out-when-sync-shadow.patch @@ -0,0 +1,109 @@ +From 4084168a694381238dadf1f5c0cc4af756ac883f Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:37 +0000 +Subject: [PATCH 09/35] intel_iommu: move ce fetching out when sync shadow + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-7-peterx@redhat.com> +Patchwork-id: 82965 +O-Subject: [RHEL-8 qemu-kvm PATCH 6/7] intel_iommu: move ce fetching out when sync shadow +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Laurent Vivier + +Bugzilla: 1629616 + +There are two callers for vtd_sync_shadow_page_table_range(): one +provided a valid context entry and one not. Move that fetching +operation into the caller vtd_sync_shadow_page_table() where we need to +fetch the context entry. + +Meanwhile, remove the error_report_once() directly since we're already +tracing all the error cases in the previous call. Instead, return error +number back to caller. This will not change anything functional since +callers are dropping it after all. + +We do this move majorly because we want to do something more later in +vtd_sync_shadow_page_table(). + +Signed-off-by: Peter Xu +Reviewed-by: Eric Auger +Reviewed-by: Maxime Coquelin +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 95ecd3df7815b4bc4f9a0f47e1c64d81434715aa) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/intel_iommu.c | 41 +++++++++++++---------------------------- + 1 file changed, 13 insertions(+), 28 deletions(-) + +diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c +index a6e87a9..c95128d 100644 +--- a/hw/i386/intel_iommu.c ++++ b/hw/i386/intel_iommu.c +@@ -1045,7 +1045,6 @@ static int vtd_sync_shadow_page_hook(IOMMUTLBEntry *entry, + return 0; + } + +-/* If context entry is NULL, we'll try to fetch it on our own. */ + static int vtd_sync_shadow_page_table_range(VTDAddressSpace *vtd_as, + VTDContextEntry *ce, + hwaddr addr, hwaddr size) +@@ -1057,39 +1056,25 @@ static int vtd_sync_shadow_page_table_range(VTDAddressSpace *vtd_as, + .notify_unmap = true, + .aw = s->aw_bits, + .as = vtd_as, ++ .domain_id = VTD_CONTEXT_ENTRY_DID(ce->hi), + }; +- VTDContextEntry ce_cache; +- int ret; +- +- if (ce) { +- /* If the caller provided context entry, use it */ +- ce_cache = *ce; +- } else { +- /* If the caller didn't provide ce, try to fetch */ +- ret = vtd_dev_to_context_entry(s, pci_bus_num(vtd_as->bus), +- vtd_as->devfn, &ce_cache); +- if (ret) { +- /* +- * This should not really happen, but in case it happens, +- * we just skip the sync for this time. After all we even +- * don't have the root table pointer! +- */ +- error_report_once("%s: invalid context entry for bus 0x%x" +- " devfn 0x%x", +- __func__, pci_bus_num(vtd_as->bus), +- vtd_as->devfn); +- return 0; +- } +- } + +- info.domain_id = VTD_CONTEXT_ENTRY_DID(ce_cache.hi); +- +- return vtd_page_walk(&ce_cache, addr, addr + size, &info); ++ return vtd_page_walk(ce, addr, addr + size, &info); + } + + static int vtd_sync_shadow_page_table(VTDAddressSpace *vtd_as) + { +- return vtd_sync_shadow_page_table_range(vtd_as, NULL, 0, UINT64_MAX); ++ int ret; ++ VTDContextEntry ce; ++ ++ ret = vtd_dev_to_context_entry(vtd_as->iommu_state, ++ pci_bus_num(vtd_as->bus), ++ vtd_as->devfn, &ce); ++ if (ret) { ++ return ret; ++ } ++ ++ return vtd_sync_shadow_page_table_range(vtd_as, &ce, 0, UINT64_MAX); + } + + /* +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-169-add-cases-for-source-vm-resuming.patch b/SOURCES/kvm-iotests-169-add-cases-for-source-vm-resuming.patch new file mode 100644 index 0000000..b7dd100 --- /dev/null +++ b/SOURCES/kvm-iotests-169-add-cases-for-source-vm-resuming.patch @@ -0,0 +1,129 @@ +From e56ee6a66cc5452a46426b11d642da792b7ee75d Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:28 +0000 +Subject: [PATCH 34/35] iotests: 169: add cases for source vm resuming + +RH-Author: John Snow +Message-id: <20181120181828.15132-25-jsnow@redhat.com> +Patchwork-id: 83068 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 24/24] iotests: 169: add cases for source vm resuming +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Test that we can resume source vm after [failed] migration, and bitmaps +are ok. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +(cherry picked from commit 3e6d88f280a53b5b399e73b1f80efe4c3db306f1) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/169 | 60 +++++++++++++++++++++++++++++++++++++++++++++- + tests/qemu-iotests/169.out | 4 ++-- + 2 files changed, 61 insertions(+), 3 deletions(-) + +diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 +index 8b7947d..69850c4 100755 +--- a/tests/qemu-iotests/169 ++++ b/tests/qemu-iotests/169 +@@ -77,6 +77,58 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): + self.assert_qmp(result, 'error/desc', + "Dirty bitmap 'bitmap0' not found"); + ++ def do_test_migration_resume_source(self, persistent, migrate_bitmaps): ++ granularity = 512 ++ ++ # regions = ((start, count), ...) ++ regions = ((0, 0x10000), ++ (0xf0000, 0x10000), ++ (0xa0201, 0x1000)) ++ ++ mig_caps = [{'capability': 'events', 'state': True}] ++ if migrate_bitmaps: ++ mig_caps.append({'capability': 'dirty-bitmaps', 'state': True}) ++ ++ result = self.vm_a.qmp('migrate-set-capabilities', ++ capabilities=mig_caps) ++ self.assert_qmp(result, 'return', {}) ++ ++ self.add_bitmap(self.vm_a, granularity, persistent) ++ for r in regions: ++ self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % r) ++ sha256 = self.get_bitmap_hash(self.vm_a) ++ ++ result = self.vm_a.qmp('migrate', uri=mig_cmd) ++ while True: ++ event = self.vm_a.event_wait('MIGRATION') ++ if event['data']['status'] == 'completed': ++ break ++ ++ # test that bitmap is still here ++ removed = (not migrate_bitmaps) and persistent ++ self.check_bitmap(self.vm_a, False if removed else sha256) ++ ++ self.vm_a.qmp('cont') ++ ++ # test that bitmap is still here after invalidation ++ self.check_bitmap(self.vm_a, sha256) ++ ++ # shutdown and check that invalidation didn't fail ++ self.vm_a.shutdown() ++ ++ # catch 'Could not reopen qcow2 layer: Bitmap already exists' ++ # possible error ++ log = self.vm_a.get_log() ++ log = re.sub(r'^\[I \d+\.\d+\] OPENED\n', '', log) ++ log = re.sub(r'^(wrote .* bytes at offset .*\n.*KiB.*ops.*sec.*\n){3}', ++ '', log) ++ log = re.sub(r'\[I \+\d+\.\d+\] CLOSED\n?$', '', log) ++ self.assertEqual(log, '') ++ ++ # test that bitmap is still persistent ++ self.vm_a.launch() ++ self.check_bitmap(self.vm_a, sha256 if persistent else False) ++ + def do_test_migration(self, persistent, migrate_bitmaps, online, + shared_storage): + granularity = 512 +@@ -152,7 +204,7 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): + + def inject_test_case(klass, name, method, *args, **kwargs): + mc = operator.methodcaller(method, *args, **kwargs) +- setattr(klass, 'test_' + name, new.instancemethod(mc, None, klass)) ++ setattr(klass, 'test_' + method + name, new.instancemethod(mc, None, klass)) + + for cmb in list(itertools.product((True, False), repeat=4)): + name = ('_' if cmb[0] else '_not_') + 'persistent_' +@@ -163,6 +215,12 @@ for cmb in list(itertools.product((True, False), repeat=4)): + inject_test_case(TestDirtyBitmapMigration, name, 'do_test_migration', + *list(cmb)) + ++for cmb in list(itertools.product((True, False), repeat=2)): ++ name = ('_' if cmb[0] else '_not_') + 'persistent_' ++ name += ('_' if cmb[1] else '_not_') + 'migbitmap' ++ ++ inject_test_case(TestDirtyBitmapMigration, name, ++ 'do_test_migration_resume_source', *list(cmb)) + + if __name__ == '__main__': + iotests.main(supported_fmts=['qcow2']) +diff --git a/tests/qemu-iotests/169.out b/tests/qemu-iotests/169.out +index b6f2576..3a89159 100644 +--- a/tests/qemu-iotests/169.out ++++ b/tests/qemu-iotests/169.out +@@ -1,5 +1,5 @@ +-................ ++.................... + ---------------------------------------------------------------------- +-Ran 16 tests ++Ran 20 tests + + OK +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-169-drop-deprecated-autoload-parameter.patch b/SOURCES/kvm-iotests-169-drop-deprecated-autoload-parameter.patch new file mode 100644 index 0000000..c99b48c --- /dev/null +++ b/SOURCES/kvm-iotests-169-drop-deprecated-autoload-parameter.patch @@ -0,0 +1,41 @@ +From 09ea133ca7aedcadbd674dc8372e04a3fe5e7d33 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:23 +0000 +Subject: [PATCH 29/35] iotests: 169: drop deprecated 'autoload' parameter + +RH-Author: John Snow +Message-id: <20181120181828.15132-20-jsnow@redhat.com> +Patchwork-id: 83075 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 19/24] iotests: 169: drop deprecated 'autoload' parameter +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +Signed-off-by: John Snow +(cherry picked from commit 304cc429a07eb6601020212a478050ebbe87df88) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/169 | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 +index f243db9..df408f8 100755 +--- a/tests/qemu-iotests/169 ++++ b/tests/qemu-iotests/169 +@@ -58,7 +58,6 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): + 'granularity': granularity} + if persistent: + params['persistent'] = True +- params['autoload'] = True + + result = vm.qmp('block-dirty-bitmap-add', **params) + self.assert_qmp(result, 'return', {}); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-222-Don-t-run-with-luks.patch b/SOURCES/kvm-iotests-222-Don-t-run-with-luks.patch new file mode 100644 index 0000000..6f27b27 --- /dev/null +++ b/SOURCES/kvm-iotests-222-Don-t-run-with-luks.patch @@ -0,0 +1,44 @@ +From aa10e475ab14f9365d52f08eaee0155c5f4aa634 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:54 +0200 +Subject: [PATCH 236/268] iotests: 222: Don't run with luks + +RH-Author: John Snow +Message-id: <20180718225511.14878-19-jsnow@redhat.com> +Patchwork-id: 81422 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 18/35] iotests: 222: Don't run with luks +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Fam Zheng + +Luks needs special parameters to operate the image. Since this test is +focusing on image fleecing, skip skip that format. + +Signed-off-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit e79c4cd1909c05a2cab6517a9c00445bd2d880a6) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/222 | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 +index ff3bfc1..0ead56d 100644 +--- a/tests/qemu-iotests/222 ++++ b/tests/qemu-iotests/222 +@@ -25,6 +25,8 @@ import iotests + from iotests import log, qemu_img, qemu_io, qemu_io_silent + + iotests.verify_platform(['linux']) ++iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk', ++ 'vhdx', 'raw']) + + patterns = [("0x5d", "0", "64k"), + ("0xd5", "1M", "64k"), +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-case-for-a-corrupted-inactive-image.patch b/SOURCES/kvm-iotests-Add-case-for-a-corrupted-inactive-image.patch new file mode 100644 index 0000000..51e9939 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-case-for-a-corrupted-inactive-image.patch @@ -0,0 +1,95 @@ +From 589a3182bf175c8e83e7910d11810610fb3134a2 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 18:00:55 +0200 +Subject: [PATCH 058/268] iotests: Add case for a corrupted inactive image + +RH-Author: Max Reitz +Message-id: <20180618180055.22739-4-mreitz@redhat.com> +Patchwork-id: 80794 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/3] iotests: Add case for a corrupted inactive image +Bugzilla: 1588039 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +Reviewed-by: John Snow +Tested-by: Jeff Cody +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +Message-id: 20180606193702.7113-4-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit c50abd175a88cd41c2c08339de91f6f6e4a7b162) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/060 | 30 ++++++++++++++++++++++++++++++ + tests/qemu-iotests/060.out | 14 ++++++++++++++ + 2 files changed, 44 insertions(+) + +diff --git a/tests/qemu-iotests/060 b/tests/qemu-iotests/060 +index 6c7407f..7bdf609 100755 +--- a/tests/qemu-iotests/060 ++++ b/tests/qemu-iotests/060 +@@ -440,6 +440,36 @@ echo "{'execute': 'qmp_capabilities'} + -drive if=none,node-name=drive,file="$TEST_IMG",driver=qcow2 \ + | _filter_qmp | _filter_qemu_io + ++echo ++echo "=== Testing incoming inactive corrupted image ===" ++echo ++ ++_make_test_img 64M ++# Create an unaligned L1 entry, so qemu will signal a corruption when ++# reading from the covered area ++poke_file "$TEST_IMG" "$l1_offset" "\x00\x00\x00\x00\x2a\x2a\x2a\x2a" ++ ++# Inactive images are effectively read-only images, so this should be a ++# non-fatal corruption (which does not modify the image) ++echo "{'execute': 'qmp_capabilities'} ++ {'execute': 'human-monitor-command', ++ 'arguments': {'command-line': 'qemu-io drive \"read 0 512\"'}} ++ {'execute': 'quit'}" \ ++ | $QEMU -qmp stdio -nographic -nodefaults \ ++ -blockdev "{'node-name': 'drive', ++ 'driver': 'qcow2', ++ 'file': { ++ 'driver': 'file', ++ 'filename': '$TEST_IMG' ++ }}" \ ++ -incoming exec:'cat /dev/null' \ ++ 2>&1 \ ++ | _filter_qmp | _filter_qemu_io ++ ++echo ++# Image should not have been marked corrupt ++_img_info --format-specific | grep 'corrupt:' ++ + # success, all done + echo "*** done" + rm -f $seq.full +diff --git a/tests/qemu-iotests/060.out b/tests/qemu-iotests/060.out +index 25d5c39..99234fb 100644 +--- a/tests/qemu-iotests/060.out ++++ b/tests/qemu-iotests/060.out +@@ -420,4 +420,18 @@ write failed: Input/output error + {"return": ""} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++ ++=== Testing incoming inactive corrupted image === ++ ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 ++QMP_VERSION ++{"return": {}} ++qcow2: Image is corrupt: L2 table offset 0x2a2a2a00 unaligned (L1 index: 0); further non-fatal corruption events will be suppressed ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_IMAGE_CORRUPTED", "data": {"device": "", "msg": "L2 table offset 0x2a2a2a00 unaligned (L1 index: 0)", "node-name": "drive", "fatal": false}} ++read failed: Input/output error ++{"return": ""} ++{"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++ ++ corrupt: false + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-creation-test-to-153.patch b/SOURCES/kvm-iotests-Add-creation-test-to-153.patch new file mode 100644 index 0000000..fa45041 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-creation-test-to-153.patch @@ -0,0 +1,101 @@ +From 3dba07a26692093927c017046a8ea10f94892e67 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:53:37 +0200 +Subject: [PATCH 156/268] iotests: Add creation test to 153 + +RH-Author: Max Reitz +Message-id: <20180618145337.633-4-mreitz@redhat.com> +Patchwork-id: 80752 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/3] iotests: Add creation test to 153 +Bugzilla: 1519144 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng +RH-Acked-by: Miroslav Rezanina + +This patch adds a test case to 153 which tries to overwrite an image +(using qemu-img create) while it is in use. Without the original user +explicitly sharing the necessary permissions (writing and truncation), +this should not be allowed. + +Signed-off-by: Max Reitz +Reviewed-by: Fam Zheng +Message-id: 20180509215336.31304-4-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit f45b638f9f967cdbea4e24704bd16a858ddcde03) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/153 | 18 ++++++++++++++++++ + tests/qemu-iotests/153.out | 13 +++++++++++++ + 2 files changed, 31 insertions(+) + +diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 +index ec508c7..673813c 100755 +--- a/tests/qemu-iotests/153 ++++ b/tests/qemu-iotests/153 +@@ -137,6 +137,24 @@ for opts1 in "" "read-only=on" "read-only=on,force-share=on"; do + _run_cmd $QEMU_IMG dd $L if="${TEST_IMG}" of="${TEST_IMG}.convert" bs=512 count=1 + _run_cmd $QEMU_IMG bench $L -c 1 "${TEST_IMG}" + _run_cmd $QEMU_IMG bench $L -w -c 1 "${TEST_IMG}" ++ ++ # qemu-img create does not support -U ++ if [ -z "$L" ]; then ++ _run_cmd $QEMU_IMG create -f $IMGFMT "${TEST_IMG}" \ ++ -b ${TEST_IMG}.base ++ # Read the file format. It used to be the case that ++ # file-posix simply truncated the file, but the qcow2 ++ # driver then failed to format it because it was unable ++ # to acquire the necessary WRITE permission. However, the ++ # truncation was already wrong, and the whole process ++ # resulted in the file being completely empty and thus its ++ # format would be detected to be raw. ++ # So we read it here to see that creation either completed ++ # successfully (thus the format is qcow2) or it aborted ++ # before the file was changed at all (thus the format stays ++ # qcow2). ++ _img_info -U | grep 'file format' ++ fi + done + _send_qemu_cmd $h "{ 'execute': 'quit', }" "" + echo +diff --git a/tests/qemu-iotests/153.out b/tests/qemu-iotests/153.out +index 2510762..3492ba7 100644 +--- a/tests/qemu-iotests/153.out ++++ b/tests/qemu-iotests/153.out +@@ -92,6 +92,11 @@ _qemu_img_wrapper bench -w -c 1 TEST_DIR/t.qcow2 + qemu-img: Could not open 'TEST_DIR/t.qcow2': Failed to get "write" lock + Is another process using the image? + ++_qemu_img_wrapper create -f qcow2 TEST_DIR/t.qcow2 -b TEST_DIR/t.qcow2.base ++qemu-img: TEST_DIR/t.qcow2: Failed to get "write" lock ++Is another process using the image? ++file format: IMGFMT ++ + == Running utility commands -U == + + _qemu_io_wrapper -U -c read 0 512 TEST_DIR/t.qcow2 +@@ -209,6 +214,11 @@ _qemu_img_wrapper bench -w -c 1 TEST_DIR/t.qcow2 + qemu-img: Could not open 'TEST_DIR/t.qcow2': Failed to get "write" lock + Is another process using the image? + ++_qemu_img_wrapper create -f qcow2 TEST_DIR/t.qcow2 -b TEST_DIR/t.qcow2.base ++qemu-img: TEST_DIR/t.qcow2: Failed to get "write" lock ++Is another process using the image? ++file format: IMGFMT ++ + == Running utility commands -U == + + _qemu_io_wrapper -U -c read 0 512 TEST_DIR/t.qcow2 +@@ -309,6 +319,9 @@ _qemu_img_wrapper bench -c 1 TEST_DIR/t.qcow2 + + _qemu_img_wrapper bench -w -c 1 TEST_DIR/t.qcow2 + ++_qemu_img_wrapper create -f qcow2 TEST_DIR/t.qcow2 -b TEST_DIR/t.qcow2.base ++file format: IMGFMT ++ + == Running utility commands -U == + + _qemu_io_wrapper -U -c read 0 512 TEST_DIR/t.qcow2 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-failure-matching-to-common.qemu.patch b/SOURCES/kvm-iotests-Add-failure-matching-to-common.qemu.patch new file mode 100644 index 0000000..1d1a2f7 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-failure-matching-to-common.qemu.patch @@ -0,0 +1,156 @@ +From e56b23f0bbf8c9c1149cfb4f0a1a3b002b5aec1b Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 20:50:59 +0100 +Subject: [PATCH 2/3] iotests: Add failure matching to common.qemu + +RH-Author: John Snow +Message-id: <20181010205100.17689-3-jsnow@redhat.com> +Patchwork-id: 82633 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/3] iotests: Add failure matching to common.qemu +Bugzilla: 1635583 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +From: Max Reitz + +Currently, common.qemu only allows to match for results indicating +success. The only way to fail is by provoking a timeout. However, +sometimes we do have a defined failure output and can match for that, +which saves us from having to wait for the timeout in case of failure. +Because failure can sometimes just result in a _notrun in the test, it +is actually important to care about being able to fail quickly. + +Also, sometimes we simply do not get any specific output in case of +success. The only way to handle this currently would be to define an +error message as the string to look for, which means that actual success +results in a timeout. This is really bad because it unnecessarily slows +down a succeeding test. + +Therefore, this patch adds a new parameter $success_or_failure to +_timed_wait_for and _send_qemu_cmd. Setting this to a non-empty string +makes both commands expect two match parameters: If the first matches, +the function succeeds. If the second matches, the function fails. + +Signed-off-by: Max Reitz +Message-id: 20180406151731.4285-2-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 81c6ddf49a76a663cea16c07a07d51b67c853209) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/common.qemu | 58 +++++++++++++++++++++++++++++++++++++----- + 1 file changed, 51 insertions(+), 7 deletions(-) + +diff --git a/tests/qemu-iotests/common.qemu b/tests/qemu-iotests/common.qemu +index 85f66b8..f285484 100644 +--- a/tests/qemu-iotests/common.qemu ++++ b/tests/qemu-iotests/common.qemu +@@ -52,11 +52,29 @@ _in_fd=4 + # response is not echoed out. + # If $mismatch_only is set, only non-matching responses will + # be echoed. ++# ++# If $success_or_failure is set, the meaning of the arguments is ++# changed as follows: ++# $2: A string to search for in the response; if found, this indicates ++# success and ${QEMU_STATUS[$1]} is set to 0. ++# $3: A string to search for in the response; if found, this indicates ++# failure and the test is either aborted (if $qemu_error_no_exit ++# is not set) or ${QEMU_STATUS[$1]} is set to -1 (otherwise). + function _timed_wait_for() + { + local h=${1} + shift + ++ if [ -z "${success_or_failure}" ]; then ++ success_match=${*} ++ failure_match= ++ else ++ success_match=${1} ++ failure_match=${2} ++ fi ++ ++ timeout=yes ++ + QEMU_STATUS[$h]=0 + while IFS= read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]} + do +@@ -64,10 +82,18 @@ function _timed_wait_for() + echo "${resp}" | _filter_testdir | _filter_qemu \ + | _filter_qemu_io | _filter_qmp | _filter_hmp + fi +- grep -q "${*}" < <(echo "${resp}") ++ if [ -n "${failure_match}" ]; then ++ grep -q "${failure_match}" < <(echo "${resp}") ++ if [ $? -eq 0 ]; then ++ timeout= ++ break ++ fi ++ fi ++ grep -q "${success_match}" < <(echo "${resp}") + if [ $? -eq 0 ]; then + return +- elif [ -z "${silent}" ] && [ -n "${mismatch_only}" ]; then ++ fi ++ if [ -z "${silent}" ] && [ -n "${mismatch_only}" ]; then + echo "${resp}" | _filter_testdir | _filter_qemu \ + | _filter_qemu_io | _filter_qmp | _filter_hmp + fi +@@ -75,8 +101,12 @@ function _timed_wait_for() + done + QEMU_STATUS[$h]=-1 + if [ -z "${qemu_error_no_exit}" ]; then +- echo "Timeout waiting for ${*} on handle ${h}" +- exit 1 # Timeout means the test failed ++ if [ -n "${timeout}" ]; then ++ echo "Timeout waiting for ${success_match} on handle ${h}" ++ else ++ echo "Wrong response matching ${failure_match} on handle ${h}" ++ fi ++ exit 1 # Timeout or wrong match mean the test failed + fi + } + +@@ -96,6 +126,11 @@ function _timed_wait_for() + # If $qemu_error_no_exit is set, then even if the expected response + # is not seen, we will not exit. $QEMU_STATUS[$1] will be set it -1 in + # that case. ++# ++# If $success_or_failure is set, then the last two strings are the ++# strings the response will be scanned for. The first of the two ++# indicates success, the latter indicates failure. Failure is handled ++# like a timeout. + function _send_qemu_cmd() + { + local h=${1} +@@ -109,14 +144,23 @@ function _send_qemu_cmd() + use_error="no" + fi + # This array element extraction is done to accommodate pathnames with spaces +- cmd=${@: 1:${#@}-1} +- shift $(($# - 1)) ++ if [ -z "${success_or_failure}" ]; then ++ cmd=${@: 1:${#@}-1} ++ shift $(($# - 1)) ++ else ++ cmd=${@: 1:${#@}-2} ++ shift $(($# - 2)) ++ fi + + while [ ${count} -gt 0 ] + do + echo "${cmd}" >&${QEMU_IN[${h}]} + if [ -n "${1}" ]; then +- qemu_error_no_exit=${use_error} _timed_wait_for ${h} "${1}" ++ if [ -z "${success_or_failure}" ]; then ++ qemu_error_no_exit=${use_error} _timed_wait_for ${h} "${1}" ++ else ++ qemu_error_no_exit=${use_error} _timed_wait_for ${h} "${1}" "${2}" ++ fi + if [ ${QEMU_STATUS[$h]} -eq 0 ]; then + return + fi +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-test-221-to-catch-qemu-img-map-regressio.patch b/SOURCES/kvm-iotests-Add-test-221-to-catch-qemu-img-map-regressio.patch new file mode 100644 index 0000000..f43b069 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-test-221-to-catch-qemu-img-map-regressio.patch @@ -0,0 +1,142 @@ +From 8e5660a0047bcefbab18d5996acb4da2a1c298ae Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 15 Oct 2018 17:44:46 +0100 +Subject: [PATCH 49/49] iotests: Add test 221 to catch qemu-img map regression + +RH-Author: Max Reitz +Message-id: <20181015174446.31974-3-mreitz@redhat.com> +Patchwork-id: 82707 +O-Subject: [RHEL-8 qemu-kvm PATCH v2 2/2] iotests: Add test 221 to catch qemu-img map regression +Bugzilla: 1639374 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Thomas Huth + +From: Eric Blake + +Although qemu-img creates aligned files (by rounding up), it +must also gracefully handle files that are not sector-aligned. +Test that the bug fixed in the previous patch does not recur. + +It's a bit annoying that we can see the (implicit) hole past +the end of the file on to the next sector boundary, so if we +ever reach the point where we report a byte-accurate size rather +than our current behavior of always rounding up, this test will +probably need a slight modification. + +Signed-off-by: Eric Blake +Signed-off-by: Kevin Wolf +(cherry picked from commit c6a9d2f6f9bc0c163b3a3073126464a2446bad5f) +Signed-off-by: Max Reitz +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/221 | 60 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/221.out | 16 +++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 77 insertions(+) + create mode 100755 tests/qemu-iotests/221 + create mode 100644 tests/qemu-iotests/221.out + +diff --git a/tests/qemu-iotests/221 b/tests/qemu-iotests/221 +new file mode 100755 +index 0000000..41c4e4b +--- /dev/null ++++ b/tests/qemu-iotests/221 +@@ -0,0 +1,60 @@ ++#!/bin/bash ++# ++# Test qemu-img vs. unaligned images ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++seq="$(basename $0)" ++echo "QA output created by $seq" ++ ++here="$PWD" ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ _cleanup_test_img ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++ ++_supported_fmt raw ++_supported_proto file ++_supported_os Linux ++ ++echo ++echo "=== Check mapping of unaligned raw image ===" ++echo ++ ++_make_test_img 43009 # qemu-img create rounds size up ++$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map ++ ++truncate --size=43009 "$TEST_IMG" # so we resize it and check again ++$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map ++ ++$QEMU_IO -c 'w 43008 1' "$TEST_IMG" | _filter_qemu_io # writing also rounds up ++$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map ++ ++truncate --size=43009 "$TEST_IMG" # so we resize it and check again ++$QEMU_IMG map --output=json "$TEST_IMG" | _filter_qemu_img_map ++ ++# success, all done ++echo '*** done' ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/221.out b/tests/qemu-iotests/221.out +new file mode 100644 +index 0000000..a9c0190 +--- /dev/null ++++ b/tests/qemu-iotests/221.out +@@ -0,0 +1,16 @@ ++QA output created by 221 ++ ++=== Check mapping of unaligned raw image === ++ ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=43009 ++[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] ++[{ "start": 0, "length": 43520, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] ++wrote 1/1 bytes at offset 43008 ++1 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, ++{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, ++{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] ++[{ "start": 0, "length": 40960, "depth": 0, "zero": true, "data": false, "offset": OFFSET}, ++{ "start": 40960, "length": 2049, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, ++{ "start": 43009, "length": 511, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 0242b2f..1cb2ccb 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -218,6 +218,7 @@ + 217 rw auto quick + 218 rw auto quick + 219 rw auto ++221 rw auto quick + 222 rw auto quick + 223 rw auto quick + 226 auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-test-for-COR-across-nodes.patch b/SOURCES/kvm-iotests-Add-test-for-COR-across-nodes.patch new file mode 100644 index 0000000..ee493a4 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-test-for-COR-across-nodes.patch @@ -0,0 +1,214 @@ +From a24ff99564c5906ef3826a4e41f483d0cfb97562 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:12 +0200 +Subject: [PATCH 045/268] iotests: Add test for COR across nodes + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-11-mreitz@redhat.com> +Patchwork-id: 80771 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 10/10] iotests: Add test for COR across nodes +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +COR across nodes (that is, you have some filter node between the +actually COR target and the node that performs the COR) cannot reliably +work together with the permission system when there is no explicit COR +node that can request the WRITE_UNCHANGED permission for its child. +This is because COR (currently) sneaks its requests by the usual +permission checks, so it can work without a WRITE* permission; but if +there is a filter node in between, that will re-issue the request, which +then passes through the usual check -- and if nobody has requested a +WRITE_UNCHANGED permission, that check will fail. + +There is no real direct fix apart from hoping that there is someone who +has requested that permission; in case of just the qemu-io HMP command +(and no guest device), however, that is not the case. The real real fix +is to implement the copy-on-read flag through an implicitly added COR +node. Such a node can request the necessary permissions as shown in +this test. + +Signed-off-by: Max Reitz +Message-id: 20180421132929.21610-10-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 3e7a95feb9b5d66cff7fee38b3c423135ed245f6) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/216 | 115 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/216.out | 28 +++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 144 insertions(+) + create mode 100755 tests/qemu-iotests/216 + create mode 100644 tests/qemu-iotests/216.out + +diff --git a/tests/qemu-iotests/216 b/tests/qemu-iotests/216 +new file mode 100755 +index 0000000..ca9b47a +--- /dev/null ++++ b/tests/qemu-iotests/216 +@@ -0,0 +1,115 @@ ++#!/usr/bin/env python ++# ++# Copy-on-read tests using a COR filter node ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++# Creator/Owner: Max Reitz ++ ++import iotests ++from iotests import log, qemu_img_pipe, qemu_io, filter_qemu_io ++ ++# Need backing file support ++iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk']) ++iotests.verify_platform(['linux']) ++ ++log('') ++log('=== Copy-on-read across nodes ===') ++log('') ++ ++# The old copy-on-read mechanism without a filter node cannot request ++# WRITE_UNCHANGED permissions for its child. Therefore it just tries ++# to sneak its write by the usual permission system and holds its ++# fingers crossed. However, that sneaking does not work so well when ++# there is a filter node in the way: That will receive the write ++# request and re-issue a new one to its child, which this time is a ++# proper write request that will make the permission system cough -- ++# unless there is someone at the top (like a guest device) that has ++# requested write permissions. ++# ++# A COR filter node, however, can request the proper permissions for ++# its child and therefore is not hit by this issue. ++ ++with iotests.FilePath('base.img') as base_img_path, \ ++ iotests.FilePath('top.img') as top_img_path, \ ++ iotests.VM() as vm: ++ ++ log('--- Setting up images ---') ++ log('') ++ ++ qemu_img_pipe('create', '-f', iotests.imgfmt, base_img_path, '64M') ++ ++ log(filter_qemu_io(qemu_io(base_img_path, '-c', 'write -P 1 0M 1M'))) ++ ++ qemu_img_pipe('create', '-f', iotests.imgfmt, '-b', base_img_path, ++ top_img_path) ++ ++ log(filter_qemu_io(qemu_io(top_img_path, '-c', 'write -P 2 1M 1M'))) ++ ++ log('') ++ log('--- Doing COR ---') ++ log('') ++ ++ # Compare with e.g. the following: ++ # vm.add_drive_raw('if=none,node-name=node0,copy-on-read=on,driver=raw,' \ ++ # 'file.driver=%s,file.file.filename=%s' % ++ # (iotests.imgfmt, top_img_path)) ++ # (Remove the blockdev-add instead.) ++ # ((Not tested here because it hits an assertion in the permission ++ # system.)) ++ ++ vm.launch() ++ ++ log(vm.qmp('blockdev-add', ++ node_name='node0', ++ driver='copy-on-read', ++ file={ ++ 'driver': 'raw', ++ 'file': { ++ 'driver': 'copy-on-read', ++ 'file': { ++ 'driver': 'raw', ++ 'file': { ++ 'driver': iotests.imgfmt, ++ 'file': { ++ 'driver': 'file', ++ 'filename': top_img_path ++ }, ++ 'backing': { ++ 'driver': iotests.imgfmt, ++ 'file': { ++ 'driver': 'file', ++ 'filename': base_img_path ++ } ++ } ++ } ++ } ++ } ++ })) ++ ++ # Trigger COR ++ log(vm.qmp('human-monitor-command', ++ command_line='qemu-io node0 "read 0 64M"')) ++ ++ vm.shutdown() ++ ++ log('') ++ log('--- Checking COR result ---') ++ log('') ++ ++ log(filter_qemu_io(qemu_io(base_img_path, '-c', 'discard 0 64M'))) ++ log(filter_qemu_io(qemu_io(top_img_path, '-c', 'read -P 1 0M 1M'))) ++ log(filter_qemu_io(qemu_io(top_img_path, '-c', 'read -P 2 1M 1M'))) +diff --git a/tests/qemu-iotests/216.out b/tests/qemu-iotests/216.out +new file mode 100644 +index 0000000..d3fc590 +--- /dev/null ++++ b/tests/qemu-iotests/216.out +@@ -0,0 +1,28 @@ ++ ++=== Copy-on-read across nodes === ++ ++--- Setting up images --- ++ ++wrote 1048576/1048576 bytes at offset 0 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++wrote 1048576/1048576 bytes at offset 1048576 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++ ++--- Doing COR --- ++ ++{u'return': {}} ++{u'return': u''} ++ ++--- Checking COR result --- ++ ++discard 67108864/67108864 bytes at offset 0 ++64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++read 1048576/1048576 bytes at offset 0 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++read 1048576/1048576 bytes at offset 1048576 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index cd5d26c..d228008 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -214,4 +214,5 @@ + 213 rw auto quick + 214 rw auto + 215 rw auto quick ++216 rw auto quick + 218 rw auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-test-for-U-force-share-conflicts.patch b/SOURCES/kvm-iotests-Add-test-for-U-force-share-conflicts.patch new file mode 100644 index 0000000..da1197d --- /dev/null +++ b/SOURCES/kvm-iotests-Add-test-for-U-force-share-conflicts.patch @@ -0,0 +1,82 @@ +From d7e3dac2030556f68038b049b4bd6deafee43808 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:31:06 +0200 +Subject: [PATCH 048/268] iotests: Add test for -U/force-share conflicts + +RH-Author: Max Reitz +Message-id: <20180618163106.23010-4-mreitz@redhat.com> +Patchwork-id: 80773 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/3] iotests: Add test for -U/force-share conflicts +Bugzilla: 1576598 +RH-Acked-by: Kevin Wolf +RH-Acked-by: Fam Zheng +RH-Acked-by: Miroslav Rezanina + +Signed-off-by: Max Reitz +Message-id: 20180502202051.15493-4-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 4e7d73c5fbd97e55ffe5af02f24d1f7dbe3bbf20) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/153 | 17 +++++++++++++++++ + tests/qemu-iotests/153.out | 16 ++++++++++++++++ + 2 files changed, 33 insertions(+) + +diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 +index a0fd815..ec508c7 100755 +--- a/tests/qemu-iotests/153 ++++ b/tests/qemu-iotests/153 +@@ -242,6 +242,23 @@ _run_cmd $QEMU_IO "${TEST_IMG}" -c 'write 0 512' + + _cleanup_qemu + ++echo ++echo "== Detecting -U and force-share conflicts ==" ++ ++echo ++echo 'No conflict:' ++$QEMU_IMG info -U --image-opts driver=null-co,force-share=on ++echo ++echo 'Conflict:' ++$QEMU_IMG info -U --image-opts driver=null-co,force-share=off ++ ++echo ++echo 'No conflict:' ++$QEMU_IO -c 'open -r -U -o driver=null-co,force-share=on' ++echo ++echo 'Conflict:' ++$QEMU_IO -c 'open -r -U -o driver=null-co,force-share=off' ++ + # success, all done + echo "*** done" + rm -f $seq.full +diff --git a/tests/qemu-iotests/153.out b/tests/qemu-iotests/153.out +index bb721cb..2510762 100644 +--- a/tests/qemu-iotests/153.out ++++ b/tests/qemu-iotests/153.out +@@ -399,4 +399,20 @@ Is another process using the image? + Closing the other + + _qemu_io_wrapper TEST_DIR/t.qcow2 -c write 0 512 ++ ++== Detecting -U and force-share conflicts == ++ ++No conflict: ++image: null-co:// ++file format: null-co ++virtual size: 1.0G (1073741824 bytes) ++disk size: unavailable ++ ++Conflict: ++qemu-img: --force-share/-U conflicts with image options ++ ++No conflict: ++ ++Conflict: ++-U conflicts with image options + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-test-for-cancelling-a-mirror-job.patch b/SOURCES/kvm-iotests-Add-test-for-cancelling-a-mirror-job.patch new file mode 100644 index 0000000..36d8519 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-test-for-cancelling-a-mirror-job.patch @@ -0,0 +1,231 @@ +From 21085d2fd5cc4b08016ac4d16bb90c77bf9551e7 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:47:36 +0200 +Subject: [PATCH 035/268] iotests: Add test for cancelling a mirror job + +RH-Author: Max Reitz +Message-id: <20180618144736.29873-4-mreitz@redhat.com> +Patchwork-id: 80745 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/3] iotests: Add test for cancelling a mirror job +Bugzilla: 1572856 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +We already have an extensive mirror test (041) which does cover +cancelling a mirror job, especially after it has emitted the READY +event. However, it does not check what exact events are emitted after +block-job-cancel is executed. More importantly, it does not use +throttling to ensure that it covers the case of block-job-cancel before +READY. + +It would be possible to add this case to 041, but considering it is +already our largest test file, it makes sense to create a new file for +these cases. + +Signed-off-by: Max Reitz +Message-id: 20180501220509.14152-3-mreitz@redhat.com +Signed-off-by: Jeff Cody +(cherry picked from commit dc885fff972c447f51572afc4c921a26b880731b) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/218 | 138 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/218.out | 30 ++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 169 insertions(+) + create mode 100644 tests/qemu-iotests/218 + create mode 100644 tests/qemu-iotests/218.out + +diff --git a/tests/qemu-iotests/218 b/tests/qemu-iotests/218 +new file mode 100644 +index 0000000..92c331b +--- /dev/null ++++ b/tests/qemu-iotests/218 +@@ -0,0 +1,138 @@ ++#!/usr/bin/env python ++# ++# This test covers what happens when a mirror block job is cancelled ++# in various phases of its existence. ++# ++# Note that this test only checks the emitted events (i.e. ++# BLOCK_JOB_COMPLETED vs. BLOCK_JOB_CANCELLED), it does not compare ++# whether the target is in sync with the source when the ++# BLOCK_JOB_COMPLETED event occurs. This is covered by other tests ++# (such as 041). ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++# Creator/Owner: Max Reitz ++ ++import iotests ++from iotests import log ++ ++iotests.verify_platform(['linux']) ++ ++ ++# Launches the VM, adds two null-co nodes (source and target), and ++# starts a blockdev-mirror job on them. ++# ++# Either both or none of speed and buf_size must be given. ++ ++def start_mirror(vm, speed=None, buf_size=None): ++ vm.launch() ++ ++ ret = vm.qmp('blockdev-add', ++ node_name='source', ++ driver='null-co', ++ size=1048576) ++ assert ret['return'] == {} ++ ++ ret = vm.qmp('blockdev-add', ++ node_name='target', ++ driver='null-co', ++ size=1048576) ++ assert ret['return'] == {} ++ ++ if speed is not None: ++ ret = vm.qmp('blockdev-mirror', ++ job_id='mirror', ++ device='source', ++ target='target', ++ sync='full', ++ speed=speed, ++ buf_size=buf_size) ++ else: ++ ret = vm.qmp('blockdev-mirror', ++ job_id='mirror', ++ device='source', ++ target='target', ++ sync='full') ++ ++ assert ret['return'] == {} ++ ++ ++log('') ++log('=== Cancel mirror job before convergence ===') ++log('') ++ ++log('--- force=false ---') ++log('') ++ ++with iotests.VM() as vm: ++ # Low speed so it does not converge ++ start_mirror(vm, 65536, 65536) ++ ++ log('Cancelling job') ++ log(vm.qmp('block-job-cancel', device='mirror', force=False)) ++ ++ log(vm.event_wait('BLOCK_JOB_CANCELLED'), ++ filters=[iotests.filter_qmp_event]) ++ ++log('') ++log('--- force=true ---') ++log('') ++ ++with iotests.VM() as vm: ++ # Low speed so it does not converge ++ start_mirror(vm, 65536, 65536) ++ ++ log('Cancelling job') ++ log(vm.qmp('block-job-cancel', device='mirror', force=True)) ++ ++ log(vm.event_wait('BLOCK_JOB_CANCELLED'), ++ filters=[iotests.filter_qmp_event]) ++ ++ ++log('') ++log('=== Cancel mirror job after convergence ===') ++log('') ++ ++log('--- force=false ---') ++log('') ++ ++with iotests.VM() as vm: ++ start_mirror(vm) ++ ++ log(vm.event_wait('BLOCK_JOB_READY'), ++ filters=[iotests.filter_qmp_event]) ++ ++ log('Cancelling job') ++ log(vm.qmp('block-job-cancel', device='mirror', force=False)) ++ ++ log(vm.event_wait('BLOCK_JOB_COMPLETED'), ++ filters=[iotests.filter_qmp_event]) ++ ++log('') ++log('--- force=true ---') ++log('') ++ ++with iotests.VM() as vm: ++ start_mirror(vm) ++ ++ log(vm.event_wait('BLOCK_JOB_READY'), ++ filters=[iotests.filter_qmp_event]) ++ ++ log('Cancelling job') ++ log(vm.qmp('block-job-cancel', device='mirror', force=True)) ++ ++ log(vm.event_wait('BLOCK_JOB_CANCELLED'), ++ filters=[iotests.filter_qmp_event]) +diff --git a/tests/qemu-iotests/218.out b/tests/qemu-iotests/218.out +new file mode 100644 +index 0000000..7dbf78e +--- /dev/null ++++ b/tests/qemu-iotests/218.out +@@ -0,0 +1,30 @@ ++ ++=== Cancel mirror job before convergence === ++ ++--- force=false --- ++ ++Cancelling job ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 65536, u'len': 1048576, u'offset': 65536}, u'event': u'BLOCK_JOB_CANCELLED'} ++ ++--- force=true --- ++ ++Cancelling job ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 65536, u'len': 1048576, u'offset': 65536}, u'event': u'BLOCK_JOB_CANCELLED'} ++ ++=== Cancel mirror job after convergence === ++ ++--- force=false --- ++ ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_READY'} ++Cancelling job ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_COMPLETED'} ++ ++--- force=true --- ++ ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_READY'} ++Cancelling job ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'mirror', u'type': u'mirror', u'speed': 0, u'len': 1048576, u'offset': 1048576}, u'event': u'BLOCK_JOB_CANCELLED'} +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 99777ec..3a89aed 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -212,3 +212,4 @@ + 211 rw auto quick + 212 rw auto quick + 213 rw auto quick ++218 rw auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-test-for-qemu-img-convert-C-compatibilit.patch b/SOURCES/kvm-iotests-Add-test-for-qemu-img-convert-C-compatibilit.patch new file mode 100644 index 0000000..cdf7bd4 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-test-for-qemu-img-convert-C-compatibilit.patch @@ -0,0 +1,71 @@ +From 74efb5d0cc7572ee21ee7457b6047fb49181b5b2 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Thu, 17 Jan 2019 19:11:11 +0000 +Subject: [PATCH 14/14] iotests: Add test for 'qemu-img convert -C' + compatibility + +RH-Author: John Snow +Message-id: <20190117191111.30782-3-jsnow@redhat.com> +Patchwork-id: 84042 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/2] iotests: Add test for 'qemu-img convert -C' compatibility +Bugzilla: 1623082 +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Laurent Vivier + +From: Fam Zheng + +Signed-off-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit 8ba4f10fa689251facd483c3ee0ef4dd4e9bec53) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/082 | 8 ++++++++ + tests/qemu-iotests/082.out | 11 +++++++++++ + 2 files changed, 19 insertions(+) + +diff --git a/tests/qemu-iotests/082 b/tests/qemu-iotests/082 +index a872f77..3e605d5 100755 +--- a/tests/qemu-iotests/082 ++++ b/tests/qemu-iotests/082 +@@ -158,6 +158,14 @@ run_qemu_img convert -o help + run_qemu_img convert -O bochs -o help + + echo ++echo === convert: -C and other options === ++ ++# Adding the help option to a command without other -o options ++run_qemu_img convert -C -S 4k -O $IMGFMT "$TEST_IMG" "$TEST_IMG".target ++run_qemu_img convert -C -S 8k -O $IMGFMT "$TEST_IMG" "$TEST_IMG".target ++run_qemu_img convert -C -c -O $IMGFMT "$TEST_IMG" "$TEST_IMG".target ++ ++echo + echo === amend: Options specified more than once === + + # Last -f should win +diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out +index 60ef87c..19e9fb1 100644 +--- a/tests/qemu-iotests/082.out ++++ b/tests/qemu-iotests/082.out +@@ -508,6 +508,17 @@ size Virtual disk size + Testing: convert -O bochs -o help + qemu-img: Format driver 'bochs' does not support image creation + ++=== convert: -C and other options === ++ ++Testing: convert -C -S 4k -O qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.target ++qemu-img: Cannot enable copy offloading when -S is used ++ ++Testing: convert -C -S 8k -O qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.target ++qemu-img: Cannot enable copy offloading when -S is used ++ ++Testing: convert -C -c -O qcow2 TEST_DIR/t.qcow2 TEST_DIR/t.qcow2.target ++qemu-img: Cannot enable copy offloading when -c is used ++ + === amend: Options specified more than once === + + Testing: amend -f foo -f qcow2 -o lazy_refcounts=on TEST_DIR/t.qcow2 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Add-test-for-rebasing-with-relative-paths.patch b/SOURCES/kvm-iotests-Add-test-for-rebasing-with-relative-paths.patch new file mode 100644 index 0000000..08392d3 --- /dev/null +++ b/SOURCES/kvm-iotests-Add-test-for-rebasing-with-relative-paths.patch @@ -0,0 +1,168 @@ +From 165fabd12568ce69a14a541c762267f7a1d161d1 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 17:48:33 +0200 +Subject: [PATCH 072/268] iotests: Add test for rebasing with relative paths + +RH-Author: Max Reitz +Message-id: <20180618174833.19439-3-mreitz@redhat.com> +Patchwork-id: 80789 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/2] iotests: Add test for rebasing with relative paths +Bugzilla: 1569835 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509182002.8044-3-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 28036a7f7044fddb79819e3c8fcb4ae5605c60e0) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/024 | 82 ++++++++++++++++++++++++++++++++++++++++++++-- + tests/qemu-iotests/024.out | 30 +++++++++++++++++ + 2 files changed, 109 insertions(+), 3 deletions(-) + +diff --git a/tests/qemu-iotests/024 b/tests/qemu-iotests/024 +index e0d77ce..4071ed6 100755 +--- a/tests/qemu-iotests/024 ++++ b/tests/qemu-iotests/024 +@@ -29,9 +29,14 @@ status=1 # failure is the default! + + _cleanup() + { +- _cleanup_test_img +- rm -f "$TEST_DIR/t.$IMGFMT.base_old" +- rm -f "$TEST_DIR/t.$IMGFMT.base_new" ++ _cleanup_test_img ++ rm -f "$TEST_DIR/t.$IMGFMT.base_old" ++ rm -f "$TEST_DIR/t.$IMGFMT.base_new" ++ ++ rm -f "$TEST_DIR/subdir/t.$IMGFMT" ++ rm -f "$TEST_DIR/subdir/t.$IMGFMT.base_old" ++ rm -f "$TEST_DIR/subdir/t.$IMGFMT.base_new" ++ rmdir "$TEST_DIR/subdir" 2> /dev/null + } + trap "_cleanup; exit \$status" 0 1 2 3 15 + +@@ -123,6 +128,77 @@ io_pattern readv $((13 * CLUSTER_SIZE)) $CLUSTER_SIZE 0 1 0x00 + io_pattern readv $((14 * CLUSTER_SIZE)) $CLUSTER_SIZE 0 1 0x11 + io_pattern readv $((15 * CLUSTER_SIZE)) $CLUSTER_SIZE 0 1 0x00 + ++echo ++echo "=== Test rebase in a subdirectory of the working directory ===" ++echo ++ ++# Clean up the old images beforehand so they do not interfere with ++# this test ++_cleanup ++ ++mkdir "$TEST_DIR/subdir" ++ ++# Relative to the overlay ++BASE_OLD_OREL="t.$IMGFMT.base_old" ++BASE_NEW_OREL="t.$IMGFMT.base_new" ++ ++# Relative to $TEST_DIR (which is going to be our working directory) ++OVERLAY_WREL="subdir/t.$IMGFMT" ++ ++BASE_OLD="$TEST_DIR/subdir/$BASE_OLD_OREL" ++BASE_NEW="$TEST_DIR/subdir/$BASE_NEW_OREL" ++OVERLAY="$TEST_DIR/$OVERLAY_WREL" ++ ++# Test done here: ++# ++# Backing (old): 11 11 -- 11 ++# Backing (new): -- 22 22 11 ++# Overlay: -- -- -- -- ++# ++# Rebasing works, we have verified that above. Here, we just want to ++# see that rebasing is done for the correct target backing file. ++ ++TEST_IMG=$BASE_OLD _make_test_img 1M ++TEST_IMG=$BASE_NEW _make_test_img 1M ++TEST_IMG=$OVERLAY _make_test_img -b "$BASE_OLD_OREL" 1M ++ ++echo ++ ++$QEMU_IO "$BASE_OLD" \ ++ -c "write -P 0x11 $((0 * CLUSTER_SIZE)) $((2 * CLUSTER_SIZE))" \ ++ -c "write -P 0x11 $((3 * CLUSTER_SIZE)) $((1 * CLUSTER_SIZE))" \ ++ | _filter_qemu_io ++ ++$QEMU_IO "$BASE_NEW" \ ++ -c "write -P 0x22 $((1 * CLUSTER_SIZE)) $((2 * CLUSTER_SIZE))" \ ++ -c "write -P 0x11 $((3 * CLUSTER_SIZE)) $((1 * CLUSTER_SIZE))" \ ++ | _filter_qemu_io ++ ++echo ++ ++pushd "$TEST_DIR" >/dev/null ++$QEMU_IMG rebase -f "$IMGFMT" -b "$BASE_NEW_OREL" "$OVERLAY_WREL" ++popd >/dev/null ++ ++# Verify the backing path is correct ++TEST_IMG=$OVERLAY _img_info | grep '^backing file' ++ ++echo ++ ++# Verify the data is correct ++$QEMU_IO "$OVERLAY" \ ++ -c "read -P 0x11 $((0 * CLUSTER_SIZE)) $CLUSTER_SIZE" \ ++ -c "read -P 0x11 $((1 * CLUSTER_SIZE)) $CLUSTER_SIZE" \ ++ -c "read -P 0x00 $((2 * CLUSTER_SIZE)) $CLUSTER_SIZE" \ ++ -c "read -P 0x11 $((3 * CLUSTER_SIZE)) $CLUSTER_SIZE" \ ++ | _filter_qemu_io ++ ++echo ++ ++# Verify that cluster #3 is not allocated (because it is the same in ++# $BASE_OLD and $BASE_NEW) ++$QEMU_IMG map "$OVERLAY" | _filter_qemu_img_map ++ + + # success, all done + echo "*** done" +diff --git a/tests/qemu-iotests/024.out b/tests/qemu-iotests/024.out +index 33cfaf5..024dc78 100644 +--- a/tests/qemu-iotests/024.out ++++ b/tests/qemu-iotests/024.out +@@ -141,4 +141,34 @@ read 65536/65536 bytes at offset 917504 + === IO: pattern 0x00 + read 65536/65536 bytes at offset 983040 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++=== Test rebase in a subdirectory of the working directory === ++ ++Formatting 'TEST_DIR/subdir/t.IMGFMT.base_old', fmt=IMGFMT size=1048576 ++Formatting 'TEST_DIR/subdir/t.IMGFMT.base_new', fmt=IMGFMT size=1048576 ++Formatting 'TEST_DIR/subdir/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=t.IMGFMT.base_old ++ ++wrote 131072/131072 bytes at offset 0 ++128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 65536/65536 bytes at offset 196608 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 131072/131072 bytes at offset 65536 ++128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 65536/65536 bytes at offset 196608 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++backing file: t.IMGFMT.base_new (actual path: TEST_DIR/subdir/t.IMGFMT.base_new) ++ ++read 65536/65536 bytes at offset 0 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 65536/65536 bytes at offset 65536 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 65536/65536 bytes at offset 131072 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 65536/65536 bytes at offset 196608 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++Offset Length File ++0 0x30000 TEST_DIR/subdir/t.IMGFMT ++0x30000 0x10000 TEST_DIR/subdir/t.IMGFMT.base_new + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Clean-up-wrap-image-in-197.patch b/SOURCES/kvm-iotests-Clean-up-wrap-image-in-197.patch new file mode 100644 index 0000000..01c6391 --- /dev/null +++ b/SOURCES/kvm-iotests-Clean-up-wrap-image-in-197.patch @@ -0,0 +1,42 @@ +From 73cff4668c6be7f24c7ce69703d5d6600badd139 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:10 +0200 +Subject: [PATCH 043/268] iotests: Clean up wrap image in 197 + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-9-mreitz@redhat.com> +Patchwork-id: 80768 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 08/10] iotests: Clean up wrap image in 197 +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Signed-off-by: Max Reitz +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Alberto Garcia +Message-id: 20180421132929.21610-8-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit 5fdc0b73eb68d107944cfa65185fb155b511e496) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/197 | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 +index 5e869fe..3ae4975 100755 +--- a/tests/qemu-iotests/197 ++++ b/tests/qemu-iotests/197 +@@ -44,6 +44,7 @@ esac + _cleanup() + { + _cleanup_test_img ++ rm -f "$TEST_WRAP" + rm -f "$BLKDBG_CONF" + } + trap "_cleanup; exit \$status" 0 1 2 3 15 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Copy-197-for-COR-filter-driver.patch b/SOURCES/kvm-iotests-Copy-197-for-COR-filter-driver.patch new file mode 100644 index 0000000..05bc310 --- /dev/null +++ b/SOURCES/kvm-iotests-Copy-197-for-COR-filter-driver.patch @@ -0,0 +1,203 @@ +From 316dee28e5130d339ad7d141eab4fcf61ec07e4c Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:11 +0200 +Subject: [PATCH 044/268] iotests: Copy 197 for COR filter driver + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-10-mreitz@redhat.com> +Patchwork-id: 80769 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 09/10] iotests: Copy 197 for COR filter driver +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +iotest 197 tests copy-on-read using the (now old) copy-on-read flag. +Copy it to 215 and modify it to use the COR filter driver instead. + +Signed-off-by: Max Reitz +Message-id: 20180421132929.21610-9-mreitz@redhat.com +Reviewed-by: Kevin Wolf +Signed-off-by: Max Reitz +(cherry picked from commit a62cbac4ce2db79c14ff299e98ee556b57467c19) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/215 | 120 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/215.out | 26 ++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 147 insertions(+) + create mode 100755 tests/qemu-iotests/215 + create mode 100644 tests/qemu-iotests/215.out + +diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215 +new file mode 100755 +index 0000000..2e616ed +--- /dev/null ++++ b/tests/qemu-iotests/215 +@@ -0,0 +1,120 @@ ++#!/bin/bash ++# ++# Test case for copy-on-read into qcow2, using the COR filter driver ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++seq="$(basename $0)" ++echo "QA output created by $seq" ++ ++here="$PWD" ++status=1 # failure is the default! ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++ ++TEST_WRAP="$TEST_DIR/t.wrap.qcow2" ++BLKDBG_CONF="$TEST_DIR/blkdebug.conf" ++ ++# Sanity check: our use of blkdebug fails if $TEST_DIR contains spaces ++# or other problems ++case "$TEST_DIR" in ++ *[^-_a-zA-Z0-9/]*) ++ _notrun "Suspicious TEST_DIR='$TEST_DIR', cowardly refusing to run" ;; ++esac ++ ++_cleanup() ++{ ++ _cleanup_test_img ++ rm -f "$TEST_WRAP" ++ rm -f "$BLKDBG_CONF" ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# Test is supported for any backing file; but we force qcow2 for our wrapper. ++_supported_fmt generic ++_supported_proto generic ++_supported_os Linux ++# LUKS support may be possible, but it complicates things. ++_unsupported_fmt luks ++ ++echo ++echo '=== Copy-on-read ===' ++echo ++ ++# Prep the images ++# VPC rounds image sizes to a specific geometry, force a specific size. ++if [ "$IMGFMT" = "vpc" ]; then ++ IMGOPTS=$(_optstr_add "$IMGOPTS" "force_size") ++fi ++_make_test_img 4G ++$QEMU_IO -c "write -P 55 3G 1k" "$TEST_IMG" | _filter_qemu_io ++IMGPROTO=file IMGFMT=qcow2 IMGOPTS= TEST_IMG_FILE="$TEST_WRAP" \ ++ _make_test_img -F "$IMGFMT" -b "$TEST_IMG" | _filter_img_create ++$QEMU_IO -f qcow2 -c "write -z -u 1M 64k" "$TEST_WRAP" | _filter_qemu_io ++ ++# Ensure that a read of two clusters, but where one is already allocated, ++# does not re-write the allocated cluster ++cat > "$BLKDBG_CONF" <&1 | _filter_qemu_io) ++case $output in ++ *allocate*) ++ _notrun "Insufficent memory to run test" ;; ++ *) printf '%s\n' "$output" ;; ++esac ++$QEMU_IO \ ++ -c "open -o driver=copy-on-read,file.driver=qcow2 $TEST_WRAP" \ ++ -c "read -P 0 $((3*1024*1024*1024 + 1024)) 1k" \ ++ | _filter_qemu_io ++ ++# Copy-on-read is incompatible with read-only ++$QEMU_IO \ ++ -c "open -r -o driver=copy-on-read,file.driver=qcow2 $TEST_WRAP" \ ++ 2>&1 | _filter_testdir ++ ++# Break the backing chain, and show that images are identical, and that ++# we properly copied over explicit zeros. ++$QEMU_IMG rebase -u -b "" -f qcow2 "$TEST_WRAP" ++$QEMU_IO -f qcow2 -c map "$TEST_WRAP" ++_check_test_img ++$QEMU_IMG compare -f $IMGFMT -F qcow2 "$TEST_IMG" "$TEST_WRAP" ++ ++# success, all done ++echo '*** done' ++status=0 +diff --git a/tests/qemu-iotests/215.out b/tests/qemu-iotests/215.out +new file mode 100644 +index 0000000..70b0f5f +--- /dev/null ++++ b/tests/qemu-iotests/215.out +@@ -0,0 +1,26 @@ ++QA output created by 215 ++ ++=== Copy-on-read === ++ ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4294967296 ++wrote 1024/1024 bytes at offset 3221225472 ++1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=IMGFMT size=4294967296 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT ++wrote 65536/65536 bytes at offset 1048576 ++64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 131072/131072 bytes at offset 1048576 ++128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 0/0 bytes at offset 0 ++0 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 2147483136/2147483136 bytes at offset 1024 ++2 GiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 1024/1024 bytes at offset 3221226496 ++1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++can't open device TEST_DIR/t.wrap.qcow2: Block node is read-only ++2 GiB (0x80010000) bytes allocated at offset 0 bytes (0x0) ++1023.938 MiB (0x3fff0000) bytes not allocated at offset 2 GiB (0x80010000) ++64 KiB (0x10000) bytes allocated at offset 3 GiB (0xc0000000) ++1023.938 MiB (0x3fff0000) bytes not allocated at offset 3 GiB (0xc0010000) ++No errors were found on the image. ++Images are identical. ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index ba7a2d1..cd5d26c 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -213,4 +213,5 @@ + 212 rw auto quick + 213 rw auto quick + 214 rw auto ++215 rw auto quick + 218 rw auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Fix-219-s-timing.patch b/SOURCES/kvm-iotests-Fix-219-s-timing.patch new file mode 100644 index 0000000..0e2d999 --- /dev/null +++ b/SOURCES/kvm-iotests-Fix-219-s-timing.patch @@ -0,0 +1,149 @@ +From 6839f724393417ae25f3b5ab8bd85bf067925cc2 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:55 +0200 +Subject: [PATCH 147/268] iotests: Fix 219's timing + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-73-kwolf@redhat.com> +Patchwork-id: 81111 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 72/73] iotests: Fix 219's timing +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: Max Reitz + +219 has two issues that may lead to sporadic failure, both of which are +the result of issuing query-jobs too early after a job has been +modified. This can then lead to different results based on whether the +modification has taken effect already or not. + +First, query-jobs is issued right after the job has been created. +Besides its current progress possibly being in any random state (which +has already been taken care of), its total progress too is basically +arbitrary, because the job may not yet have been able to determine it. +This patch addresses this by just filtering the total progress, like +what has been done for the current progress already. However, for more +clarity, the filtering is changed to replace the values by a string +'FILTERED' instead of deleting them. + +Secondly, query-jobs is issued right after a job has been resumed. The +job may or may not yet have had the time to actually perform any I/O, +and thus its current progress may or may not have advanced. To make +sure it has indeed advanced (which is what the reference output already +assumes), keep querying it until it has. + +Signed-off-by: Max Reitz +Message-id: 20180606190628.8170-1-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 83f90b535a0d5e64056c087aa4022ea35c59bcd0) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/219 | 26 ++++++++++++++++++++------ + tests/qemu-iotests/219.out | 10 +++++----- + 2 files changed, 25 insertions(+), 11 deletions(-) + +diff --git a/tests/qemu-iotests/219 b/tests/qemu-iotests/219 +index 898a26e..c03bbdb 100755 +--- a/tests/qemu-iotests/219 ++++ b/tests/qemu-iotests/219 +@@ -42,11 +42,24 @@ def test_pause_resume(vm): + iotests.log(vm.qmp(pause_cmd, **{pause_arg: 'job0'})) + pause_wait(vm, 'job0') + iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) +- iotests.log(vm.qmp('query-jobs')) ++ result = vm.qmp('query-jobs') ++ iotests.log(result) ++ ++ old_progress = result['return'][0]['current-progress'] ++ total_progress = result['return'][0]['total-progress'] + + iotests.log(vm.qmp(resume_cmd, **{resume_arg: 'job0'})) + iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) +- iotests.log(vm.qmp('query-jobs')) ++ if old_progress < total_progress: ++ # Wait for the job to advance ++ while result['return'][0]['current-progress'] == old_progress: ++ result = vm.qmp('query-jobs') ++ iotests.log(result) ++ else: ++ # Already reached the end, so the job cannot advance ++ # any further; therefore, the query-jobs result can be ++ # logged immediately ++ iotests.log(vm.qmp('query-jobs')) + + def test_job_lifecycle(vm, job, job_args, has_ready=False): + iotests.log('') +@@ -58,12 +71,13 @@ def test_job_lifecycle(vm, job, job_args, has_ready=False): + iotests.log(vm.qmp(job, job_id='job0', **job_args)) + + # Depending on the storage, the first request may or may not have completed +- # yet, so filter out the progress. Later query-job calls don't need the +- # filtering because the progress is made deterministic by the block job +- # speed ++ # yet (and the total progress may not have been fully determined yet), so ++ # filter out the progress. Later query-job calls don't need the filtering ++ # because the progress is made deterministic by the block job speed + result = vm.qmp('query-jobs') + for j in result['return']: +- del j['current-progress'] ++ j['current-progress'] = 'FILTERED' ++ j['total-progress'] = 'FILTERED' + iotests.log(result) + + # undefined -> created -> running +diff --git a/tests/qemu-iotests/219.out b/tests/qemu-iotests/219.out +index 346801b..6dc07bc 100644 +--- a/tests/qemu-iotests/219.out ++++ b/tests/qemu-iotests/219.out +@@ -3,7 +3,7 @@ Launching VM... + + Starting block job: drive-mirror (auto-finalize: True; auto-dismiss: True) + {u'return': {}} +-{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'mirror'}]} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + +@@ -93,7 +93,7 @@ Waiting for PENDING state... + + Starting block job: drive-backup (auto-finalize: True; auto-dismiss: True) + {u'return': {}} +-{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + +@@ -144,7 +144,7 @@ Waiting for PENDING state... + + Starting block job: drive-backup (auto-finalize: True; auto-dismiss: False) + {u'return': {}} +-{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + +@@ -203,7 +203,7 @@ Waiting for PENDING state... + + Starting block job: drive-backup (auto-finalize: False; auto-dismiss: True) + {u'return': {}} +-{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + +@@ -262,7 +262,7 @@ Waiting for PENDING state... + + Starting block job: drive-backup (auto-finalize: False; auto-dismiss: False) + {u'return': {}} +-{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': [{u'status': u'running', u'current-progress': 'FILTERED', u'total-progress': 'FILTERED', u'id': u'job0', u'type': u'backup'}]} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + {u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Let-216-make-use-of-qemu-io-s-exit-code.patch b/SOURCES/kvm-iotests-Let-216-make-use-of-qemu-io-s-exit-code.patch new file mode 100644 index 0000000..591382a --- /dev/null +++ b/SOURCES/kvm-iotests-Let-216-make-use-of-qemu-io-s-exit-code.patch @@ -0,0 +1,115 @@ +From 3fd5087a6d6d8c366ab68ff804f975ef0c77b796 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:43:12 +0200 +Subject: [PATCH 053/268] iotests: Let 216 make use of qemu-io's exit code + +RH-Author: Max Reitz +Message-id: <20180618164312.24423-6-mreitz@redhat.com> +Patchwork-id: 80776 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 5/5] iotests: Let 216 make use of qemu-io's exit code +Bugzilla: 1519617 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +As a showcase of how you can use qemu-io's exit code to determine +success or failure (same for qemu-img), this test is changed to use +qemu_io_silent() instead of qemu_io(), and to assert the exit code +instead of logging the filtered result. + +One real advantage of this is that in case of an error, you get a +backtrace that helps you locate the issue in the test file quickly. + +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509194302.21585-6-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit e4ca4e981a0a389d6af5dc5d8b5fbdd1a05276a0) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/216 | 23 ++++++++++++----------- + tests/qemu-iotests/216.out | 17 ++--------------- + 2 files changed, 14 insertions(+), 26 deletions(-) + +diff --git a/tests/qemu-iotests/216 b/tests/qemu-iotests/216 +index ca9b47a..3c0ae54 100755 +--- a/tests/qemu-iotests/216 ++++ b/tests/qemu-iotests/216 +@@ -20,7 +20,7 @@ + # Creator/Owner: Max Reitz + + import iotests +-from iotests import log, qemu_img_pipe, qemu_io, filter_qemu_io ++from iotests import log, qemu_img, qemu_io_silent + + # Need backing file support + iotests.verify_image_format(supported_fmts=['qcow2', 'qcow', 'qed', 'vmdk']) +@@ -50,14 +50,13 @@ with iotests.FilePath('base.img') as base_img_path, \ + log('--- Setting up images ---') + log('') + +- qemu_img_pipe('create', '-f', iotests.imgfmt, base_img_path, '64M') ++ assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 ++ assert qemu_io_silent(base_img_path, '-c', 'write -P 1 0M 1M') == 0 ++ assert qemu_img('create', '-f', iotests.imgfmt, '-b', base_img_path, ++ top_img_path) == 0 ++ assert qemu_io_silent(top_img_path, '-c', 'write -P 2 1M 1M') == 0 + +- log(filter_qemu_io(qemu_io(base_img_path, '-c', 'write -P 1 0M 1M'))) +- +- qemu_img_pipe('create', '-f', iotests.imgfmt, '-b', base_img_path, +- top_img_path) +- +- log(filter_qemu_io(qemu_io(top_img_path, '-c', 'write -P 2 1M 1M'))) ++ log('Done') + + log('') + log('--- Doing COR ---') +@@ -110,6 +109,8 @@ with iotests.FilePath('base.img') as base_img_path, \ + log('--- Checking COR result ---') + log('') + +- log(filter_qemu_io(qemu_io(base_img_path, '-c', 'discard 0 64M'))) +- log(filter_qemu_io(qemu_io(top_img_path, '-c', 'read -P 1 0M 1M'))) +- log(filter_qemu_io(qemu_io(top_img_path, '-c', 'read -P 2 1M 1M'))) ++ assert qemu_io_silent(base_img_path, '-c', 'discard 0 64M') == 0 ++ assert qemu_io_silent(top_img_path, '-c', 'read -P 1 0M 1M') == 0 ++ assert qemu_io_silent(top_img_path, '-c', 'read -P 2 1M 1M') == 0 ++ ++ log('Done') +diff --git a/tests/qemu-iotests/216.out b/tests/qemu-iotests/216.out +index d3fc590..45ea857 100644 +--- a/tests/qemu-iotests/216.out ++++ b/tests/qemu-iotests/216.out +@@ -3,12 +3,7 @@ + + --- Setting up images --- + +-wrote 1048576/1048576 bytes at offset 0 +-1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +- +-wrote 1048576/1048576 bytes at offset 1048576 +-1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +- ++Done + + --- Doing COR --- + +@@ -17,12 +12,4 @@ wrote 1048576/1048576 bytes at offset 1048576 + + --- Checking COR result --- + +-discard 67108864/67108864 bytes at offset 0 +-64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +- +-read 1048576/1048576 bytes at offset 0 +-1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +- +-read 1048576/1048576 bytes at offset 1048576 +-1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +- ++Done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Move-qmp_to_opts-to-VM.patch b/SOURCES/kvm-iotests-Move-qmp_to_opts-to-VM.patch new file mode 100644 index 0000000..400352f --- /dev/null +++ b/SOURCES/kvm-iotests-Move-qmp_to_opts-to-VM.patch @@ -0,0 +1,132 @@ +From 1f4ca318d904030aeaee0905b724335b3346acc3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:35 +0200 +Subject: [PATCH 127/268] iotests: Move qmp_to_opts() to VM + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-53-kwolf@redhat.com> +Patchwork-id: 81085 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 52/73] iotests: Move qmp_to_opts() to VM +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +qmp_to_opts() used to be a method of QMPTestCase, but recently we +started to add more Python test cases that don't make use of +QMPTestCase. In order to make the method usable there, move it to VM. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 62a9428812c0f4aacbf2f7fdf449fa4f4ab3775c) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/041 | 6 +++--- + tests/qemu-iotests/155 | 2 +- + tests/qemu-iotests/iotests.py | 45 ++++++++++++++++++++++--------------------- + 3 files changed, 27 insertions(+), 26 deletions(-) + +diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 +index e945879..c20ac7d 100755 +--- a/tests/qemu-iotests/041 ++++ b/tests/qemu-iotests/041 +@@ -1030,9 +1030,9 @@ class TestOrphanedSource(iotests.QMPTestCase): + 'read-only': 'on' } + + self.vm = iotests.VM() +- self.vm.add_blockdev(self.qmp_to_opts(blk0)) +- self.vm.add_blockdev(self.qmp_to_opts(blk1)) +- self.vm.add_blockdev(self.qmp_to_opts(blk2)) ++ self.vm.add_blockdev(self.vm.qmp_to_opts(blk0)) ++ self.vm.add_blockdev(self.vm.qmp_to_opts(blk1)) ++ self.vm.add_blockdev(self.vm.qmp_to_opts(blk2)) + self.vm.launch() + + def tearDown(self): +diff --git a/tests/qemu-iotests/155 b/tests/qemu-iotests/155 +index 42dae04..63a5b5e 100755 +--- a/tests/qemu-iotests/155 ++++ b/tests/qemu-iotests/155 +@@ -63,7 +63,7 @@ class BaseClass(iotests.QMPTestCase): + 'driver': iotests.imgfmt, + 'file': {'driver': 'file', + 'filename': source_img}} +- self.vm.add_blockdev(self.qmp_to_opts(blockdev)) ++ self.vm.add_blockdev(self.vm.qmp_to_opts(blockdev)) + self.vm.add_device('virtio-blk,id=qdev0,drive=source') + self.vm.launch() + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 824f87d..d190ce7 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -373,6 +373,27 @@ class VM(qtest.QEMUQtestMachine): + return self.qmp('human-monitor-command', + command_line='qemu-io %s "%s"' % (drive, cmd)) + ++ def flatten_qmp_object(self, obj, output=None, basestr=''): ++ if output is None: ++ output = dict() ++ if isinstance(obj, list): ++ for i in range(len(obj)): ++ self.flatten_qmp_object(obj[i], output, basestr + str(i) + '.') ++ elif isinstance(obj, dict): ++ for key in obj: ++ self.flatten_qmp_object(obj[key], output, basestr + key + '.') ++ else: ++ output[basestr[:-1]] = obj # Strip trailing '.' ++ return output ++ ++ def qmp_to_opts(self, obj): ++ obj = self.flatten_qmp_object(obj) ++ output_list = list() ++ for key in obj: ++ output_list += [key + '=' + obj[key]] ++ return ','.join(output_list) ++ ++ + + index_re = re.compile(r'([^\[]+)\[([^\]]+)\]') + +@@ -400,26 +421,6 @@ class QMPTestCase(unittest.TestCase): + self.fail('invalid index "%s" in path "%s" in "%s"' % (idx, path, str(d))) + return d + +- def flatten_qmp_object(self, obj, output=None, basestr=''): +- if output is None: +- output = dict() +- if isinstance(obj, list): +- for i in range(len(obj)): +- self.flatten_qmp_object(obj[i], output, basestr + str(i) + '.') +- elif isinstance(obj, dict): +- for key in obj: +- self.flatten_qmp_object(obj[key], output, basestr + key + '.') +- else: +- output[basestr[:-1]] = obj # Strip trailing '.' +- return output +- +- def qmp_to_opts(self, obj): +- obj = self.flatten_qmp_object(obj) +- output_list = list() +- for key in obj: +- output_list += [key + '=' + obj[key]] +- return ','.join(output_list) +- + def assert_qmp_absent(self, d, path): + try: + result = self.dictpath(d, path) +@@ -454,8 +455,8 @@ class QMPTestCase(unittest.TestCase): + '''Asserts that the given filename is a json: filename and that its + content is equal to the given reference object''' + self.assertEqual(json_filename[:5], 'json:') +- self.assertEqual(self.flatten_qmp_object(json.loads(json_filename[5:])), +- self.flatten_qmp_object(reference)) ++ self.assertEqual(self.vm.flatten_qmp_object(json.loads(json_filename[5:])), ++ self.vm.flatten_qmp_object(reference)) + + def cancel_and_wait(self, drive='drive0', force=False, resume=False): + '''Cancel a block job and wait for it to finish, returning the event''' +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-New-test-223-for-exporting-dirty-bitmap-over.patch b/SOURCES/kvm-iotests-New-test-223-for-exporting-dirty-bitmap-over.patch new file mode 100644 index 0000000..fca9f0c --- /dev/null +++ b/SOURCES/kvm-iotests-New-test-223-for-exporting-dirty-bitmap-over.patch @@ -0,0 +1,259 @@ +From da30d6d37ac6ff8fdf873dcd5a519fca72dd59e9 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 24 Jul 2018 13:06:13 +0200 +Subject: [PATCH 253/268] iotests: New test 223 for exporting dirty bitmap over + NBD + +RH-Author: John Snow +Message-id: <20180718225511.14878-36-jsnow@redhat.com> +Patchwork-id: 81424 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 35/35] iotests: New test 223 for exporting dirty bitmap over NBD +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Eric Blake + +Although this test is NOT a full test of image fleecing (as it +intentionally uses just a single block device directly exported +over NBD, rather than trying to set up a blockdev-backup job with +multiple BDS involved), it DOES prove that qemu as a server is +able to properly expose a dirty bitmap over NBD. + +When coupled with image fleecing, it is then possible for a +third-party client to do an incremental backup by using +qemu-img map with the x-dirty-bitmap option to learn which parts +of the file are dirty (perhaps confusingly, they are the portions +mapped as "data":false - which is part of the reason this is +still in the x- experimental namespace), along with another +normal client (perhaps 'qemu-nbd -c' to expose the server over +/dev/nbd0 and then just use normal I/O on that block device) to +read the dirty sections. + +Signed-off-by: Eric Blake +Message-Id: <20180702191458.28741-3-eblake@redhat.com> +Tested-by: John Snow +Reviewed-by: John Snow +(cherry picked from commit a1532a225a183c9fa60b9c1e8ac8a00c7771f64d) +Signed-off-by: John Snow +--- + tests/qemu-iotests/223 | 138 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/223.out | 49 ++++++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 188 insertions(+) + create mode 100644 tests/qemu-iotests/223 + create mode 100644 tests/qemu-iotests/223.out + +diff --git a/tests/qemu-iotests/223 b/tests/qemu-iotests/223 +new file mode 100644 +index 0000000..b63b7a4 +--- /dev/null ++++ b/tests/qemu-iotests/223 +@@ -0,0 +1,138 @@ ++#!/bin/bash ++# ++# Test reading dirty bitmap over NBD ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++seq="$(basename $0)" ++echo "QA output created by $seq" ++ ++here="$PWD" ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ _cleanup_test_img ++ _cleanup_qemu ++ rm -f "$TEST_DIR/nbd" ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++. ./common.qemu ++ ++_supported_fmt qcow2 ++_supported_proto file # uses NBD as well ++_supported_os Linux ++ ++function do_run_qemu() ++{ ++ echo Testing: "$@" ++ $QEMU -nographic -qmp stdio -serial none "$@" ++ echo ++} ++ ++function run_qemu() ++{ ++ do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ ++ | _filter_qemu | _filter_imgfmt \ ++ | _filter_actual_image_size ++} ++ ++echo ++echo "=== Create partially sparse image, then add dirty bitmap ===" ++echo ++ ++_make_test_img 4M ++$QEMU_IO -c 'w -P 0x11 1M 2M' "$TEST_IMG" | _filter_qemu_io ++run_qemu < >(_filter_nbd) ++ ++silent= ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"qmp_capabilities"}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"blockdev-add", ++ "arguments":{"driver":"qcow2", "node-name":"n", ++ "file":{"driver":"file", "filename":"'"$TEST_IMG"'"}}}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-block-dirty-bitmap-disable", ++ "arguments":{"node":"n", "name":"b"}}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-start", ++ "arguments":{"addr":{"type":"unix", ++ "data":{"path":"'"$TEST_DIR/nbd"'"}}}}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-add", ++ "arguments":{"device":"n"}}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"x-nbd-server-add-bitmap", ++ "arguments":{"name":"n", "bitmap":"b"}}' "return" ++ ++echo ++echo "=== Contrast normal status with dirty-bitmap status ===" ++echo ++ ++QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT ++IMG="driver=nbd,export=n,server.type=unix,server.path=$TEST_DIR/nbd" ++$QEMU_IO -r -c 'r -P 0 0 1m' -c 'r -P 0x11 1m 1m' \ ++ -c 'r -P 0x22 2m 2m' --image-opts "$IMG" | _filter_qemu_io ++$QEMU_IMG map --output=json --image-opts \ ++ "$IMG" | _filter_qemu_img_map ++$QEMU_IMG map --output=json --image-opts \ ++ "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b" | _filter_qemu_img_map ++ ++echo ++echo "=== End NBD server ===" ++echo ++ ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-remove", ++ "arguments":{"name":"n"}}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"nbd-server-stop"}' "return" ++_send_qemu_cmd $QEMU_HANDLE '{"execute":"quit"}' "return" ++ ++# success, all done ++echo '*** done' ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/223.out b/tests/qemu-iotests/223.out +new file mode 100644 +index 0000000..33021c8 +--- /dev/null ++++ b/tests/qemu-iotests/223.out +@@ -0,0 +1,49 @@ ++QA output created by 223 ++ ++=== Create partially sparse image, then add dirty bitmap === ++ ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 ++wrote 2097152/2097152 bytes at offset 1048576 ++2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++Testing: ++QMP_VERSION ++{"return": {}} ++{"return": {}} ++{"return": {}} ++{"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++ ++ ++=== Write part of the file under active bitmap === ++ ++wrote 2097152/2097152 bytes at offset 2097152 ++2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ ++=== End dirty bitmap, and start serving image over NBD === ++ ++{"return": {}} ++{"return": {}} ++{"return": {}} ++{"return": {}} ++{"return": {}} ++{"return": {}} ++ ++=== Contrast normal status with dirty-bitmap status === ++ ++read 1048576/1048576 bytes at offset 0 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 1048576/1048576 bytes at offset 1048576 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 2097152/2097152 bytes at offset 2097152 ++2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": false}, ++{ "start": 1048576, "length": 3145728, "depth": 0, "zero": false, "data": true}] ++[{ "start": 0, "length": 2097152, "depth": 0, "zero": false, "data": true}, ++{ "start": 2097152, "length": 2097152, "depth": 0, "zero": false, "data": false}] ++ ++=== End NBD server === ++ ++{"return": {}} ++{"return": {}} ++{"return": {}} ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index be55905..401258f 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -219,4 +219,5 @@ + 218 rw auto quick + 219 rw auto + 222 rw auto quick ++223 rw auto quick + 226 auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Repairing-error-during-snapshot-deletion.patch b/SOURCES/kvm-iotests-Repairing-error-during-snapshot-deletion.patch new file mode 100644 index 0000000..917e485 --- /dev/null +++ b/SOURCES/kvm-iotests-Repairing-error-during-snapshot-deletion.patch @@ -0,0 +1,192 @@ +From d18ad7f9dc56a87b3dc6ea111e753ebe3c2b0173 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 17:16:55 +0200 +Subject: [PATCH 055/268] iotests: Repairing error during snapshot deletion + +RH-Author: Max Reitz +Message-id: <20180618171655.25987-3-mreitz@redhat.com> +Patchwork-id: 80784 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/2] iotests: Repairing error during snapshot deletion +Bugzilla: 1527085 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +This adds a test for an I/O error during snapshot deletion, and maybe +more importantly, for how to repair the resulting image. If the +snapshot has been deleted before the error occurs, the only negative +result will be leaked clusters -- and those should be repairable with +qemu-img check -r leaks. + +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509200059.31125-3-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit b41ad73a3bb972eb43cf52d28669f67ea3fe1762) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/217 | 90 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/217.out | 42 ++++++++++++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 133 insertions(+) + create mode 100755 tests/qemu-iotests/217 + create mode 100644 tests/qemu-iotests/217.out + +diff --git a/tests/qemu-iotests/217 b/tests/qemu-iotests/217 +new file mode 100755 +index 0000000..d3ab5d7 +--- /dev/null ++++ b/tests/qemu-iotests/217 +@@ -0,0 +1,90 @@ ++#!/bin/bash ++# ++# I/O errors when working with internal qcow2 snapshots, and repairing ++# the result ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++ ++seq=$(basename $0) ++echo "QA output created by $seq" ++ ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ _cleanup_test_img ++ rm -f "$TEST_DIR/blkdebug.conf" ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++ ++# This test is specific to qcow2 ++_supported_fmt qcow2 ++_supported_proto file ++_supported_os Linux ++ ++# This test needs clusters with at least a refcount of 2 so that ++# OFLAG_COPIED is not set. refcount_bits=1 is therefore unsupported. ++_unsupported_imgopts 'refcount_bits=1[^0-9]' ++ ++echo ++echo '=== Simulating an I/O error during snapshot deletion ===' ++echo ++ ++_make_test_img 64M ++$QEMU_IO -c 'write 0 64k' "$TEST_IMG" | _filter_qemu_io ++ ++# Create the snapshot ++$QEMU_IMG snapshot -c foo "$TEST_IMG" ++ ++# Verify the snapshot is there ++echo ++_img_info | grep 'Snapshot list' ++echo '(Snapshot filtered)' ++echo ++ ++# Try to delete the snapshot (with an error happening when freeing the ++# then leaked clusters) ++cat > "$TEST_DIR/blkdebug.conf" < +Date: Mon, 18 Jun 2018 14:59:43 +0200 +Subject: [PATCH 070/268] iotests: Rework 113 + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-8-mreitz@redhat.com> +Patchwork-id: 80757 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 7/7] iotests: Rework 113 +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +This test case has been broken since 398e6ad014df261d (roughly half a +year). qemu-img amend requires its output image to be R/W, so it opens +it as such; the node is then turned into an read-only node automatically +which is now accompanied by a warning, however. This warning has not +been part of the reference output. + +For one thing, this warning shows that we cannot keep the test case as +it is. We would need a format that has no create_opts but that does +have write support -- we do not have such a format, though. + +Another thing is that qemu now actually checks whether an image format +supports amendment instead of whether it has create_opts (since the +former always implies the latter). So we can now use any format that +does not support amendment (even if it supports creation) and thus test +the same code path. + +The reason nobody has noticed the breakage until now of course is the +fact that nobody runs the iotests for nbd+bochs. There actually was +never any reason to set the protocol to "nbd" but because that was +technically correct; functionally it made no difference. So that is the +first thing we are going to change: Make the protocol "file" instead so +that people might actually notice breakage here. + +Secondly, now that bochs no longer works for the amend test case, we +have to change the format there anyway. Set let us just bend the truth +a bit, declare this test a raw test. In fact, that does not even +concern the bochs test cases, other than the output now reading 'bochs' +instead of 'IMGFMT'. + +So with this test now being a raw test, we can rework the amend test +case to use raw instead. + +Signed-off-by: Max Reitz +Reviewed-by: John Snow +Message-id: 20180509210023.20283-8-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit dee6ddd8a6b7978d0bc8ef8e1f006282ce30e4fa) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/113 | 19 +++++++++---------- + tests/qemu-iotests/113.out | 7 ++++--- + 2 files changed, 13 insertions(+), 13 deletions(-) + +diff --git a/tests/qemu-iotests/113 b/tests/qemu-iotests/113 +index 19b68b2..4e09810 100755 +--- a/tests/qemu-iotests/113 ++++ b/tests/qemu-iotests/113 +@@ -38,16 +38,17 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 + . ./common.rc + . ./common.filter + +-# We can only test one format here because we need its sample file +-_supported_fmt bochs +-_supported_proto nbd ++# Some of these test cases use bochs, but others do use raw, so this ++# is only half a lie. ++_supported_fmt raw ++_supported_proto file + _supported_os Linux + + echo + echo '=== Unsupported image creation in qemu-img create ===' + echo + +-$QEMU_IMG create -f $IMGFMT nbd://example.com 2>&1 64M | _filter_imgfmt ++$QEMU_IMG create -f bochs nbd://example.com 2>&1 64M + + echo + echo '=== Unsupported image creation in qemu-img convert ===' +@@ -56,17 +57,15 @@ echo + # We could use any input image format here, but this is a bochs test, so just + # use the bochs image + _use_sample_img empty.bochs.bz2 +-$QEMU_IMG convert -f $IMGFMT -O $IMGFMT "$TEST_IMG" nbd://example.com 2>&1 \ +- | _filter_imgfmt ++$QEMU_IMG convert -f bochs -O bochs "$TEST_IMG" nbd://example.com + + echo + echo '=== Unsupported format in qemu-img amend ===' + echo + +-# The protocol does not matter here +-_use_sample_img empty.bochs.bz2 +-$QEMU_IMG amend -f $IMGFMT -o foo=bar "$TEST_IMG" 2>&1 | _filter_imgfmt +- ++TEST_IMG="$TEST_DIR/t.$IMGFMT" ++_make_test_img 1M ++$QEMU_IMG amend -f $IMGFMT -o size=2M "$TEST_IMG" 2>&1 | _filter_imgfmt + + # success, all done + echo +diff --git a/tests/qemu-iotests/113.out b/tests/qemu-iotests/113.out +index 00bdfd6..3557e2b 100644 +--- a/tests/qemu-iotests/113.out ++++ b/tests/qemu-iotests/113.out +@@ -2,14 +2,15 @@ QA output created by 113 + + === Unsupported image creation in qemu-img create === + +-qemu-img: nbd://example.com: Format driver 'IMGFMT' does not support image creation ++qemu-img: nbd://example.com: Format driver 'bochs' does not support image creation + + === Unsupported image creation in qemu-img convert === + +-qemu-img: Format driver 'IMGFMT' does not support image creation ++qemu-img: Format driver 'bochs' does not support image creation + + === Unsupported format in qemu-img amend === + +-qemu-img: Format driver 'IMGFMT' does not support any options to amend ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 ++qemu-img: Format driver 'IMGFMT' does not support option amendment + + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Split-214-off-of-122.patch b/SOURCES/kvm-iotests-Split-214-off-of-122.patch new file mode 100644 index 0000000..59b1c37 --- /dev/null +++ b/SOURCES/kvm-iotests-Split-214-off-of-122.patch @@ -0,0 +1,301 @@ +From 8cfcce3f112c40f24bb40b5cf30ced810916de52 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:12:03 +0200 +Subject: [PATCH 036/268] iotests: Split 214 off of 122 + +RH-Author: Max Reitz +Message-id: <20180618161212.14444-2-mreitz@redhat.com> +Patchwork-id: 80761 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 01/10] iotests: Split 214 off of 122 +Bugzilla: 1518738 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Commit abd3622cc03cf41ed542126a540385f30a4c0175 added a case to 122 +regarding how the qcow2 driver handles an incorrect compressed data +length value. This does not really fit into 122, as that file is +supposed to contain qemu-img convert test cases, which this case is not. +So this patch splits it off into its own file; maybe we will even get +more qcow2-only compression tests in the future. + +Also, that test case does not work with refcount_bits=1, so mark that +option as unsupported. + +Signed-off-by: Max Reitz +Message-id: 20180406164108.26118-1-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Alberto Garcia +Signed-off-by: Max Reitz +(cherry picked from commit 6cba5377f54d7ea859a29c1877785e7101794683) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/122 | 47 ---------------------- + tests/qemu-iotests/122.out | 33 ---------------- + tests/qemu-iotests/214 | 97 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/214.out | 35 +++++++++++++++++ + tests/qemu-iotests/group | 1 + + 5 files changed, 133 insertions(+), 80 deletions(-) + create mode 100755 tests/qemu-iotests/214 + create mode 100644 tests/qemu-iotests/214.out + +diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122 +index 6cf4fcb..45b359c 100755 +--- a/tests/qemu-iotests/122 ++++ b/tests/qemu-iotests/122 +@@ -130,53 +130,6 @@ $QEMU_IO -c "read -P 0 1024k 1022k" "$TEST_IMG" 2>&1 | _filter_qemu_io | _fil + + + echo +-echo "=== Corrupted size field in compressed cluster descriptor ===" +-echo +-# Create an empty image and fill half of it with compressed data. +-# The L2 entries of the two compressed clusters are located at +-# 0x800000 and 0x800008, their original values are 0x4008000000a00000 +-# and 0x4008000000a00802 (5 sectors for compressed data each). +-_make_test_img 8M -o cluster_size=2M +-$QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG" \ +- 2>&1 | _filter_qemu_io | _filter_testdir +- +-# Reduce size of compressed data to 4 sectors: this corrupts the image. +-poke_file "$TEST_IMG" $((0x800000)) "\x40\x06" +-$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir +- +-# 'qemu-img check' however doesn't see anything wrong because it +-# doesn't try to decompress the data and the refcounts are consistent. +-# TODO: update qemu-img so this can be detected. +-_check_test_img +- +-# Increase size of compressed data to the maximum (8192 sectors). +-# This makes QEMU read more data (8192 sectors instead of 5, host +-# addresses [0xa00000, 0xdfffff]), but the decompression algorithm +-# stops once we have enough to restore the uncompressed cluster, so +-# the rest of the data is ignored. +-poke_file "$TEST_IMG" $((0x800000)) "\x7f\xfe" +-# Do it also for the second compressed cluster (L2 entry at 0x800008). +-# In this case the compressed data would span 3 host clusters +-# (host addresses: [0xa00802, 0xe00801]) +-poke_file "$TEST_IMG" $((0x800008)) "\x7f\xfe" +- +-# Here the image is too small so we're asking QEMU to read beyond the +-# end of the image. +-$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir +-# But if we grow the image we won't be reading beyond its end anymore. +-$QEMU_IO -c "write -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir +-$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir +- +-# The refcount data is however wrong because due to the increased size +-# of the compressed data it now reaches the following host clusters. +-# This can be repaired by qemu-img check by increasing the refcount of +-# those clusters. +-# TODO: update qemu-img to correct the compressed cluster size instead. +-_check_test_img -r all +-$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir +-$QEMU_IO -c "read -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir +- +-echo + echo "=== Full allocation with -S 0 ===" + echo + +diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out +index a6b7fe0..47d8656 100644 +--- a/tests/qemu-iotests/122.out ++++ b/tests/qemu-iotests/122.out +@@ -99,39 +99,6 @@ read 1024/1024 bytes at offset 1047552 + read 1046528/1046528 bytes at offset 1048576 + 1022 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +-=== Corrupted size field in compressed cluster descriptor === +- +-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=8388608 +-wrote 2097152/2097152 bytes at offset 0 +-2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +-wrote 2097152/2097152 bytes at offset 2097152 +-2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +-read failed: Input/output error +-No errors were found on the image. +-read 4194304/4194304 bytes at offset 0 +-4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +-wrote 4194304/4194304 bytes at offset 4194304 +-4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +-read 4194304/4194304 bytes at offset 0 +-4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +-ERROR cluster 6 refcount=1 reference=3 +-ERROR cluster 7 refcount=1 reference=2 +-Repairing cluster 6 refcount=1 reference=3 +-Repairing cluster 7 refcount=1 reference=2 +-Repairing OFLAG_COPIED data cluster: l2_entry=8000000000c00000 refcount=3 +-Repairing OFLAG_COPIED data cluster: l2_entry=8000000000e00000 refcount=2 +-The following inconsistencies were found and repaired: +- +- 0 leaked clusters +- 4 corruptions +- +-Double checking the fixed image now... +-No errors were found on the image. +-read 4194304/4194304 bytes at offset 0 +-4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +-read 4194304/4194304 bytes at offset 4194304 +-4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +- + === Full allocation with -S 0 === + + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 +diff --git a/tests/qemu-iotests/214 b/tests/qemu-iotests/214 +new file mode 100755 +index 0000000..c46ca2a +--- /dev/null ++++ b/tests/qemu-iotests/214 +@@ -0,0 +1,97 @@ ++#!/bin/bash ++# ++# Test qcow2 image compression ++# ++# Copyright (C) 2018 Igalia, S.L. ++# Author: Alberto Garcia ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++seq=$(basename "$0") ++echo "QA output created by $seq" ++ ++here=$PWD ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ _cleanup_test_img ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++ ++_supported_fmt qcow2 ++_supported_proto file ++_supported_os Linux ++ ++# Repairing the corrupted image requires qemu-img check to store a ++# refcount up to 3, which requires at least two refcount bits. ++_unsupported_imgopts 'refcount_bits=1[^0-9]' ++ ++ ++echo ++echo "=== Corrupted size field in compressed cluster descriptor ===" ++echo ++# Create an empty image and fill half of it with compressed data. ++# The L2 entries of the two compressed clusters are located at ++# 0x800000 and 0x800008, their original values are 0x4008000000a00000 ++# and 0x4008000000a00802 (5 sectors for compressed data each). ++_make_test_img 8M -o cluster_size=2M ++$QEMU_IO -c "write -c -P 0x11 0 2M" -c "write -c -P 0x11 2M 2M" "$TEST_IMG" \ ++ 2>&1 | _filter_qemu_io | _filter_testdir ++ ++# Reduce size of compressed data to 4 sectors: this corrupts the image. ++poke_file "$TEST_IMG" $((0x800000)) "\x40\x06" ++$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir ++ ++# 'qemu-img check' however doesn't see anything wrong because it ++# doesn't try to decompress the data and the refcounts are consistent. ++# TODO: update qemu-img so this can be detected. ++_check_test_img ++ ++# Increase size of compressed data to the maximum (8192 sectors). ++# This makes QEMU read more data (8192 sectors instead of 5, host ++# addresses [0xa00000, 0xdfffff]), but the decompression algorithm ++# stops once we have enough to restore the uncompressed cluster, so ++# the rest of the data is ignored. ++poke_file "$TEST_IMG" $((0x800000)) "\x7f\xfe" ++# Do it also for the second compressed cluster (L2 entry at 0x800008). ++# In this case the compressed data would span 3 host clusters ++# (host addresses: [0xa00802, 0xe00801]) ++poke_file "$TEST_IMG" $((0x800008)) "\x7f\xfe" ++ ++# Here the image is too small so we're asking QEMU to read beyond the ++# end of the image. ++$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir ++# But if we grow the image we won't be reading beyond its end anymore. ++$QEMU_IO -c "write -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir ++$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir ++ ++# The refcount data is however wrong because due to the increased size ++# of the compressed data it now reaches the following host clusters. ++# This can be repaired by qemu-img check by increasing the refcount of ++# those clusters. ++# TODO: update qemu-img to correct the compressed cluster size instead. ++_check_test_img -r all ++$QEMU_IO -c "read -P 0x11 0 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir ++$QEMU_IO -c "read -P 0x22 4M 4M" "$TEST_IMG" 2>&1 | _filter_qemu_io | _filter_testdir ++ ++# success, all done ++echo '*** done' ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/214.out b/tests/qemu-iotests/214.out +new file mode 100644 +index 0000000..0fcd8dc +--- /dev/null ++++ b/tests/qemu-iotests/214.out +@@ -0,0 +1,35 @@ ++QA output created by 214 ++ ++=== Corrupted size field in compressed cluster descriptor === ++ ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=8388608 ++wrote 2097152/2097152 bytes at offset 0 ++2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 2097152/2097152 bytes at offset 2097152 ++2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read failed: Input/output error ++No errors were found on the image. ++read 4194304/4194304 bytes at offset 0 ++4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 4194304/4194304 bytes at offset 4194304 ++4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 4194304/4194304 bytes at offset 0 ++4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ERROR cluster 6 refcount=1 reference=3 ++ERROR cluster 7 refcount=1 reference=2 ++Repairing cluster 6 refcount=1 reference=3 ++Repairing cluster 7 refcount=1 reference=2 ++Repairing OFLAG_COPIED data cluster: l2_entry=8000000000c00000 refcount=3 ++Repairing OFLAG_COPIED data cluster: l2_entry=8000000000e00000 refcount=2 ++The following inconsistencies were found and repaired: ++ ++ 0 leaked clusters ++ 4 corruptions ++ ++Double checking the fixed image now... ++No errors were found on the image. ++read 4194304/4194304 bytes at offset 0 ++4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 4194304/4194304 bytes at offset 4194304 ++4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 3a89aed..ba7a2d1 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -212,4 +212,5 @@ + 211 rw auto quick + 212 rw auto quick + 213 rw auto quick ++214 rw auto + 218 rw auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Test-help-option-for-unsupporting-formats.patch b/SOURCES/kvm-iotests-Test-help-option-for-unsupporting-formats.patch new file mode 100644 index 0000000..242dd20 --- /dev/null +++ b/SOURCES/kvm-iotests-Test-help-option-for-unsupporting-formats.patch @@ -0,0 +1,99 @@ +From b9ae582a81d3f92dcb23adce12c0068a6e3abaf5 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:59:42 +0200 +Subject: [PATCH 069/268] iotests: Test help option for unsupporting formats + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-7-mreitz@redhat.com> +Patchwork-id: 80760 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 6/7] iotests: Test help option for unsupporting formats +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +This adds test cases to 082 for qemu-img create/convert/amend "-o help" +on formats that do not support creation or amendment, respectively. + +Signed-off-by: Max Reitz +Reviewed-by: John Snow +Reviewed-by: Eric Blake +Message-id: 20180509210023.20283-7-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit e53995eb19b546b18d1a34cd6eaa07faedbade79) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/082 | 9 +++++++++ + tests/qemu-iotests/082.out | 9 +++++++++ + 2 files changed, 18 insertions(+) + +diff --git a/tests/qemu-iotests/082 b/tests/qemu-iotests/082 +index d5c83d4..a872f77 100755 +--- a/tests/qemu-iotests/082 ++++ b/tests/qemu-iotests/082 +@@ -97,6 +97,9 @@ run_qemu_img create -f $IMGFMT -o backing_file="$TEST_IMG" -o ,, -o help "$TEST_ + run_qemu_img create -f $IMGFMT -o help + run_qemu_img create -o help + ++# Try help option for a format that does not support creation ++run_qemu_img create -f bochs -o help ++ + echo + echo === convert: Options specified more than once === + +@@ -151,6 +154,9 @@ run_qemu_img convert -O $IMGFMT -o backing_file="$TEST_IMG" -o ,, -o help "$TEST + run_qemu_img convert -O $IMGFMT -o help + run_qemu_img convert -o help + ++# Try help option for a format that does not support creation ++run_qemu_img convert -O bochs -o help ++ + echo + echo === amend: Options specified more than once === + +@@ -201,6 +207,9 @@ run_qemu_img amend -f $IMGFMT -o backing_file="$TEST_IMG" -o ,, -o help "$TEST_I + run_qemu_img amend -f $IMGFMT -o help + run_qemu_img convert -o help + ++# Try help option for a format that does not support amendment ++run_qemu_img amend -f bochs -o help ++ + # success, all done + echo "*** done" + rm -f $seq.full +diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out +index 4e52dce..60ef87c 100644 +--- a/tests/qemu-iotests/082.out ++++ b/tests/qemu-iotests/082.out +@@ -249,6 +249,9 @@ Testing: create -o help + Supported options: + size Virtual disk size + ++Testing: create -f bochs -o help ++qemu-img: Format driver 'bochs' does not support image creation ++ + === convert: Options specified more than once === + + Testing: create -f qcow2 TEST_DIR/t.qcow2 128M +@@ -502,6 +505,9 @@ Testing: convert -o help + Supported options: + size Virtual disk size + ++Testing: convert -O bochs -o help ++qemu-img: Format driver 'bochs' does not support image creation ++ + === amend: Options specified more than once === + + Testing: amend -f foo -f qcow2 -o lazy_refcounts=on TEST_DIR/t.qcow2 +@@ -763,4 +769,7 @@ Note that not all of these options may be amendable. + Testing: convert -o help + Supported options: + size Virtual disk size ++ ++Testing: amend -f bochs -o help ++qemu-img: Format driver 'bochs' does not support option amendment + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Test-migration-with-blockdev.patch b/SOURCES/kvm-iotests-Test-migration-with-blockdev.patch new file mode 100644 index 0000000..b7d374d --- /dev/null +++ b/SOURCES/kvm-iotests-Test-migration-with-blockdev.patch @@ -0,0 +1,210 @@ +From 39bdd3797d4fc450f129a5923f885017f54ad6c4 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Fri, 14 Dec 2018 09:49:46 +0000 +Subject: [PATCH 2/2] iotests: Test migration with -blockdev + +RH-Author: Kevin Wolf +Message-id: <20181214094946.26226-3-kwolf@redhat.com> +Patchwork-id: 83512 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 2/2] iotests: Test migration with -blockdev +Bugzilla: 1659395 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +Check that block node activation and inactivation works with a block +graph that is built with individually created nodes. + +RHEL: Disabled query-migrate call on the destination; this returns only +an empty object in 2.12. Changed reference output for Python 2 (we lack +the compatibility patches from upstream that make the output independent +from the Python version). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 330ca111ea0979d8c6fc9b3958f72d6dce164d5a) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/234 | 123 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/234.out | 28 +++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 152 insertions(+) + create mode 100755 tests/qemu-iotests/234 + create mode 100644 tests/qemu-iotests/234.out + +diff --git a/tests/qemu-iotests/234 b/tests/qemu-iotests/234 +new file mode 100755 +index 0000000..2b0f869 +--- /dev/null ++++ b/tests/qemu-iotests/234 +@@ -0,0 +1,123 @@ ++#!/usr/bin/env python ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++# Creator/Owner: Kevin Wolf ++# ++# Check that block node activation and inactivation works with a block graph ++# that is built with individually created nodes ++ ++import iotests ++import os ++ ++iotests.verify_image_format(supported_fmts=['qcow2']) ++iotests.verify_platform(['linux']) ++ ++with iotests.FilePath('img') as img_path, \ ++ iotests.FilePath('backing') as backing_path, \ ++ iotests.FilePath('mig_fifo_a') as fifo_a, \ ++ iotests.FilePath('mig_fifo_b') as fifo_b, \ ++ iotests.VM(path_suffix='a') as vm_a, \ ++ iotests.VM(path_suffix='b') as vm_b: ++ ++ iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, backing_path, '64M') ++ iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, img_path, '64M') ++ ++ os.mkfifo(fifo_a) ++ os.mkfifo(fifo_b) ++ ++ iotests.log('Launching source VM...') ++ (vm_a.add_blockdev('file,filename=%s,node-name=drive0-file' % (img_path)) ++ .add_blockdev('%s,file=drive0-file,node-name=drive0' % (iotests.imgfmt)) ++ .add_blockdev('file,filename=%s,node-name=drive0-backing-file' % (backing_path)) ++ .add_blockdev('%s,file=drive0-backing-file,node-name=drive0-backing' % (iotests.imgfmt)) ++ .launch()) ++ ++ iotests.log('Launching destination VM...') ++ (vm_b.add_blockdev('file,filename=%s,node-name=drive0-file' % (img_path)) ++ .add_blockdev('%s,file=drive0-file,node-name=drive0' % (iotests.imgfmt)) ++ .add_blockdev('file,filename=%s,node-name=drive0-backing-file' % (backing_path)) ++ .add_blockdev('%s,file=drive0-backing-file,node-name=drive0-backing' % (iotests.imgfmt)) ++ .add_incoming("exec: cat '%s'" % (fifo_a)) ++ .launch()) ++ ++ # Add a child node that was created after the parent node. The reverse case ++ # is covered by the -blockdev options above. ++ iotests.log(vm_a.qmp('blockdev-snapshot', node='drive0-backing', ++ overlay='drive0')) ++ iotests.log(vm_b.qmp('blockdev-snapshot', node='drive0-backing', ++ overlay='drive0')) ++ ++ iotests.log('Enabling migration QMP events on A...') ++ iotests.log(vm_a.qmp('migrate-set-capabilities', capabilities=[ ++ { ++ 'capability': 'events', ++ 'state': True ++ } ++ ])) ++ ++ iotests.log('Starting migration to B...') ++ iotests.log(vm_a.qmp('migrate', uri='exec:cat >%s' % (fifo_a))) ++ with iotests.Timeout(3, 'Migration does not complete'): ++ while True: ++ event = vm_a.event_wait('MIGRATION') ++ iotests.log(event, filters=[iotests.filter_qmp_event]) ++ if event['data']['status'] == 'completed': ++ break ++ ++ iotests.log(vm_a.qmp('query-migrate')['return']['status']) ++ # Returns only {} on this QEMU version (no status) ++ # iotests.log(vm_b.qmp('query-migrate')['return']['status']) ++ ++ iotests.log(vm_a.qmp('query-status')) ++ iotests.log(vm_b.qmp('query-status')) ++ ++ iotests.log('Add a second parent to drive0-file...') ++ iotests.log(vm_b.qmp('blockdev-add', driver='raw', file='drive0-file', ++ node_name='drive0-raw')) ++ ++ iotests.log('Restart A with -incoming and second parent...') ++ vm_a.shutdown() ++ (vm_a.add_blockdev('raw,file=drive0-file,node-name=drive0-raw') ++ .add_incoming("exec: cat '%s'" % (fifo_b)) ++ .launch()) ++ ++ iotests.log(vm_a.qmp('blockdev-snapshot', node='drive0-backing', ++ overlay='drive0')) ++ ++ iotests.log('Enabling migration QMP events on B...') ++ iotests.log(vm_b.qmp('migrate-set-capabilities', capabilities=[ ++ { ++ 'capability': 'events', ++ 'state': True ++ } ++ ])) ++ ++ iotests.log('Starting migration back to A...') ++ iotests.log(vm_b.qmp('migrate', uri='exec:cat >%s' % (fifo_b))) ++ with iotests.Timeout(3, 'Migration does not complete'): ++ while True: ++ event = vm_b.event_wait('MIGRATION') ++ iotests.log(event, filters=[iotests.filter_qmp_event]) ++ if event['data']['status'] == 'completed': ++ break ++ ++ # Returns only {} on this QEMU version (no status) ++ # iotests.log(vm_a.qmp('query-migrate')['return']['status']) ++ iotests.log(vm_b.qmp('query-migrate')['return']['status']) ++ ++ iotests.log(vm_a.qmp('query-status')) ++ iotests.log(vm_b.qmp('query-status')) +diff --git a/tests/qemu-iotests/234.out b/tests/qemu-iotests/234.out +new file mode 100644 +index 0000000..c7cd95f +--- /dev/null ++++ b/tests/qemu-iotests/234.out +@@ -0,0 +1,28 @@ ++Launching source VM... ++Launching destination VM... ++{u'return': {}} ++{u'return': {}} ++Enabling migration QMP events on A... ++{u'return': {}} ++Starting migration to B... ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'setup'}, u'event': u'MIGRATION'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'active'}, u'event': u'MIGRATION'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'completed'}, u'event': u'MIGRATION'} ++completed ++{u'return': {u'status': u'postmigrate', u'singlestep': False, u'running': False}} ++{u'return': {u'status': u'running', u'singlestep': False, u'running': True}} ++Add a second parent to drive0-file... ++{u'return': {}} ++Restart A with -incoming and second parent... ++{u'return': {}} ++Enabling migration QMP events on B... ++{u'return': {}} ++Starting migration back to A... ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'setup'}, u'event': u'MIGRATION'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'active'}, u'event': u'MIGRATION'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'completed'}, u'event': u'MIGRATION'} ++completed ++{u'return': {u'status': u'running', u'singlestep': False, u'running': True}} ++{u'return': {u'status': u'postmigrate', u'singlestep': False, u'running': False}} +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 303daa5..b30689e 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -224,3 +224,4 @@ + 226 auto quick + 229 auto quick + 231 auto quick ++234 auto quick migration +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-Test-post-backing-convert-target-behavior.patch b/SOURCES/kvm-iotests-Test-post-backing-convert-target-behavior.patch new file mode 100644 index 0000000..6944d7d --- /dev/null +++ b/SOURCES/kvm-iotests-Test-post-backing-convert-target-behavior.patch @@ -0,0 +1,116 @@ +From 79f400ed6efa42347cd551a4a2da3ba1688c431c Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 18:04:51 +0200 +Subject: [PATCH 074/268] iotests: Test post-backing convert target behavior + +RH-Author: Max Reitz +Message-id: <20180618180451.23808-3-mreitz@redhat.com> +Patchwork-id: 80797 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/2] iotests: Test post-backing convert target behavior +Bugzilla: 1527898 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Jeffrey Cody + +This adds a test case to 122 for what happens when you convert to a +target with a backing file that is shorter than the target, and the +image format does not support efficient zero writes (as is the case with +qcow2 v2). + +Signed-off-by: Max Reitz +Message-id: 20180501165750.19242-3-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 0682854f899d03c78befce9f5aae4a7e37655d25) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/122 | 42 ++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/122.out | 18 ++++++++++++++++++ + 2 files changed, 60 insertions(+) + +diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122 +index 45b359c..d8c8ad7 100755 +--- a/tests/qemu-iotests/122 ++++ b/tests/qemu-iotests/122 +@@ -77,6 +77,48 @@ $QEMU_IO -c "read -P 0 0 3M" "$TEST_IMG".orig 2>&1 | _filter_qemu_io | _filter_t + + + echo ++echo "=== Converting to an overlay larger than its backing file ===" ++echo ++ ++TEST_IMG="$TEST_IMG".base _make_test_img 256M ++# Needs to be at least how much an L2 table covers ++# (64 kB/entry * 64 kB / 8 B/entry = 512 MB) ++# That way, qcow2 will yield at least two status request responses. ++# With just a single response, it would always say "Allocated in the ++# backing file", so the optimization qemu-img convert tries to do is ++# done automatically. Once it has to be queried twice, however (and ++# one of the queries is completely after the end of the backing file), ++# the block layer will automatically add a ZERO flag that qemu-img ++# convert used to follow up with a zero write to the target. ++# We do not want such a zero write, however, because we are past the ++# end of the backing file on the target as well, so we do not need to ++# write anything there. ++_make_test_img -b "$TEST_IMG".base 768M ++ ++# Use compat=0.10 as the output so there is no zero cluster support ++$QEMU_IMG convert -O $IMGFMT -B "$TEST_IMG".base -o compat=0.10 \ ++ "$TEST_IMG" "$TEST_IMG".orig ++# See that nothing has been allocated past 64M ++$QEMU_IMG map "$TEST_IMG".orig | _filter_qemu_img_map ++ ++echo ++ ++# Just before the end of the backing file ++$QEMU_IO -c 'write -P 0x11 255M 1M' "$TEST_IMG".base 2>&1 | _filter_qemu_io ++# Somewhere in the second L2 table ++$QEMU_IO -c 'write -P 0x22 600M 1M' "$TEST_IMG" 2>&1 | _filter_qemu_io ++ ++$QEMU_IMG convert -O $IMGFMT -B "$TEST_IMG".base -o compat=0.10 \ ++ "$TEST_IMG" "$TEST_IMG".orig ++ ++$QEMU_IMG map "$TEST_IMG".orig | _filter_qemu_img_map ++$QEMU_IO -c 'read -P 0x11 255M 1M' \ ++ -c 'read -P 0x22 600M 1M' \ ++ "$TEST_IMG".orig \ ++ | _filter_qemu_io ++ ++ ++echo + echo "=== Concatenate multiple source images ===" + echo + +diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out +index 47d8656..6c7ee1d 100644 +--- a/tests/qemu-iotests/122.out ++++ b/tests/qemu-iotests/122.out +@@ -28,6 +28,24 @@ read 3145728/3145728 bytes at offset 0 + read 3145728/3145728 bytes at offset 0 + 3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + ++=== Converting to an overlay larger than its backing file === ++ ++Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=268435456 ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=805306368 backing_file=TEST_DIR/t.IMGFMT.base ++Offset Length File ++ ++wrote 1048576/1048576 bytes at offset 267386880 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 1048576/1048576 bytes at offset 629145600 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++Offset Length File ++0xff00000 0x100000 TEST_DIR/t.IMGFMT.base ++0x25800000 0x100000 TEST_DIR/t.IMGFMT.orig ++read 1048576/1048576 bytes at offset 267386880 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++read 1048576/1048576 bytes at offset 629145600 ++1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++ + === Concatenate multiple source images === + + Formatting 'TEST_DIR/t.IMGFMT.1', fmt=IMGFMT size=4194304 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-add-222-to-test-basic-fleecing.patch b/SOURCES/kvm-iotests-add-222-to-test-basic-fleecing.patch new file mode 100644 index 0000000..28bd0a3 --- /dev/null +++ b/SOURCES/kvm-iotests-add-222-to-test-basic-fleecing.patch @@ -0,0 +1,275 @@ +From dd5fb82a6b69419a52aa56a57d1997b286fa843f Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 24 Jul 2018 12:25:31 +0200 +Subject: [PATCH 229/268] iotests: add 222 to test basic fleecing + +RH-Author: John Snow +Message-id: <20180718225511.14878-12-jsnow@redhat.com> +Patchwork-id: 81407 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 11/35] iotests: add 222 to test basic fleecing +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +Signed-off-by: John Snow +Message-Id: <20180702194630.9360-3-jsnow@redhat.com> +Reviewed-by: Eric Blake +Signed-off-by: Eric Blake +(cherry picked from commit bacebdedbf921a2c641a34486ff543089d338f32) +Signed-off-by: John Snow +--- + tests/qemu-iotests/222 | 155 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/222.out | 67 ++++++++++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 223 insertions(+) + create mode 100644 tests/qemu-iotests/222 + create mode 100644 tests/qemu-iotests/222.out + +diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 +new file mode 100644 +index 0000000..ff3bfc1 +--- /dev/null ++++ b/tests/qemu-iotests/222 +@@ -0,0 +1,155 @@ ++#!/usr/bin/env python ++# ++# This test covers the basic fleecing workflow, which provides a ++# point-in-time snapshot of a node that can be queried over NBD. ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# John helped, too. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++# Creator/Owner: John Snow ++ ++import iotests ++from iotests import log, qemu_img, qemu_io, qemu_io_silent ++ ++iotests.verify_platform(['linux']) ++ ++patterns = [("0x5d", "0", "64k"), ++ ("0xd5", "1M", "64k"), ++ ("0xdc", "32M", "64k"), ++ ("0xcd", "0x3ff0000", "64k")] # 64M - 64K ++ ++overwrite = [("0xab", "0", "64k"), # Full overwrite ++ ("0xad", "0x00f8000", "64k"), # Partial-left (1M-32K) ++ ("0x1d", "0x2008000", "64k"), # Partial-right (32M+32K) ++ ("0xea", "0x3fe0000", "64k")] # Adjacent-left (64M - 128K) ++ ++zeroes = [("0", "0x00f8000", "32k"), # Left-end of partial-left (1M-32K) ++ ("0", "0x2010000", "32k"), # Right-end of partial-right (32M+64K) ++ ("0", "0x3fe0000", "64k")] # overwrite[3] ++ ++remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1] ++ ("0xdc", "32M", "32k"), # Left-end of partial-right [2] ++ ("0xcd", "0x3ff0000", "64k")] # patterns[3] ++ ++with iotests.FilePath('base.img') as base_img_path, \ ++ iotests.FilePath('fleece.img') as fleece_img_path, \ ++ iotests.FilePath('nbd.sock') as nbd_sock_path, \ ++ iotests.VM() as vm: ++ ++ log('--- Setting up images ---') ++ log('') ++ ++ assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0 ++ assert qemu_img('create', '-f', "qcow2", fleece_img_path, '64M') == 0 ++ ++ for p in patterns: ++ qemu_io('-f', iotests.imgfmt, ++ '-c', 'write -P%s %s %s' % p, base_img_path) ++ ++ log('Done') ++ ++ log('') ++ log('--- Launching VM ---') ++ log('') ++ ++ vm.add_drive(base_img_path) ++ vm.launch() ++ log('Done') ++ ++ log('') ++ log('--- Setting up Fleecing Graph ---') ++ log('') ++ ++ src_node = "drive0" ++ tgt_node = "fleeceNode" ++ ++ # create tgt_node backed by src_node ++ log(vm.qmp("blockdev-add", **{ ++ "driver": "qcow2", ++ "node-name": tgt_node, ++ "file": { ++ "driver": "file", ++ "filename": fleece_img_path, ++ }, ++ "backing": src_node, ++ })) ++ ++ # Establish COW from source to fleecing node ++ log(vm.qmp("blockdev-backup", ++ device=src_node, ++ target=tgt_node, ++ sync="none")) ++ ++ log('') ++ log('--- Setting up NBD Export ---') ++ log('') ++ ++ nbd_uri = 'nbd+unix:///%s?socket=%s' % (tgt_node, nbd_sock_path) ++ log(vm.qmp("nbd-server-start", ++ **{"addr": { "type": "unix", ++ "data": { "path": nbd_sock_path } } })) ++ ++ log(vm.qmp("nbd-server-add", device=tgt_node)) ++ ++ log('') ++ log('--- Sanity Check ---') ++ log('') ++ ++ for p in (patterns + zeroes): ++ cmd = "read -P%s %s %s" % p ++ log(cmd) ++ assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0 ++ ++ log('') ++ log('--- Testing COW ---') ++ log('') ++ ++ for p in overwrite: ++ cmd = "write -P%s %s %s" % p ++ log(cmd) ++ log(vm.hmp_qemu_io(src_node, cmd)) ++ ++ log('') ++ log('--- Verifying Data ---') ++ log('') ++ ++ for p in (patterns + zeroes): ++ cmd = "read -P%s %s %s" % p ++ log(cmd) ++ assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0 ++ ++ log('') ++ log('--- Cleanup ---') ++ log('') ++ ++ log(vm.qmp('block-job-cancel', device=src_node)) ++ log(vm.event_wait('BLOCK_JOB_CANCELLED'), ++ filters=[iotests.filter_qmp_event]) ++ log(vm.qmp('nbd-server-stop')) ++ log(vm.qmp('blockdev-del', node_name=tgt_node)) ++ vm.shutdown() ++ ++ log('') ++ log('--- Confirming writes ---') ++ log('') ++ ++ for p in (overwrite + remainder): ++ cmd = "read -P%s %s %s" % p ++ log(cmd) ++ assert qemu_io_silent(base_img_path, '-c', cmd) == 0 ++ ++ log('') ++ log('Done') +diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/222.out +new file mode 100644 +index 0000000..48f336a +--- /dev/null ++++ b/tests/qemu-iotests/222.out +@@ -0,0 +1,67 @@ ++--- Setting up images --- ++ ++Done ++ ++--- Launching VM --- ++ ++Done ++ ++--- Setting up Fleecing Graph --- ++ ++{u'return': {}} ++{u'return': {}} ++ ++--- Setting up NBD Export --- ++ ++{u'return': {}} ++{u'return': {}} ++ ++--- Sanity Check --- ++ ++read -P0x5d 0 64k ++read -P0xd5 1M 64k ++read -P0xdc 32M 64k ++read -P0xcd 0x3ff0000 64k ++read -P0 0x00f8000 32k ++read -P0 0x2010000 32k ++read -P0 0x3fe0000 64k ++ ++--- Testing COW --- ++ ++write -P0xab 0 64k ++{u'return': u''} ++write -P0xad 0x00f8000 64k ++{u'return': u''} ++write -P0x1d 0x2008000 64k ++{u'return': u''} ++write -P0xea 0x3fe0000 64k ++{u'return': u''} ++ ++--- Verifying Data --- ++ ++read -P0x5d 0 64k ++read -P0xd5 1M 64k ++read -P0xdc 32M 64k ++read -P0xcd 0x3ff0000 64k ++read -P0 0x00f8000 32k ++read -P0 0x2010000 32k ++read -P0 0x3fe0000 64k ++ ++--- Cleanup --- ++ ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'drive0', u'type': u'backup', u'speed': 0, u'len': 67108864, u'offset': 393216}, u'event': u'BLOCK_JOB_CANCELLED'} ++{u'return': {}} ++{u'return': {}} ++ ++--- Confirming writes --- ++ ++read -P0xab 0 64k ++read -P0xad 0x00f8000 64k ++read -P0x1d 0x2008000 64k ++read -P0xea 0x3fe0000 64k ++read -P0xd5 0x108000 32k ++read -P0xdc 32M 32k ++read -P0xcd 0x3ff0000 64k ++ ++Done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 11498fd..be55905 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -218,4 +218,5 @@ + 217 rw auto quick + 218 rw auto quick + 219 rw auto ++222 rw auto quick + 226 auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-add-test-226-for-file-driver-types.patch b/SOURCES/kvm-iotests-add-test-226-for-file-driver-types.patch new file mode 100644 index 0000000..818e665 --- /dev/null +++ b/SOURCES/kvm-iotests-add-test-226-for-file-driver-types.patch @@ -0,0 +1,150 @@ +From 07df50ffab364f9b24a903c56b9fabce2e60f65f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Fri, 13 Jul 2018 14:50:02 +0200 +Subject: [PATCH 218/268] iotests: add test 226 for file driver types + +RH-Author: Kevin Wolf +Message-id: <20180713145002.20953-3-kwolf@redhat.com> +Patchwork-id: 81351 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/2] iotests: add test 226 for file driver types +Bugzilla: 1525829 +RH-Acked-by: John Snow +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: John Snow + +Test that we're rejecting what we ought to for file, +host_driver and host_cdrom drivers. Test that we're +seeing the deprecated message for block and chardevs +on the file driver. + +Signed-off-by: John Snow +Signed-off-by: Kevin Wolf +(cherry picked from commit 2d4cb49ddaa219ed3f5a985ecf8b188aeb2b3d6b) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/226 | 66 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/226.out | 26 ++++++++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 93 insertions(+) + create mode 100755 tests/qemu-iotests/226 + create mode 100644 tests/qemu-iotests/226.out + +diff --git a/tests/qemu-iotests/226 b/tests/qemu-iotests/226 +new file mode 100755 +index 0000000..460aea2 +--- /dev/null ++++ b/tests/qemu-iotests/226 +@@ -0,0 +1,66 @@ ++#!/bin/bash ++# ++# This test covers expected filetypes for the file, host_cdrom and ++# host_device drivers. ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++# creator ++owner=jsnow@redhat.com ++ ++seq=`basename $0` ++echo "QA output created by $seq" ++ ++here=`pwd` ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ rmdir "$TEST_IMG" ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++. ./common.pattern ++ ++# Generic format, but tests file-protocol specific error handling ++_supported_fmt generic ++_supported_proto file ++_supported_os Linux ++ ++# Create something decidedly not a file, blockdev or chardev... ++mkdir "$TEST_IMG" ++ ++for PROTO in "file" "host_device" "host_cdrom"; do ++ echo ++ echo "=== Testing with driver:$PROTO ===" ++ echo ++ echo "== Testing RO ==" ++ $QEMU_IO -c "open -r -o driver=$PROTO,filename=$TEST_IMG" 2>&1 | _filter_imgfmt | _filter_testdir ++ $QEMU_IO -c "open -r -o driver=$PROTO,filename=/dev/null" 2>&1 | _filter_imgfmt ++ echo "== Testing RW ==" ++ $QEMU_IO -c "open -o driver=$PROTO,filename=$TEST_IMG" 2>&1 | _filter_imgfmt | _filter_testdir ++ $QEMU_IO -c "open -o driver=$PROTO,filename=/dev/null" 2>&1 | _filter_imgfmt ++done ++ ++# success, all done ++echo ++echo "*** done" ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/226.out b/tests/qemu-iotests/226.out +new file mode 100644 +index 0000000..8c0d060 +--- /dev/null ++++ b/tests/qemu-iotests/226.out +@@ -0,0 +1,26 @@ ++QA output created by 226 ++ ++=== Testing with driver:file === ++ ++== Testing RO == ++can't open: A regular file was expected by the 'file' driver, but something else was given ++warning: Opening a character device as a file using the 'file' driver is deprecated ++== Testing RW == ++can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory ++warning: Opening a character device as a file using the 'file' driver is deprecated ++ ++=== Testing with driver:host_device === ++ ++== Testing RO == ++can't open: 'host_device' driver expects either a character or block device ++== Testing RW == ++can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory ++ ++=== Testing with driver:host_cdrom === ++ ++== Testing RO == ++can't open: 'host_cdrom' driver expects either a character or block device ++== Testing RW == ++can't open: Could not open 'TEST_DIR/t.IMGFMT': Is a directory ++ ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 5c55adc..11498fd 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -218,3 +218,4 @@ + 217 rw auto quick + 218 rw auto quick + 219 rw auto ++226 auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-improve-169.patch b/SOURCES/kvm-iotests-improve-169.patch new file mode 100644 index 0000000..31201e4 --- /dev/null +++ b/SOURCES/kvm-iotests-improve-169.patch @@ -0,0 +1,70 @@ +From 3df2da67983b88351cbe4e8d0169cc40cc5ab282 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:27 +0000 +Subject: [PATCH 33/35] iotests: improve 169 + +RH-Author: John Snow +Message-id: <20181120181828.15132-24-jsnow@redhat.com> +Patchwork-id: 83072 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 23/24] iotests: improve 169 +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +Before previous patch, iotest 169 was actually broken for the case +test_persistent__not_migbitmap__offline_shared, while formally +passing. + +After migration log of vm_b had message: + + qemu-system-x86_64: Could not reopen qcow2 layer: Bitmap already + exists: bitmap0 + +which means that invalidation failed and bs->drv = NULL. + +It was because we've loaded bitmap twice: on open and on invalidation. + +Add code to 169, to catch such fails. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +(cherry picked from commit b9247fc1a8ffe5c367fa049f295fbb58c8ca9d05) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/169 | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169 +index df408f8..8b7947d 100755 +--- a/tests/qemu-iotests/169 ++++ b/tests/qemu-iotests/169 +@@ -24,6 +24,7 @@ import time + import itertools + import operator + import new ++import re + from iotests import qemu_img + + +@@ -133,6 +134,14 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase): + + if should_migrate: + self.vm_b.shutdown() ++ ++ # catch 'Could not reopen qcow2 layer: Bitmap already exists' ++ # possible error ++ log = self.vm_b.get_log() ++ log = re.sub(r'^\[I \d+\.\d+\] OPENED\n', '', log) ++ log = re.sub(r'\[I \+\d+\.\d+\] CLOSED\n?$', '', log) ++ self.assertEqual(log, '') ++ + # recreate vm_b, as we don't want -incoming option (this will lead + # to "cat" process left alive after test finish) + self.vm_b = iotests.VM(path_suffix='b') +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests-improve-pause_job.patch b/SOURCES/kvm-iotests-improve-pause_job.patch new file mode 100644 index 0000000..b2b5360 --- /dev/null +++ b/SOURCES/kvm-iotests-improve-pause_job.patch @@ -0,0 +1,55 @@ +From ee9edb4c02451eec077a7a6afc5ddb1b343e8ef6 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:56 +0200 +Subject: [PATCH 148/268] iotests: improve pause_job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-74-kwolf@redhat.com> +Patchwork-id: 81118 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 73/73] iotests: improve pause_job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +It's possible, that job was finished during waiting. In this case we +will see error message "Timeout waiting for job to pause" which is not +very informative. So, let's check during waiting iteration that the job +exists. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-id: 20180601115923.17159-1-vsementsov@virtuozzo.com +Signed-off-by: Max Reitz +(cherry picked from commit c1bac161bb7ad27243776e90971c51cc38c2e1b6) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 9 +++++++-- + 1 file changed, 7 insertions(+), 2 deletions(-) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 0dbfbfd..4e67fbb 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -591,9 +591,14 @@ class QMPTestCase(unittest.TestCase): + with Timeout(1, "Timeout waiting for job to pause"): + while True: + result = self.vm.qmp('query-block-jobs') ++ found = False + for job in result['return']: +- if job['device'] == job_id and job['paused'] == True and job['busy'] == False: +- return job ++ if job['device'] == job_id: ++ found = True ++ if job['paused'] == True and job['busy'] == False: ++ return job ++ break ++ assert found + + def pause_job(self, job_id='job0', wait=True): + result = self.vm.qmp('block-job-pause', device=job_id) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iotests.py-Add-qemu_io_silent.patch b/SOURCES/kvm-iotests.py-Add-qemu_io_silent.patch new file mode 100644 index 0000000..ec8a80c --- /dev/null +++ b/SOURCES/kvm-iotests.py-Add-qemu_io_silent.patch @@ -0,0 +1,52 @@ +From 2491da98d7e2b2026075fb1ab87bc1a242f3dd1e Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:43:11 +0200 +Subject: [PATCH 052/268] iotests.py: Add qemu_io_silent + +RH-Author: Max Reitz +Message-id: <20180618164312.24423-5-mreitz@redhat.com> +Patchwork-id: 80777 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 4/5] iotests.py: Add qemu_io_silent +Bugzilla: 1519617 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +With qemu-io now returning a useful exit code, some tests may find it +sufficient to just query that instead of logging (and filtering) the +whole output. + +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509194302.21585-5-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 745f2bf4a5b973d1a69b21db97f9e4219da60624) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 26e6046..2ce85a1 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -120,6 +120,15 @@ def qemu_io(*args): + sys.stderr.write('qemu-io received signal %i: %s\n' % (-exitcode, ' '.join(args))) + return subp.communicate()[0] + ++def qemu_io_silent(*args): ++ '''Run qemu-io and return the exit code, suppressing stdout''' ++ args = qemu_io_args + list(args) ++ exitcode = subprocess.call(args, stdout=open('/dev/null', 'w')) ++ if exitcode < 0: ++ sys.stderr.write('qemu-io received signal %i: %s\n' % ++ (-exitcode, ' '.join(args))) ++ return exitcode ++ + + class QemuIoInteractive: + def __init__(self, *args): +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iscsi-Create-and-use-iscsi_co_wait_for_task.patch b/SOURCES/kvm-iscsi-Create-and-use-iscsi_co_wait_for_task.patch new file mode 100644 index 0000000..98c5451 --- /dev/null +++ b/SOURCES/kvm-iscsi-Create-and-use-iscsi_co_wait_for_task.patch @@ -0,0 +1,139 @@ +From 0729a3f3d2313e3429c27004ba91bce8644e4765 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:47 +0200 +Subject: [PATCH 173/268] iscsi: Create and use iscsi_co_wait_for_task + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-8-famz@redhat.com> +Patchwork-id: 81160 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 07/13] iscsi: Create and use iscsi_co_wait_for_task +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +This loop is repeated a growing number times. Make a helper. + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Eric Blake +Message-id: 20180601092648.24614-8-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 66e75c03b2247bda6dcaa883b700291bc0f7f0ef) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/iscsi.c | 54 +++++++++++++++++------------------------------------- + 1 file changed, 17 insertions(+), 37 deletions(-) + +diff --git a/block/iscsi.c b/block/iscsi.c +index a1a0044..338f3dd 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -557,6 +557,17 @@ static inline bool iscsi_allocmap_is_valid(IscsiLun *iscsilun, + offset / iscsilun->cluster_size) == size); + } + ++static void coroutine_fn iscsi_co_wait_for_task(IscsiTask *iTask, ++ IscsiLun *iscsilun) ++{ ++ while (!iTask->complete) { ++ iscsi_set_events(iscsilun); ++ qemu_mutex_unlock(&iscsilun->mutex); ++ qemu_coroutine_yield(); ++ qemu_mutex_lock(&iscsilun->mutex); ++ } ++} ++ + static int coroutine_fn + iscsi_co_writev_flags(BlockDriverState *bs, int64_t sector_num, int nb_sectors, + QEMUIOVector *iov, int flags) +@@ -618,12 +629,7 @@ retry: + scsi_task_set_iov_out(iTask.task, (struct scsi_iovec *) iov->iov, + iov->niov); + #endif +- while (!iTask.complete) { +- iscsi_set_events(iscsilun); +- qemu_mutex_unlock(&iscsilun->mutex); +- qemu_coroutine_yield(); +- qemu_mutex_lock(&iscsilun->mutex); +- } ++ iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); +@@ -694,13 +700,7 @@ retry: + ret = -ENOMEM; + goto out_unlock; + } +- +- while (!iTask.complete) { +- iscsi_set_events(iscsilun); +- qemu_mutex_unlock(&iscsilun->mutex); +- qemu_coroutine_yield(); +- qemu_mutex_lock(&iscsilun->mutex); +- } ++ iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.do_retry) { + if (iTask.task != NULL) { +@@ -864,13 +864,8 @@ retry: + #if LIBISCSI_API_VERSION < (20160603) + scsi_task_set_iov_in(iTask.task, (struct scsi_iovec *) iov->iov, iov->niov); + #endif +- while (!iTask.complete) { +- iscsi_set_events(iscsilun); +- qemu_mutex_unlock(&iscsilun->mutex); +- qemu_coroutine_yield(); +- qemu_mutex_lock(&iscsilun->mutex); +- } + ++ iscsi_co_wait_for_task(&iTask, iscsilun); + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); + iTask.task = NULL; +@@ -907,12 +902,7 @@ retry: + return -ENOMEM; + } + +- while (!iTask.complete) { +- iscsi_set_events(iscsilun); +- qemu_mutex_unlock(&iscsilun->mutex); +- qemu_coroutine_yield(); +- qemu_mutex_lock(&iscsilun->mutex); +- } ++ iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); +@@ -1144,12 +1134,7 @@ retry: + goto out_unlock; + } + +- while (!iTask.complete) { +- iscsi_set_events(iscsilun); +- qemu_mutex_unlock(&iscsilun->mutex); +- qemu_coroutine_yield(); +- qemu_mutex_lock(&iscsilun->mutex); +- } ++ iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.task != NULL) { + scsi_free_scsi_task(iTask.task); +@@ -1245,12 +1230,7 @@ retry: + return -ENOMEM; + } + +- while (!iTask.complete) { +- iscsi_set_events(iscsilun); +- qemu_mutex_unlock(&iscsilun->mutex); +- qemu_coroutine_yield(); +- qemu_mutex_lock(&iscsilun->mutex); +- } ++ iscsi_co_wait_for_task(&iTask, iscsilun); + + if (iTask.status == SCSI_STATUS_CHECK_CONDITION && + iTask.task->sense.key == SCSI_SENSE_ILLEGAL_REQUEST && +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iscsi-Don-t-blindly-use-designator-length-in-respons.patch b/SOURCES/kvm-iscsi-Don-t-blindly-use-designator-length-in-respons.patch new file mode 100644 index 0000000..f9bfd17 --- /dev/null +++ b/SOURCES/kvm-iscsi-Don-t-blindly-use-designator-length-in-respons.patch @@ -0,0 +1,42 @@ +From b00154f43c01657e4299c486f451ad50891d80f1 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:52 +0200 +Subject: [PATCH 178/268] iscsi: Don't blindly use designator length in + response for memcpy + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-13-famz@redhat.com> +Patchwork-id: 81162 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 12/13] iscsi: Don't blindly use designator length in response for memcpy +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Per SCSI definition the designator_length we receive from INQUIRY is 8, +12 or at most 16, but we should be careful because the remote iscsi +target may misbehave, otherwise we could have a buffer overflow. + +Reported-by: Max Reitz +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/iscsi.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/iscsi.c b/block/iscsi.c +index fbcd5bb..751884d 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -2226,7 +2226,7 @@ static void iscsi_populate_target_desc(unsigned char *desc, IscsiLun *lun) + desc[5] = (dd->designator_type & 0xF) + | ((dd->association & 3) << 4); + desc[7] = dd->designator_length; +- memcpy(desc + 8, dd->designator, dd->designator_length); ++ memcpy(desc + 8, dd->designator, MIN(dd->designator_length, 20)); + + desc[28] = 0; + desc[29] = (lun->block_size >> 16) & 0xFF; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iscsi-Drop-deprecated-drive-parameter-filename.patch b/SOURCES/kvm-iscsi-Drop-deprecated-drive-parameter-filename.patch new file mode 100644 index 0000000..3052ce7 --- /dev/null +++ b/SOURCES/kvm-iscsi-Drop-deprecated-drive-parameter-filename.patch @@ -0,0 +1,82 @@ +From cd19aec9abcc51c2d0103389c5c4a28649ccaf9f Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:14 +0200 +Subject: [PATCH 016/268] iscsi: Drop deprecated -drive parameter "filename" + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-8-armbru@redhat.com> +Patchwork-id: 80740 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 07/23] iscsi: Drop deprecated -drive parameter "filename" +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Parameter "filename" is deprecated since commit 5c3ad1a6a8f, v2.10.0. +Time to get rid of it. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit deadbb8ebb5c253da9b8ed02ab51a0fadf60edc7) +Signed-off-by: Miroslav Rezanina +--- + block/iscsi.c | 23 ++--------------------- + 1 file changed, 2 insertions(+), 21 deletions(-) + +diff --git a/block/iscsi.c b/block/iscsi.c +index d19ae0e..658462b 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -1732,10 +1732,6 @@ static QemuOptsList runtime_opts = { + .name = "timeout", + .type = QEMU_OPT_NUMBER, + }, +- { +- .name = "filename", +- .type = QEMU_OPT_STRING, +- }, + { /* end of list */ } + }, + }; +@@ -1751,27 +1747,12 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, + char *initiator_name = NULL; + QemuOpts *opts; + Error *local_err = NULL; +- const char *transport_name, *portal, *target, *filename; ++ const char *transport_name, *portal, *target; + #if LIBISCSI_API_VERSION >= (20160603) + enum iscsi_transport_type transport; + #endif + int i, ret = 0, timeout = 0, lun; + +- /* If we are given a filename, parse the filename, with precedence given to +- * filename encoded options */ +- filename = qdict_get_try_str(options, "filename"); +- if (filename) { +- warn_report("'filename' option specified. " +- "This is an unsupported option, and may be deprecated " +- "in the future"); +- iscsi_parse_filename(filename, options, &local_err); +- if (local_err) { +- ret = -EINVAL; +- error_propagate(errp, local_err); +- goto exit; +- } +- } +- + opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); + qemu_opts_absorb_qdict(opts, options, &local_err); + if (local_err) { +@@ -1989,7 +1970,7 @@ out: + } + memset(iscsilun, 0, sizeof(IscsiLun)); + } +-exit: ++ + return ret; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iscsi-Implement-copy-offloading.patch b/SOURCES/kvm-iscsi-Implement-copy-offloading.patch new file mode 100644 index 0000000..6e1385a --- /dev/null +++ b/SOURCES/kvm-iscsi-Implement-copy-offloading.patch @@ -0,0 +1,291 @@ +From d1046c844b369fb9cebc8f4f64274a9642152526 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:48 +0200 +Subject: [PATCH 174/268] iscsi: Implement copy offloading + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-9-famz@redhat.com> +Patchwork-id: 81159 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 08/13] iscsi: Implement copy offloading +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Issue EXTENDED COPY (LID1) command to implement the copy_range API. + +The parameter data construction code is modified from libiscsi's +iscsi-dd.c. + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Message-id: 20180601092648.24614-9-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 604dfaaa3270081da689991afe83d94d3e8231df) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/iscsi.c | 219 +++++++++++++++++++++++++++++++++++++++++++++++ + include/scsi/constants.h | 4 + + 2 files changed, 223 insertions(+) + +diff --git a/block/iscsi.c b/block/iscsi.c +index 338f3dd..fbcd5bb 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -2187,6 +2187,221 @@ static void coroutine_fn iscsi_co_invalidate_cache(BlockDriverState *bs, + iscsi_allocmap_invalidate(iscsilun); + } + ++static int coroutine_fn iscsi_co_copy_range_from(BlockDriverState *bs, ++ BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags flags) ++{ ++ return bdrv_co_copy_range_to(src, src_offset, dst, dst_offset, bytes, flags); ++} ++ ++static struct scsi_task *iscsi_xcopy_task(int param_len) ++{ ++ struct scsi_task *task; ++ ++ task = g_new0(struct scsi_task, 1); ++ ++ task->cdb[0] = EXTENDED_COPY; ++ task->cdb[10] = (param_len >> 24) & 0xFF; ++ task->cdb[11] = (param_len >> 16) & 0xFF; ++ task->cdb[12] = (param_len >> 8) & 0xFF; ++ task->cdb[13] = param_len & 0xFF; ++ task->cdb_size = 16; ++ task->xfer_dir = SCSI_XFER_WRITE; ++ task->expxferlen = param_len; ++ ++ return task; ++} ++ ++static void iscsi_populate_target_desc(unsigned char *desc, IscsiLun *lun) ++{ ++ struct scsi_inquiry_device_designator *dd = lun->dd; ++ ++ memset(desc, 0, 32); ++ desc[0] = 0xE4; /* IDENT_DESCR_TGT_DESCR */ ++ desc[4] = dd->code_set; ++ desc[5] = (dd->designator_type & 0xF) ++ | ((dd->association & 3) << 4); ++ desc[7] = dd->designator_length; ++ memcpy(desc + 8, dd->designator, dd->designator_length); ++ ++ desc[28] = 0; ++ desc[29] = (lun->block_size >> 16) & 0xFF; ++ desc[30] = (lun->block_size >> 8) & 0xFF; ++ desc[31] = lun->block_size & 0xFF; ++} ++ ++static void iscsi_xcopy_desc_hdr(uint8_t *hdr, int dc, int cat, int src_index, ++ int dst_index) ++{ ++ hdr[0] = 0x02; /* BLK_TO_BLK_SEG_DESCR */ ++ hdr[1] = ((dc << 1) | cat) & 0xFF; ++ hdr[2] = (XCOPY_BLK2BLK_SEG_DESC_SIZE >> 8) & 0xFF; ++ /* don't account for the first 4 bytes in descriptor header*/ ++ hdr[3] = (XCOPY_BLK2BLK_SEG_DESC_SIZE - 4 /* SEG_DESC_SRC_INDEX_OFFSET */) & 0xFF; ++ hdr[4] = (src_index >> 8) & 0xFF; ++ hdr[5] = src_index & 0xFF; ++ hdr[6] = (dst_index >> 8) & 0xFF; ++ hdr[7] = dst_index & 0xFF; ++} ++ ++static void iscsi_xcopy_populate_desc(uint8_t *desc, int dc, int cat, ++ int src_index, int dst_index, int num_blks, ++ uint64_t src_lba, uint64_t dst_lba) ++{ ++ iscsi_xcopy_desc_hdr(desc, dc, cat, src_index, dst_index); ++ ++ /* The caller should verify the request size */ ++ assert(num_blks < 65536); ++ desc[10] = (num_blks >> 8) & 0xFF; ++ desc[11] = num_blks & 0xFF; ++ desc[12] = (src_lba >> 56) & 0xFF; ++ desc[13] = (src_lba >> 48) & 0xFF; ++ desc[14] = (src_lba >> 40) & 0xFF; ++ desc[15] = (src_lba >> 32) & 0xFF; ++ desc[16] = (src_lba >> 24) & 0xFF; ++ desc[17] = (src_lba >> 16) & 0xFF; ++ desc[18] = (src_lba >> 8) & 0xFF; ++ desc[19] = src_lba & 0xFF; ++ desc[20] = (dst_lba >> 56) & 0xFF; ++ desc[21] = (dst_lba >> 48) & 0xFF; ++ desc[22] = (dst_lba >> 40) & 0xFF; ++ desc[23] = (dst_lba >> 32) & 0xFF; ++ desc[24] = (dst_lba >> 24) & 0xFF; ++ desc[25] = (dst_lba >> 16) & 0xFF; ++ desc[26] = (dst_lba >> 8) & 0xFF; ++ desc[27] = dst_lba & 0xFF; ++} ++ ++static void iscsi_xcopy_populate_header(unsigned char *buf, int list_id, int str, ++ int list_id_usage, int prio, ++ int tgt_desc_len, ++ int seg_desc_len, int inline_data_len) ++{ ++ buf[0] = list_id; ++ buf[1] = ((str & 1) << 5) | ((list_id_usage & 3) << 3) | (prio & 7); ++ buf[2] = (tgt_desc_len >> 8) & 0xFF; ++ buf[3] = tgt_desc_len & 0xFF; ++ buf[8] = (seg_desc_len >> 24) & 0xFF; ++ buf[9] = (seg_desc_len >> 16) & 0xFF; ++ buf[10] = (seg_desc_len >> 8) & 0xFF; ++ buf[11] = seg_desc_len & 0xFF; ++ buf[12] = (inline_data_len >> 24) & 0xFF; ++ buf[13] = (inline_data_len >> 16) & 0xFF; ++ buf[14] = (inline_data_len >> 8) & 0xFF; ++ buf[15] = inline_data_len & 0xFF; ++} ++ ++static void iscsi_xcopy_data(struct iscsi_data *data, ++ IscsiLun *src, int64_t src_lba, ++ IscsiLun *dst, int64_t dst_lba, ++ uint16_t num_blocks) ++{ ++ uint8_t *buf; ++ const int src_offset = XCOPY_DESC_OFFSET; ++ const int dst_offset = XCOPY_DESC_OFFSET + IDENT_DESCR_TGT_DESCR_SIZE; ++ const int seg_offset = dst_offset + IDENT_DESCR_TGT_DESCR_SIZE; ++ ++ data->size = XCOPY_DESC_OFFSET + ++ IDENT_DESCR_TGT_DESCR_SIZE * 2 + ++ XCOPY_BLK2BLK_SEG_DESC_SIZE; ++ data->data = g_malloc0(data->size); ++ buf = data->data; ++ ++ /* Initialise the parameter list header */ ++ iscsi_xcopy_populate_header(buf, 1, 0, 2 /* LIST_ID_USAGE_DISCARD */, ++ 0, 2 * IDENT_DESCR_TGT_DESCR_SIZE, ++ XCOPY_BLK2BLK_SEG_DESC_SIZE, ++ 0); ++ ++ /* Initialise CSCD list with one src + one dst descriptor */ ++ iscsi_populate_target_desc(&buf[src_offset], src); ++ iscsi_populate_target_desc(&buf[dst_offset], dst); ++ ++ /* Initialise one segment descriptor */ ++ iscsi_xcopy_populate_desc(&buf[seg_offset], 0, 0, 0, 1, num_blocks, ++ src_lba, dst_lba); ++} ++ ++static int coroutine_fn iscsi_co_copy_range_to(BlockDriverState *bs, ++ BdrvChild *src, ++ uint64_t src_offset, ++ BdrvChild *dst, ++ uint64_t dst_offset, ++ uint64_t bytes, ++ BdrvRequestFlags flags) ++{ ++ IscsiLun *dst_lun = dst->bs->opaque; ++ IscsiLun *src_lun; ++ struct IscsiTask iscsi_task; ++ struct iscsi_data data; ++ int r = 0; ++ int block_size; ++ ++ if (src->bs->drv->bdrv_co_copy_range_to != iscsi_co_copy_range_to) { ++ return -ENOTSUP; ++ } ++ src_lun = src->bs->opaque; ++ ++ if (!src_lun->dd || !dst_lun->dd) { ++ return -ENOTSUP; ++ } ++ if (!is_byte_request_lun_aligned(dst_offset, bytes, dst_lun)) { ++ return -ENOTSUP; ++ } ++ if (!is_byte_request_lun_aligned(src_offset, bytes, src_lun)) { ++ return -ENOTSUP; ++ } ++ if (dst_lun->block_size != src_lun->block_size || ++ !dst_lun->block_size) { ++ return -ENOTSUP; ++ } ++ ++ block_size = dst_lun->block_size; ++ if (bytes / block_size > 65535) { ++ return -ENOTSUP; ++ } ++ ++ iscsi_xcopy_data(&data, ++ src_lun, src_offset / block_size, ++ dst_lun, dst_offset / block_size, ++ bytes / block_size); ++ ++ iscsi_co_init_iscsitask(dst_lun, &iscsi_task); ++ ++ qemu_mutex_lock(&dst_lun->mutex); ++ iscsi_task.task = iscsi_xcopy_task(data.size); ++retry: ++ if (iscsi_scsi_command_async(dst_lun->iscsi, dst_lun->lun, ++ iscsi_task.task, iscsi_co_generic_cb, ++ &data, ++ &iscsi_task) != 0) { ++ r = -EIO; ++ goto out_unlock; ++ } ++ ++ iscsi_co_wait_for_task(&iscsi_task, dst_lun); ++ ++ if (iscsi_task.do_retry) { ++ iscsi_task.complete = 0; ++ goto retry; ++ } ++ ++ if (iscsi_task.status != SCSI_STATUS_GOOD) { ++ r = iscsi_task.err_code; ++ goto out_unlock; ++ } ++ ++out_unlock: ++ g_free(iscsi_task.task); ++ qemu_mutex_unlock(&dst_lun->mutex); ++ g_free(iscsi_task.err_str); ++ return r; ++} ++ + static QemuOptsList iscsi_create_opts = { + .name = "iscsi-create-opts", + .head = QTAILQ_HEAD_INITIALIZER(iscsi_create_opts.head), +@@ -2221,6 +2436,8 @@ static BlockDriver bdrv_iscsi = { + + .bdrv_co_block_status = iscsi_co_block_status, + .bdrv_co_pdiscard = iscsi_co_pdiscard, ++ .bdrv_co_copy_range_from = iscsi_co_copy_range_from, ++ .bdrv_co_copy_range_to = iscsi_co_copy_range_to, + .bdrv_co_pwrite_zeroes = iscsi_co_pwrite_zeroes, + .bdrv_co_readv = iscsi_co_readv, + .bdrv_co_writev_flags = iscsi_co_writev_flags, +@@ -2256,6 +2473,8 @@ static BlockDriver bdrv_iser = { + + .bdrv_co_block_status = iscsi_co_block_status, + .bdrv_co_pdiscard = iscsi_co_pdiscard, ++ .bdrv_co_copy_range_from = iscsi_co_copy_range_from, ++ .bdrv_co_copy_range_to = iscsi_co_copy_range_to, + .bdrv_co_pwrite_zeroes = iscsi_co_pwrite_zeroes, + .bdrv_co_readv = iscsi_co_readv, + .bdrv_co_writev_flags = iscsi_co_writev_flags, +diff --git a/include/scsi/constants.h b/include/scsi/constants.h +index a141dd7..083a8e8 100644 +--- a/include/scsi/constants.h ++++ b/include/scsi/constants.h +@@ -311,4 +311,8 @@ + #define MMC_PROFILE_HDDVD_RW_DL 0x005A + #define MMC_PROFILE_INVALID 0xFFFF + ++#define XCOPY_DESC_OFFSET 16 ++#define IDENT_DESCR_TGT_DESCR_SIZE 32 ++#define XCOPY_BLK2BLK_SEG_DESC_SIZE 28 ++ + #endif +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iscsi-Query-and-save-device-designator-when-opening.patch b/SOURCES/kvm-iscsi-Query-and-save-device-designator-when-opening.patch new file mode 100644 index 0000000..59a3953 --- /dev/null +++ b/SOURCES/kvm-iscsi-Query-and-save-device-designator-when-opening.patch @@ -0,0 +1,112 @@ +From 35bdd02a4c49dffc4fe6daccf6ba505f5c1a3bcd Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:46 +0200 +Subject: [PATCH 172/268] iscsi: Query and save device designator when opening + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-7-famz@redhat.com> +Patchwork-id: 81157 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 06/13] iscsi: Query and save device designator when opening +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +The device designator data returned in INQUIRY command will be useful to +fill in source/target fields during copy offloading. Do this when +connecting to the target and save the data for later use. + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Message-id: 20180601092648.24614-7-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit cc9743c236cce8a35449e3ef67140287b68bb705) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/iscsi.c | 41 +++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 41 insertions(+) + +diff --git a/block/iscsi.c b/block/iscsi.c +index 1705187..a1a0044 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -69,6 +69,7 @@ typedef struct IscsiLun { + QemuMutex mutex; + struct scsi_inquiry_logical_block_provisioning lbp; + struct scsi_inquiry_block_limits bl; ++ struct scsi_inquiry_device_designator *dd; + unsigned char *zeroblock; + /* The allocmap tracks which clusters (pages) on the iSCSI target are + * allocated and which are not. In case a target returns zeros for +@@ -1737,6 +1738,30 @@ static QemuOptsList runtime_opts = { + }, + }; + ++static void iscsi_save_designator(IscsiLun *lun, ++ struct scsi_inquiry_device_identification *inq_di) ++{ ++ struct scsi_inquiry_device_designator *desig, *copy = NULL; ++ ++ for (desig = inq_di->designators; desig; desig = desig->next) { ++ if (desig->association || ++ desig->designator_type > SCSI_DESIGNATOR_TYPE_NAA) { ++ continue; ++ } ++ /* NAA works better than T10 vendor ID based designator. */ ++ if (!copy || copy->designator_type < desig->designator_type) { ++ copy = desig; ++ } ++ } ++ if (copy) { ++ lun->dd = g_new(struct scsi_inquiry_device_designator, 1); ++ *lun->dd = *copy; ++ lun->dd->next = NULL; ++ lun->dd->designator = g_malloc(copy->designator_length); ++ memcpy(lun->dd->designator, copy->designator, copy->designator_length); ++ } ++} ++ + static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, + Error **errp) + { +@@ -1904,6 +1929,7 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, + struct scsi_task *inq_task; + struct scsi_inquiry_logical_block_provisioning *inq_lbp; + struct scsi_inquiry_block_limits *inq_bl; ++ struct scsi_inquiry_device_identification *inq_di; + switch (inq_vpd->pages[i]) { + case SCSI_INQUIRY_PAGECODE_LOGICAL_BLOCK_PROVISIONING: + inq_task = iscsi_do_inquiry(iscsilun->iscsi, iscsilun->lun, 1, +@@ -1929,6 +1955,17 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, + sizeof(struct scsi_inquiry_block_limits)); + scsi_free_scsi_task(inq_task); + break; ++ case SCSI_INQUIRY_PAGECODE_DEVICE_IDENTIFICATION: ++ inq_task = iscsi_do_inquiry(iscsilun->iscsi, iscsilun->lun, 1, ++ SCSI_INQUIRY_PAGECODE_DEVICE_IDENTIFICATION, ++ (void **) &inq_di, errp); ++ if (inq_task == NULL) { ++ ret = -EINVAL; ++ goto out; ++ } ++ iscsi_save_designator(iscsilun, inq_di); ++ scsi_free_scsi_task(inq_task); ++ break; + default: + break; + } +@@ -1985,6 +2022,10 @@ static void iscsi_close(BlockDriverState *bs) + iscsi_logout_sync(iscsi); + } + iscsi_destroy_context(iscsi); ++ if (iscsilun->dd) { ++ g_free(iscsilun->dd->designator); ++ g_free(iscsilun->dd); ++ } + g_free(iscsilun->zeroblock); + iscsi_allocmap_free(iscsilun); + qemu_mutex_destroy(&iscsilun->mutex); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-iscsi-Support-auto-read-only-option.patch b/SOURCES/kvm-iscsi-Support-auto-read-only-option.patch new file mode 100644 index 0000000..d308345 --- /dev/null +++ b/SOURCES/kvm-iscsi-Support-auto-read-only-option.patch @@ -0,0 +1,49 @@ +From 8cc07ae02e7cfda9605331cae792924bf18a9221 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:39 +0000 +Subject: [PATCH 09/14] iscsi: Support auto-read-only option + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-10-kwolf@redhat.com> +Patchwork-id: 83957 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 09/12] iscsi: Support auto-read-only option +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +If read-only=off, but auto-read-only=on is given, open the volume +read-write if we have the permissions, but instead of erroring out for +read-only volumes, just degrade to read-only. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 8f3bf50d340372662a35564c669e567c6c846943) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/iscsi.c | 8 +++++--- + 1 file changed, 5 insertions(+), 3 deletions(-) + +diff --git a/block/iscsi.c b/block/iscsi.c +index 2b45458..c412b12 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -1877,9 +1877,11 @@ static int iscsi_open(BlockDriverState *bs, QDict *options, int flags, + /* Check the write protect flag of the LUN if we want to write */ + if (iscsilun->type == TYPE_DISK && (flags & BDRV_O_RDWR) && + iscsilun->write_protected) { +- error_setg(errp, "Cannot open a write protected LUN as read-write"); +- ret = -EACCES; +- goto out; ++ ret = bdrv_apply_auto_read_only(bs, "LUN is write protected", errp); ++ if (ret < 0) { ++ goto out; ++ } ++ flags &= ~BDRV_O_RDWR; + } + + iscsi_readcapacity_sync(iscsilun, &local_err); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-JOB_STATUS_CHANGE-QMP-event.patch b/SOURCES/kvm-job-Add-JOB_STATUS_CHANGE-QMP-event.patch new file mode 100644 index 0000000..6066fbf --- /dev/null +++ b/SOURCES/kvm-job-Add-JOB_STATUS_CHANGE-QMP-event.patch @@ -0,0 +1,1333 @@ +From 3ae077b6e0f0fa7d2875929ee602eef655d67181 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:31 +0200 +Subject: [PATCH 123/268] job: Add JOB_STATUS_CHANGE QMP event + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-49-kwolf@redhat.com> +Patchwork-id: 81121 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 48/73] job: Add JOB_STATUS_CHANGE QMP event +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a QMP event that is emitted whenever a job transitions from +one status to another. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 1dac83f1a10c4c66858075970e199f4e4a8d8b71) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + job.c | 10 +++ + qapi/job.json | 14 ++++ + tests/qemu-iotests/030 | 17 +++- + tests/qemu-iotests/040 | 2 + + tests/qemu-iotests/041 | 17 +++- + tests/qemu-iotests/094.out | 7 ++ + tests/qemu-iotests/095 | 2 +- + tests/qemu-iotests/095.out | 6 ++ + tests/qemu-iotests/109 | 2 +- + tests/qemu-iotests/109.out | 178 ++++++++++++++++++++++++++++++++++++------ + tests/qemu-iotests/124 | 8 ++ + tests/qemu-iotests/127.out | 7 ++ + tests/qemu-iotests/141 | 13 ++- + tests/qemu-iotests/141.out | 29 +++++++ + tests/qemu-iotests/144 | 2 +- + tests/qemu-iotests/144.out | 7 ++ + tests/qemu-iotests/156 | 2 +- + tests/qemu-iotests/156.out | 7 ++ + tests/qemu-iotests/185 | 12 +-- + tests/qemu-iotests/185.out | 10 +++ + tests/qemu-iotests/191 | 4 +- + tests/qemu-iotests/191.out | 132 +++++++++++++++++++++++++++++++ + tests/qemu-iotests/iotests.py | 5 ++ + 23 files changed, 449 insertions(+), 44 deletions(-) + +diff --git a/job.c b/job.c +index 2046d2f..599a104 100644 +--- a/job.c ++++ b/job.c +@@ -30,6 +30,7 @@ + #include "qemu/id.h" + #include "qemu/main-loop.h" + #include "trace-root.h" ++#include "qapi/qapi-events-job.h" + + static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); + +@@ -157,6 +158,11 @@ static int job_txn_apply(JobTxn *txn, int fn(Job *), bool lock) + return rc; + } + ++static bool job_is_internal(Job *job) ++{ ++ return (job->id == NULL); ++} ++ + static void job_state_transition(Job *job, JobStatus s1) + { + JobStatus s0 = job->status; +@@ -166,6 +172,10 @@ static void job_state_transition(Job *job, JobStatus s1) + JobStatus_str(s0), JobStatus_str(s1)); + assert(JobSTT[s0][s1]); + job->status = s1; ++ ++ if (!job_is_internal(job) && s1 != s0) { ++ qapi_event_send_job_status_change(job->id, job->status, &error_abort); ++ } + } + + int job_apply_verb(Job *job, JobVerb verb, Error **errp) +diff --git a/qapi/job.json b/qapi/job.json +index a472c0c..9fbdd0c 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -92,3 +92,17 @@ + { 'enum': 'JobVerb', + 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', + 'finalize' ] } ++ ++## ++# @JOB_STATUS_CHANGE: ++# ++# Emitted when a job transitions to a different status. ++# ++# @id: The job identifier ++# @status: The new job status ++# ++# Since: 2.13 ++## ++{ 'event': 'JOB_STATUS_CHANGE', ++ 'data': { 'id': 'str', ++ 'status': 'JobStatus' } } +diff --git a/tests/qemu-iotests/030 b/tests/qemu-iotests/030 +index 640a6df..1dbc2dd 100755 +--- a/tests/qemu-iotests/030 ++++ b/tests/qemu-iotests/030 +@@ -304,8 +304,7 @@ class TestParallelOps(iotests.QMPTestCase): + result = self.vm.qmp('block-stream', device='node5', base=self.imgs[3], job_id='stream-node6') + self.assert_qmp(result, 'error/class', 'GenericError') + +- event = self.vm.get_qmp_event(wait=True) +- self.assertEqual(event['event'], 'BLOCK_JOB_READY') ++ event = self.vm.event_wait(name='BLOCK_JOB_READY') + self.assert_qmp(event, 'data/device', 'commit-drive0') + self.assert_qmp(event, 'data/type', 'commit') + self.assert_qmp_absent(event, 'data/error') +@@ -565,6 +564,8 @@ class TestEIO(TestErrors): + self.assert_qmp(event, 'data/offset', self.STREAM_BUFFER_SIZE) + self.assert_qmp(event, 'data/len', self.image_len) + completed = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + + self.assert_no_active_block_jobs() + self.vm.shutdown() +@@ -596,6 +597,8 @@ class TestEIO(TestErrors): + self.assert_qmp(event, 'data/offset', self.image_len) + self.assert_qmp(event, 'data/len', self.image_len) + completed = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + + self.assert_no_active_block_jobs() + self.vm.shutdown() +@@ -637,6 +640,8 @@ class TestEIO(TestErrors): + self.assert_qmp(event, 'data/offset', self.image_len) + self.assert_qmp(event, 'data/len', self.image_len) + completed = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + + self.assert_no_active_block_jobs() + self.vm.shutdown() +@@ -663,6 +668,8 @@ class TestEIO(TestErrors): + self.assert_qmp(event, 'data/offset', self.STREAM_BUFFER_SIZE) + self.assert_qmp(event, 'data/len', self.image_len) + completed = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + + self.assert_no_active_block_jobs() + self.vm.shutdown() +@@ -722,6 +729,8 @@ class TestENOSPC(TestErrors): + self.assert_qmp(event, 'data/offset', self.image_len) + self.assert_qmp(event, 'data/len', self.image_len) + completed = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + + self.assert_no_active_block_jobs() + self.vm.shutdown() +@@ -751,7 +760,9 @@ class TestStreamStop(iotests.QMPTestCase): + + time.sleep(0.1) + events = self.vm.get_qmp_events(wait=False) +- self.assertEqual(events, [], 'unexpected QMP event: %s' % events) ++ for e in events: ++ self.assert_qmp(e, 'event', 'JOB_STATUS_CHANGE') ++ self.assert_qmp(e, 'data/id', 'drive0') + + self.cancel_and_wait(resume=True) + +diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 +index 90b5b4f..1beb5e6 100755 +--- a/tests/qemu-iotests/040 ++++ b/tests/qemu-iotests/040 +@@ -162,6 +162,8 @@ class TestSingleDrive(ImageCommitTestCase): + elif event['event'] == 'BLOCK_JOB_CANCELLED': + self.assert_qmp(event, 'data/device', 'drive0') + cancelled = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + else: + self.fail("Unexpected event %s" % (event['event'])) + +diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 +index a860a31..e945879 100755 +--- a/tests/qemu-iotests/041 ++++ b/tests/qemu-iotests/041 +@@ -445,6 +445,8 @@ new_state = "1" + self.assert_qmp(event, 'data/device', 'drive0') + self.assert_qmp(event, 'data/error', 'Input/output error') + completed = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') + + self.assert_no_active_block_jobs() + self.vm.shutdown() +@@ -457,6 +459,10 @@ new_state = "1" + self.assert_qmp(result, 'return', {}) + + event = self.vm.get_qmp_event(wait=True) ++ while event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') ++ event = self.vm.get_qmp_event(wait=True) ++ + self.assertEquals(event['event'], 'BLOCK_JOB_ERROR') + self.assert_qmp(event, 'data/device', 'drive0') + self.assert_qmp(event, 'data/operation', 'read') +@@ -478,6 +484,10 @@ new_state = "1" + self.assert_qmp(result, 'return', {}) + + event = self.vm.get_qmp_event(wait=True) ++ while event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') ++ event = self.vm.get_qmp_event(wait=True) ++ + self.assertEquals(event['event'], 'BLOCK_JOB_ERROR') + self.assert_qmp(event, 'data/device', 'drive0') + self.assert_qmp(event, 'data/operation', 'read') +@@ -608,7 +618,7 @@ new_state = "1" + on_target_error='ignore') + self.assert_qmp(result, 'return', {}) + +- event = self.vm.get_qmp_event(wait=True) ++ event = self.vm.event_wait(name='BLOCK_JOB_ERROR') + self.assertEquals(event['event'], 'BLOCK_JOB_ERROR') + self.assert_qmp(event, 'data/device', 'drive0') + self.assert_qmp(event, 'data/operation', 'write') +@@ -784,7 +794,12 @@ class TestGranularity(iotests.QMPTestCase): + sync='full', target=target_img, + mode='absolute-paths', granularity=8192) + self.assert_qmp(result, 'return', {}) ++ + event = self.vm.get_qmp_event(wait=60.0) ++ while event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', 'drive0') ++ event = self.vm.get_qmp_event(wait=60.0) ++ + # Failures will manifest as COMPLETED/ERROR. + self.assert_qmp(event, 'event', 'BLOCK_JOB_READY') + self.complete_and_wait(drive='drive0', wait_ready=False) +diff --git a/tests/qemu-iotests/094.out b/tests/qemu-iotests/094.out +index f52baff..665b630 100644 +--- a/tests/qemu-iotests/094.out ++++ b/tests/qemu-iotests/094.out +@@ -2,10 +2,17 @@ QA output created by 094 + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/source.IMGFMT', fmt=IMGFMT size=67108864 + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 67108864, "offset": 67108864, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + *** done +diff --git a/tests/qemu-iotests/095 b/tests/qemu-iotests/095 +index 030adb2..72ecc22 100755 +--- a/tests/qemu-iotests/095 ++++ b/tests/qemu-iotests/095 +@@ -72,7 +72,7 @@ _send_qemu_cmd $h "{ 'execute': 'qmp_capabilities' }" "return" + + _send_qemu_cmd $h "{ 'execute': 'block-commit', + 'arguments': { 'device': 'test', +- 'top': '"${TEST_IMG}.snp1"' } }" "BLOCK_JOB_COMPLETED" ++ 'top': '"${TEST_IMG}.snp1"' } }" '"status": "null"' + + _cleanup_qemu + +diff --git a/tests/qemu-iotests/095.out b/tests/qemu-iotests/095.out +index 73875ca..8c093df 100644 +--- a/tests/qemu-iotests/095.out ++++ b/tests/qemu-iotests/095.out +@@ -11,8 +11,14 @@ virtual size: 5.0M (5242880 bytes) + === Running QEMU Live Commit Test === + + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "test"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "test"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "test"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "test"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "test", "len": 104857600, "offset": 104857600, "speed": 0, "type": "commit"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "test"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "test"}} + + === Base image info after commit and resize === + image: TEST_DIR/t.IMGFMT.base +diff --git a/tests/qemu-iotests/109 b/tests/qemu-iotests/109 +index d70b574..acbd079 100755 +--- a/tests/qemu-iotests/109 ++++ b/tests/qemu-iotests/109 +@@ -64,7 +64,7 @@ function run_qemu() + + _send_qemu_cmd $QEMU_HANDLE '' "$qmp_event" + if test "$qmp_event" = BLOCK_JOB_ERROR; then +- _send_qemu_cmd $QEMU_HANDLE '' "BLOCK_JOB_COMPLETED" ++ _send_qemu_cmd $QEMU_HANDLE '' '"status": "null"' + fi + _send_qemu_cmd $QEMU_HANDLE '{"execute":"query-block-jobs"}' "return" + _send_qemu_cmd $QEMU_HANDLE '{"execute":"quit"}' "return" +diff --git a/tests/qemu-iotests/109.out b/tests/qemu-iotests/109.out +index 8a9b936..ad0ee6f 100644 +--- a/tests/qemu-iotests/109.out ++++ b/tests/qemu-iotests/109.out +@@ -6,23 +6,35 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": 0, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -32,23 +44,35 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": 512, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 197120, "offset": 197120, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 197120, "offset": 197120, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -58,23 +82,35 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": 262144, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -84,23 +120,35 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": 0, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 1024, "offset": 1024, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 1024, "offset": 1024, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -110,23 +158,35 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": 0, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 65536, "offset": 65536, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -136,23 +196,35 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + Formatting 'TEST_DIR/t.raw.src', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": 0, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -161,23 +233,35 @@ Images are identical. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": OFFSET, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2560, "offset": 2560, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2560, "offset": 2560, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -186,23 +270,35 @@ Images are identical. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": OFFSET, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 31457280, "offset": 31457280, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 31457280, "offset": 31457280, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -211,23 +307,35 @@ Images are identical. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": OFFSET, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 327680, "offset": 327680, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 327680, "offset": 327680, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -236,23 +344,35 @@ Images are identical. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_ERROR", "data": {"device": "src", "operation": "write", "action": "report"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": LEN, "offset": OFFSET, "speed": 0, "type": "mirror", "error": "Operation not permitted"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + {"return": []} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 2048, "offset": 2048, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 2048, "offset": 2048, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + +@@ -261,23 +381,37 @@ Images are identical. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + {"return": {}} + WARNING: Image format was not specified for 'TEST_DIR/t.raw' and probing guessed raw. +-Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. +-Specify the 'raw' format explicitly to remove the restrictions. ++ Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted. ++ Specify the 'raw' format explicitly to remove the restrictions. ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "src"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} + {"return": [{"auto-finalize": true, "io-status": "ok", "device": "src", "auto-dismiss": true, "busy": false, "len": 512, "offset": 512, "status": "ready", "paused": false, "speed": 0, "ready": true, "type": "mirror"}]} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "src"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 512, "offset": 512, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "src"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "src"}} + Warning: Image size mismatch! + Images are identical. + *** done +diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 +index 8e76e62..3ea4ac5 100755 +--- a/tests/qemu-iotests/124 ++++ b/tests/qemu-iotests/124 +@@ -151,10 +151,17 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): + return self.wait_qmp_backup(kwargs['device'], error) + + ++ def ignore_job_status_change_events(self): ++ while True: ++ e = self.vm.event_wait(name="JOB_STATUS_CHANGE") ++ if e['data']['status'] == 'null': ++ break ++ + def wait_qmp_backup(self, device, error='Input/output error'): + event = self.vm.event_wait(name="BLOCK_JOB_COMPLETED", + match={'data': {'device': device}}) + self.assertNotEqual(event, None) ++ self.ignore_job_status_change_events() + + try: + failure = self.dictpath(event, 'data/error') +@@ -172,6 +179,7 @@ class TestIncrementalBackupBase(iotests.QMPTestCase): + event = self.vm.event_wait(name='BLOCK_JOB_CANCELLED', + match={'data': {'device': device}}) + self.assertNotEqual(event, None) ++ self.ignore_job_status_change_events() + + + def create_anchor_backup(self, drive=None): +diff --git a/tests/qemu-iotests/127.out b/tests/qemu-iotests/127.out +index 543d075..83b522d 100644 +--- a/tests/qemu-iotests/127.out ++++ b/tests/qemu-iotests/127.out +@@ -5,10 +5,17 @@ Formatting 'TEST_DIR/t.IMGFMT.overlay1', fmt=IMGFMT size=65536 backing_file=TEST + wrote 42/42 bytes at offset 0 + 42 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "mirror"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "mirror"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "mirror", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "mirror"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "mirror", "len": 65536, "offset": 65536, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "mirror"}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} + *** done +diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141 +index 2f9d7b9..4246d38 100755 +--- a/tests/qemu-iotests/141 ++++ b/tests/qemu-iotests/141 +@@ -107,7 +107,7 @@ test_blockjob \ + 'format': '$IMGFMT', + 'sync': 'none'}}" \ + 'return' \ +- 'BLOCK_JOB_CANCELLED' ++ '"status": "null"' + + echo + echo '=== Testing drive-mirror ===' +@@ -124,7 +124,7 @@ test_blockjob \ + 'format': '$IMGFMT', + 'sync': 'none'}}" \ + 'BLOCK_JOB_READY' \ +- 'BLOCK_JOB_COMPLETED' ++ '"status": "null"' + + echo + echo '=== Testing active block-commit ===' +@@ -138,7 +138,7 @@ test_blockjob \ + "{'execute': 'block-commit', + 'arguments': {'job-id': 'job0', 'device': 'drv0'}}" \ + 'BLOCK_JOB_READY' \ +- 'BLOCK_JOB_COMPLETED' ++ '"status": "null"' + + echo + echo '=== Testing non-active block-commit ===' +@@ -157,7 +157,7 @@ test_blockjob \ + 'top': '$TEST_DIR/m.$IMGFMT', + 'speed': 1}}" \ + 'return' \ +- 'BLOCK_JOB_CANCELLED' ++ '"status": "null"' + + echo + echo '=== Testing block-stream ===' +@@ -170,8 +170,7 @@ echo + $QEMU_IO -c 'write 0 1M' "$TEST_DIR/b.$IMGFMT" | _filter_qemu_io + + # With some data to stream (and @speed set to 1), block-stream will not complete +-# until we send the block-job-cancel command. Therefore, no event other than +-# BLOCK_JOB_CANCELLED will be emitted. ++# until we send the block-job-cancel command. + + test_blockjob \ + "{'execute': 'block-stream', +@@ -179,7 +178,7 @@ test_blockjob \ + 'device': 'drv0', + 'speed': 1}}" \ + 'return' \ +- 'BLOCK_JOB_CANCELLED' ++ '"status": "null"' + + _cleanup_qemu + +diff --git a/tests/qemu-iotests/141.out b/tests/qemu-iotests/141.out +index 82e763b..f252c86 100644 +--- a/tests/qemu-iotests/141.out ++++ b/tests/qemu-iotests/141.out +@@ -8,31 +8,50 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/m. + + {"return": {}} + Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} + {"return": {}} + {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 0, "speed": 0, "type": "backup"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} + {"return": {}} + + === Testing drive-mirror === + + {"return": {}} + Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT backing_fmt=IMGFMT ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}} + {"return": {}} + {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is used as backing hd of 'NODE_NAME'"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} + {"return": {}} + + === Testing active block-commit === + + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} + {"return": {}} + {"error": {"class": "GenericError", "desc": "Node 'drv0' is busy: node is used as backing hd of 'NODE_NAME'"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} + {"return": {}} + + === Testing non-active block-commit === +@@ -40,10 +59,15 @@ Formatting 'TEST_DIR/o.IMGFMT', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t. + wrote 1048576/1048576 bytes at offset 0 + 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} + {"return": {}} + {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 524288, "speed": 1, "type": "commit"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} + {"return": {}} + + === Testing block-stream === +@@ -51,9 +75,14 @@ wrote 1048576/1048576 bytes at offset 0 + wrote 1048576/1048576 bytes at offset 0 + 1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "job0"}} + {"return": {}} + {"error": {"class": "GenericError", "desc": "Node drv0 is in use"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "aborting", "id": "job0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_CANCELLED", "data": {"device": "job0", "len": 1048576, "offset": 524288, "speed": 1, "type": "stream"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "job0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "job0"}} + {"return": {}} + *** done +diff --git a/tests/qemu-iotests/144 b/tests/qemu-iotests/144 +index 00de3c33..4b91571 100755 +--- a/tests/qemu-iotests/144 ++++ b/tests/qemu-iotests/144 +@@ -93,7 +93,7 @@ _send_qemu_cmd $h "{ 'execute': 'block-job-complete', + 'arguments': { + 'device': 'virtio0' + } +- }" "COMPLETED" ++ }" '"status": "null"' + + echo + echo === Performing Live Snapshot 2 === +diff --git a/tests/qemu-iotests/144.out b/tests/qemu-iotests/144.out +index 014b281..5529920 100644 +--- a/tests/qemu-iotests/144.out ++++ b/tests/qemu-iotests/144.out +@@ -12,10 +12,17 @@ Formatting 'TEST_DIR/tmp.qcow2', fmt=qcow2 size=536870912 backing_file=TEST_DIR/ + + === Performing block-commit on active layer === + ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "virtio0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "virtio0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "virtio0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "virtio0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} + {"return": {}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "virtio0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "virtio0"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "virtio0", "len": 0, "offset": 0, "speed": 0, "type": "commit"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "virtio0"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "virtio0"}} + + === Performing Live Snapshot 2 === + +diff --git a/tests/qemu-iotests/156 b/tests/qemu-iotests/156 +index e75dc4d..0a9a098 100755 +--- a/tests/qemu-iotests/156 ++++ b/tests/qemu-iotests/156 +@@ -119,7 +119,7 @@ _send_qemu_cmd $QEMU_HANDLE \ + + _send_qemu_cmd $QEMU_HANDLE \ + '' \ +- 'BLOCK_JOB_COMPLETED' ++ '"status": "null"' + + # Remove the source images + rm -f "$TEST_IMG{,.backing,.overlay}" +diff --git a/tests/qemu-iotests/156.out b/tests/qemu-iotests/156.out +index f96a564..34c057b 100644 +--- a/tests/qemu-iotests/156.out ++++ b/tests/qemu-iotests/156.out +@@ -12,13 +12,20 @@ wrote 131072/131072 bytes at offset 131072 + 128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": ""} + Formatting 'TEST_DIR/t.IMGFMT.target.overlay', fmt=IMGFMT size=1048576 backing_file=TEST_DIR/t.IMGFMT.target ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "source"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "source"}} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "source"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_READY", "data": {"device": "source", "len": 131072, "offset": 131072, "speed": 0, "type": "mirror"}} + wrote 65536/65536 bytes at offset 196608 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + {"return": ""} + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "waiting", "id": "source"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "pending", "id": "source"}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "source", "len": 196608, "offset": 196608, "speed": 0, "type": "mirror"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "concluded", "id": "source"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "null", "id": "source"}} + + read 65536/65536 bytes at offset 0 + 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 +index 9a2d317..567ba67 100755 +--- a/tests/qemu-iotests/185 ++++ b/tests/qemu-iotests/185 +@@ -118,8 +118,10 @@ _send_qemu_cmd $h \ + # If we don't sleep here 'quit' command races with disk I/O + sleep 0.5 + ++# Ignore the JOB_STATUS_CHANGE events while shutting down the VM. Depending on ++# the timing, jobs may or may not transition through a paused state. + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" +-wait=1 _cleanup_qemu ++wait=1 _cleanup_qemu | grep -v 'JOB_STATUS_CHANGE' + + echo + echo === Start active commit job and exit qemu === +@@ -141,7 +143,7 @@ _send_qemu_cmd $h \ + sleep 0.5 + + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" +-wait=1 _cleanup_qemu ++wait=1 _cleanup_qemu | grep -v 'JOB_STATUS_CHANGE' + + echo + echo === Start mirror job and exit qemu === +@@ -166,7 +168,7 @@ _send_qemu_cmd $h \ + sleep 0.5 + + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" +-wait=1 _cleanup_qemu ++wait=1 _cleanup_qemu | grep -v 'JOB_STATUS_CHANGE' + + echo + echo === Start backup job and exit qemu === +@@ -190,7 +192,7 @@ _send_qemu_cmd $h \ + sleep 0.5 + + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" +-wait=1 _cleanup_qemu ++wait=1 _cleanup_qemu | grep -v 'JOB_STATUS_CHANGE' + + echo + echo === Start streaming job and exit qemu === +@@ -211,7 +213,7 @@ _send_qemu_cmd $h \ + sleep 0.5 + + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" +-wait=1 _cleanup_qemu ++wait=1 _cleanup_qemu | grep -v 'JOB_STATUS_CHANGE' + + _check_test_img + +diff --git a/tests/qemu-iotests/185.out b/tests/qemu-iotests/185.out +index 57eaf8d..4e0ca0d 100644 +--- a/tests/qemu-iotests/185.out ++++ b/tests/qemu-iotests/185.out +@@ -17,6 +17,8 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q + + === Start commit job and exit qemu === + ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +@@ -25,6 +27,8 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q + === Start active commit job and exit qemu === + + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +@@ -34,6 +38,8 @@ Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=67108864 backing_file=TEST_DIR/t.q + + {"return": {}} + Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +@@ -43,6 +49,8 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l + + {"return": {}} + Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 lazy_refcounts=off refcount_bits=16 ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +@@ -51,6 +59,8 @@ Formatting 'TEST_DIR/t.qcow2.copy', fmt=qcow2 size=67108864 cluster_size=65536 l + === Start streaming job and exit qemu === + + {"return": {}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "created", "id": "disk"}} ++{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "JOB_STATUS_CHANGE", "data": {"status": "running", "id": "disk"}} + {"return": {}} + {"return": {}} + {"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +diff --git a/tests/qemu-iotests/191 b/tests/qemu-iotests/191 +index dfad655..b3629ff 100755 +--- a/tests/qemu-iotests/191 ++++ b/tests/qemu-iotests/191 +@@ -83,7 +83,7 @@ _send_qemu_cmd $h \ + 'device': 'top', + 'base':'$TEST_IMG.base', + 'top': '$TEST_IMG.mid' } }" \ +- "BLOCK_JOB_COMPLETED" ++ '"status": "null"' + _send_qemu_cmd $h "" "^}" + + echo +@@ -131,7 +131,7 @@ _send_qemu_cmd $h \ + 'device': 'top', + 'base':'$TEST_IMG.base', + 'top': '$TEST_IMG.mid' } }" \ +- "BLOCK_JOB_COMPLETED" ++ '"status": "null"' + _send_qemu_cmd $h "" "^}" + + echo +diff --git a/tests/qemu-iotests/191.out b/tests/qemu-iotests/191.out +index 190c5f0..31a0c7d 100644 +--- a/tests/qemu-iotests/191.out ++++ b/tests/qemu-iotests/191.out +@@ -16,6 +16,28 @@ wrote 65536/65536 bytes at offset 1048576 + === Perform commit job === + + { ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "created", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "running", ++ "id": "commit0" ++ } ++} ++{ + "return": { + } + } +@@ -24,6 +46,28 @@ wrote 65536/65536 bytes at offset 1048576 + "seconds": TIMESTAMP, + "microseconds": TIMESTAMP + }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "waiting", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "pending", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, + "event": "BLOCK_JOB_COMPLETED", + "data": { + "device": "commit0", +@@ -33,6 +77,28 @@ wrote 65536/65536 bytes at offset 1048576 + "type": "commit" + } + } ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "concluded", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "null", ++ "id": "commit0" ++ } ++} + + === Check that both top and top2 point to base now === + +@@ -356,6 +422,28 @@ wrote 65536/65536 bytes at offset 1048576 + === Perform commit job === + + { ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "created", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "running", ++ "id": "commit0" ++ } ++} ++{ + "return": { + } + } +@@ -364,6 +452,28 @@ wrote 65536/65536 bytes at offset 1048576 + "seconds": TIMESTAMP, + "microseconds": TIMESTAMP + }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "waiting", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "pending", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, + "event": "BLOCK_JOB_COMPLETED", + "data": { + "device": "commit0", +@@ -373,6 +483,28 @@ wrote 65536/65536 bytes at offset 1048576 + "type": "commit" + } + } ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "concluded", ++ "id": "commit0" ++ } ++} ++{ ++ "timestamp": { ++ "seconds": TIMESTAMP, ++ "microseconds": TIMESTAMP ++ }, ++ "event": "JOB_STATUS_CHANGE", ++ "data": { ++ "status": "null", ++ "id": "commit0" ++ } ++} + + === Check that both top and top2 point to base now === + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 2ce85a1..824f87d 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -474,6 +474,9 @@ class QMPTestCase(unittest.TestCase): + self.assert_qmp(event, 'data/device', drive) + result = event + cancelled = True ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', drive) ++ + + self.assert_no_active_block_jobs() + return result +@@ -489,6 +492,8 @@ class QMPTestCase(unittest.TestCase): + self.assert_qmp(event, 'data/offset', event['data']['len']) + self.assert_no_active_block_jobs() + return event ++ elif event['event'] == 'JOB_STATUS_CHANGE': ++ self.assert_qmp(event, 'data/id', drive) + + def wait_ready(self, drive='drive0'): + '''Wait until a block job BLOCK_JOB_READY event''' +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-Job.aio_context.patch b/SOURCES/kvm-job-Add-Job.aio_context.patch new file mode 100644 index 0000000..72278c9 --- /dev/null +++ b/SOURCES/kvm-job-Add-Job.aio_context.patch @@ -0,0 +1,112 @@ +From 7d49650cc825e639180ba1d7580e15cf56fb21da Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:07 +0200 +Subject: [PATCH 099/268] job: Add Job.aio_context + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-25-kwolf@redhat.com> +Patchwork-id: 81075 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 24/73] job: Add Job.aio_context +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +When block jobs need an AioContext, they just take it from their main +block node. Generic jobs don't have a main block node, so we need to +assign them an AioContext explicitly. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 08be6fe26f6c76d900fc987f58d322b94bc4e248) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 5 ++++- + include/qemu/job.h | 7 ++++++- + job.c | 4 +++- + 3 files changed, 13 insertions(+), 3 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index f4f9956..0a0b1c4 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -216,6 +216,7 @@ static void block_job_attached_aio_context(AioContext *new_context, + { + BlockJob *job = opaque; + ++ job->job.aio_context = new_context; + if (job->driver->attached_aio_context) { + job->driver->attached_aio_context(job, new_context); + } +@@ -247,6 +248,7 @@ static void block_job_detach_aio_context(void *opaque) + block_job_drain(job); + } + ++ job->job.aio_context = NULL; + job_unref(&job->job); + } + +@@ -899,7 +901,8 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return NULL; + } + +- job = job_create(job_id, &driver->job_driver, errp); ++ job = job_create(job_id, &driver->job_driver, blk_get_aio_context(blk), ++ errp); + if (job == NULL) { + blk_unref(blk); + return NULL; +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 5dfbec5..01e083f 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -47,6 +47,9 @@ typedef struct Job { + /** Current state; See @JobStatus for details. */ + JobStatus status; + ++ /** AioContext to run the job coroutine in */ ++ AioContext *aio_context; ++ + /** + * Set to true if the job should cancel itself. The flag must + * always be tested just before toggling the busy flag from false +@@ -79,9 +82,11 @@ struct JobDriver { + * + * @job_id: The id of the newly-created job, or %NULL for internal jobs + * @driver: The class object for the newly-created job. ++ * @ctx: The AioContext to run the job coroutine in. + * @errp: Error object. + */ +-void *job_create(const char *job_id, const JobDriver *driver, Error **errp); ++void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, ++ Error **errp); + + /** + * Add a reference to Job refcnt, it will be decreased with job_unref, and then +diff --git a/job.c b/job.c +index 1abca6a..01074d0 100644 +--- a/job.c ++++ b/job.c +@@ -121,7 +121,8 @@ Job *job_get(const char *id) + return NULL; + } + +-void *job_create(const char *job_id, const JobDriver *driver, Error **errp) ++void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, ++ Error **errp) + { + Job *job; + +@@ -140,6 +141,7 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + job->driver = driver; + job->id = g_strdup(job_id); + job->refcnt = 1; ++ job->aio_context = ctx; + + job_state_transition(job, JOB_STATUS_CREATED); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-JobDriver.job_type.patch b/SOURCES/kvm-job-Add-JobDriver.job_type.patch new file mode 100644 index 0000000..170f9b1 --- /dev/null +++ b/SOURCES/kvm-job-Add-JobDriver.job_type.patch @@ -0,0 +1,240 @@ +From 85085499e2916b505c836f266fc6f4120b52fad1 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:01 +0200 +Subject: [PATCH 093/268] job: Add JobDriver.job_type + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-19-kwolf@redhat.com> +Patchwork-id: 81063 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 18/73] job: Add JobDriver.job_type +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves the job_type field from BlockJobDriver to JobDriver. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 252291eaeafcd234a602d71cdf9415dbfc7bc867) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + block/commit.c | 2 +- + block/mirror.c | 4 ++-- + block/stream.c | 2 +- + blockjob.c | 16 +++++++--------- + include/block/blockjob_int.h | 3 --- + include/qemu/job.h | 11 +++++++++++ + job.c | 10 ++++++++++ + 8 files changed, 33 insertions(+), 17 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index c49ea92..baf8d43 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -525,8 +525,8 @@ static void coroutine_fn backup_run(void *opaque) + static const BlockJobDriver backup_job_driver = { + .job_driver = { + .instance_size = sizeof(BackupBlockJob), ++ .job_type = JOB_TYPE_BACKUP, + }, +- .job_type = JOB_TYPE_BACKUP, + .start = backup_run, + .commit = backup_commit, + .abort = backup_abort, +diff --git a/block/commit.c b/block/commit.c +index afa2b2b..32d29c8 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -217,8 +217,8 @@ out: + static const BlockJobDriver commit_job_driver = { + .job_driver = { + .instance_size = sizeof(CommitBlockJob), ++ .job_type = JOB_TYPE_COMMIT, + }, +- .job_type = JOB_TYPE_COMMIT, + .start = commit_run, + }; + +diff --git a/block/mirror.c b/block/mirror.c +index ed72656..35fcc1f 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -988,8 +988,8 @@ static void mirror_drain(BlockJob *job) + static const BlockJobDriver mirror_job_driver = { + .job_driver = { + .instance_size = sizeof(MirrorBlockJob), ++ .job_type = JOB_TYPE_MIRROR, + }, +- .job_type = JOB_TYPE_MIRROR, + .start = mirror_run, + .complete = mirror_complete, + .pause = mirror_pause, +@@ -1000,8 +1000,8 @@ static const BlockJobDriver mirror_job_driver = { + static const BlockJobDriver commit_active_job_driver = { + .job_driver = { + .instance_size = sizeof(MirrorBlockJob), ++ .job_type = JOB_TYPE_COMMIT, + }, +- .job_type = JOB_TYPE_COMMIT, + .start = mirror_run, + .complete = mirror_complete, + .pause = mirror_pause, +diff --git a/block/stream.c b/block/stream.c +index 048bceb..cb723f1 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -211,8 +211,8 @@ out: + static const BlockJobDriver stream_job_driver = { + .job_driver = { + .instance_size = sizeof(StreamBlockJob), ++ .job_type = JOB_TYPE_STREAM, + }, +- .job_type = JOB_TYPE_STREAM, + .start = stream_run, + }; + +diff --git a/blockjob.c b/blockjob.c +index 2a38447..ea71ec0 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -309,9 +309,7 @@ static void block_job_detach_aio_context(void *opaque) + static char *child_job_get_parent_desc(BdrvChild *c) + { + BlockJob *job = c->opaque; +- return g_strdup_printf("%s job '%s'", +- JobType_str(job->driver->job_type), +- job->job.id); ++ return g_strdup_printf("%s job '%s'", job_type_str(&job->job), job->job.id); + } + + static void child_job_drained_begin(BdrvChild *c) +@@ -847,7 +845,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + return NULL; + } + info = g_new0(BlockJobInfo, 1); +- info->type = g_strdup(JobType_str(job->driver->job_type)); ++ info->type = g_strdup(job_type_str(&job->job)); + info->device = g_strdup(job->job.id); + info->len = job->len; + info->busy = atomic_read(&job->busy); +@@ -878,7 +876,7 @@ static void block_job_event_cancelled(BlockJob *job) + return; + } + +- qapi_event_send_block_job_cancelled(job->driver->job_type, ++ qapi_event_send_block_job_cancelled(job_type(&job->job), + job->job.id, + job->len, + job->offset, +@@ -892,7 +890,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg) + return; + } + +- qapi_event_send_block_job_completed(job->driver->job_type, ++ qapi_event_send_block_job_completed(job_type(&job->job), + job->job.id, + job->len, + job->offset, +@@ -906,7 +904,7 @@ static int block_job_event_pending(BlockJob *job) + { + block_job_state_transition(job, BLOCK_JOB_STATUS_PENDING); + if (!job->auto_finalize && !block_job_is_internal(job)) { +- qapi_event_send_block_job_pending(job->driver->job_type, ++ qapi_event_send_block_job_pending(job_type(&job->job), + job->job.id, + &error_abort); + } +@@ -980,7 +978,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + block_job_sleep_timer_cb, job); + + error_setg(&job->blocker, "block device is in use by block job: %s", +- JobType_str(driver->job_type)); ++ job_type_str(&job->job)); + block_job_add_bdrv(job, "main node", bs, 0, BLK_PERM_ALL, &error_abort); + bs->job = job; + +@@ -1184,7 +1182,7 @@ void block_job_event_ready(BlockJob *job) + return; + } + +- qapi_event_send_block_job_ready(job->driver->job_type, ++ qapi_event_send_block_job_ready(job_type(&job->job), + job->job.id, + job->len, + job->offset, +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 8e7e0a2..1e62d6d 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -38,9 +38,6 @@ struct BlockJobDriver { + /** Generic JobDriver callbacks and settings */ + JobDriver job_driver; + +- /** String describing the operation, part of query-block-jobs QMP API */ +- JobType job_type; +- + /** Mandatory: Entrypoint for the Coroutine. */ + CoroutineEntry *start; + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index b4b49f1..279ce68 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -26,6 +26,8 @@ + #ifndef JOB_H + #define JOB_H + ++#include "qapi/qapi-types-block-core.h" ++ + typedef struct JobDriver JobDriver; + + /** +@@ -45,6 +47,9 @@ typedef struct Job { + struct JobDriver { + /** Derived Job struct size */ + size_t instance_size; ++ ++ /** Enum describing the operation */ ++ JobType job_type; + }; + + +@@ -57,4 +62,10 @@ struct JobDriver { + */ + void *job_create(const char *job_id, const JobDriver *driver, Error **errp); + ++/** Returns the JobType of a given Job. */ ++JobType job_type(const Job *job); ++ ++/** Returns the enum string for the JobType of a given Job. */ ++const char *job_type_str(const Job *job); ++ + #endif +diff --git a/job.c b/job.c +index 87fd484..83724a4 100644 +--- a/job.c ++++ b/job.c +@@ -29,6 +29,16 @@ + #include "qemu/job.h" + #include "qemu/id.h" + ++JobType job_type(const Job *job) ++{ ++ return job->driver->job_type; ++} ++ ++const char *job_type_str(const Job *job) ++{ ++ return JobType_str(job_type(job)); ++} ++ + void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + { + Job *job; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-error-message-for-failing-jobs.patch b/SOURCES/kvm-job-Add-error-message-for-failing-jobs.patch new file mode 100644 index 0000000..295a3ba --- /dev/null +++ b/SOURCES/kvm-job-Add-error-message-for-failing-jobs.patch @@ -0,0 +1,257 @@ +From a04691e941048c14853d40cbb2a174e4e9b473a5 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:39 +0200 +Subject: [PATCH 131/268] job: Add error message for failing jobs + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-57-kwolf@redhat.com> +Patchwork-id: 81110 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 56/73] job: Add error message for failing jobs +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +So far we relied on job->ret and strerror() to produce an error message +for failed jobs. Not surprisingly, this tends to result in completely +useless messages. + +This adds a Job.error field that can contain an error string for a +failing job, and a parameter to job_completed() that sets the field. As +a default, if NULL is passed, we continue to use strerror(job->ret). + +All existing callers are changed to pass NULL. They can be improved in +separate patches. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 1266c9b9f5fa05877b979eece5963a2bd99c3bfd) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + block/commit.c | 2 +- + block/mirror.c | 2 +- + block/stream.c | 2 +- + include/qemu/job.h | 7 ++++++- + job-qmp.c | 9 ++------- + job.c | 16 ++++++++++++++-- + tests/test-bdrv-drain.c | 2 +- + tests/test-blockjob-txn.c | 2 +- + tests/test-blockjob.c | 2 +- + 10 files changed, 29 insertions(+), 17 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 4e228e9..5661435 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -321,7 +321,7 @@ static void backup_complete(Job *job, void *opaque) + { + BackupCompleteData *data = opaque; + +- job_completed(job, data->ret); ++ job_completed(job, data->ret, NULL); + g_free(data); + } + +diff --git a/block/commit.c b/block/commit.c +index 6206661..e1814d9 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -117,7 +117,7 @@ static void commit_complete(Job *job, void *opaque) + * bdrv_set_backing_hd() to fail. */ + block_job_remove_all_bdrv(bjob); + +- job_completed(job, ret); ++ job_completed(job, ret, NULL); + g_free(data); + + /* If bdrv_drop_intermediate() didn't already do that, remove the commit +diff --git a/block/mirror.c b/block/mirror.c +index dcb66ec..435268b 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -581,7 +581,7 @@ static void mirror_exit(Job *job, void *opaque) + blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); + blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort); + +- job_completed(job, data->ret); ++ job_completed(job, data->ret, NULL); + + g_free(data); + bdrv_drained_end(src); +diff --git a/block/stream.c b/block/stream.c +index a5d6e0c..9264b68 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -93,7 +93,7 @@ out: + } + + g_free(s->backing_file_str); +- job_completed(job, data->ret); ++ job_completed(job, data->ret, NULL); + g_free(data); + } + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 8c8badf..1d82053 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -124,6 +124,9 @@ typedef struct Job { + /** Estimated progress_current value at the completion of the job */ + int64_t progress_total; + ++ /** Error string for a failed job (NULL if, and only if, job->ret == 0) */ ++ char *error; ++ + /** ret code passed to job_completed. */ + int ret; + +@@ -466,13 +469,15 @@ void job_transition_to_ready(Job *job); + /** + * @job: The job being completed. + * @ret: The status code. ++ * @error: The error message for a failing job (only with @ret < 0). If @ret is ++ * negative, but NULL is given for @error, strerror() is used. + * + * Marks @job as completed. If @ret is non-zero, the job transaction it is part + * of is aborted. If @ret is zero, the job moves into the WAITING state. If it + * is the last job to complete in its transaction, all jobs in the transaction + * move from WAITING to PENDING. + */ +-void job_completed(Job *job, int ret); ++void job_completed(Job *job, int ret, Error *error); + + /** Asynchronously complete the specified @job. */ + void job_complete(Job *job, Error **errp); +diff --git a/job-qmp.c b/job-qmp.c +index 7f38f63..410775d 100644 +--- a/job-qmp.c ++++ b/job-qmp.c +@@ -136,14 +136,9 @@ void qmp_job_dismiss(const char *id, Error **errp) + static JobInfo *job_query_single(Job *job, Error **errp) + { + JobInfo *info; +- const char *errmsg = NULL; + + assert(!job_is_internal(job)); + +- if (job->ret < 0) { +- errmsg = strerror(-job->ret); +- } +- + info = g_new(JobInfo, 1); + *info = (JobInfo) { + .id = g_strdup(job->id), +@@ -151,8 +146,8 @@ static JobInfo *job_query_single(Job *job, Error **errp) + .status = job->status, + .current_progress = job->progress_current, + .total_progress = job->progress_total, +- .has_error = !!errmsg, +- .error = g_strdup(errmsg), ++ .has_error = !!job->error, ++ .error = g_strdup(job->error), + }; + + return info; +diff --git a/job.c b/job.c +index f026661..84e1402 100644 +--- a/job.c ++++ b/job.c +@@ -369,6 +369,7 @@ void job_unref(Job *job) + + QLIST_REMOVE(job, job_list); + ++ g_free(job->error); + g_free(job->id); + g_free(job); + } +@@ -660,6 +661,9 @@ static void job_update_rc(Job *job) + job->ret = -ECANCELED; + } + if (job->ret) { ++ if (!job->error) { ++ job->error = g_strdup(strerror(-job->ret)); ++ } + job_state_transition(job, JOB_STATUS_ABORTING); + } + } +@@ -782,6 +786,7 @@ static int job_prepare(Job *job) + { + if (job->ret == 0 && job->driver->prepare) { + job->ret = job->driver->prepare(job); ++ job_update_rc(job); + } + return job->ret; + } +@@ -855,10 +860,17 @@ static void job_completed_txn_success(Job *job) + } + } + +-void job_completed(Job *job, int ret) ++void job_completed(Job *job, int ret, Error *error) + { + assert(job && job->txn && !job_is_completed(job)); ++ + job->ret = ret; ++ if (error) { ++ assert(job->ret < 0); ++ job->error = g_strdup(error_get_pretty(error)); ++ error_free(error); ++ } ++ + job_update_rc(job); + trace_job_completed(job, ret, job->ret); + if (job->ret) { +@@ -876,7 +888,7 @@ void job_cancel(Job *job, bool force) + } + job_cancel_async(job, force); + if (!job_started(job)) { +- job_completed(job, -ECANCELED); ++ job_completed(job, -ECANCELED, NULL); + } else if (job->deferred_to_main_loop) { + job_completed_txn_abort(job); + } else { +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 2cba63b..a11c4cf 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -498,7 +498,7 @@ typedef struct TestBlockJob { + + static void test_job_completed(Job *job, void *opaque) + { +- job_completed(job, 0); ++ job_completed(job, 0, NULL); + } + + static void coroutine_fn test_job_start(void *opaque) +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index fce8366..58d9b87 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -34,7 +34,7 @@ static void test_block_job_complete(Job *job, void *opaque) + rc = -ECANCELED; + } + +- job_completed(job, rc); ++ job_completed(job, rc, NULL); + bdrv_unref(bs); + } + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index e408d52..cb42f06 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -167,7 +167,7 @@ static void cancel_job_completed(Job *job, void *opaque) + { + CancelJob *s = opaque; + s->completed = true; +- job_completed(job, 0); ++ job_completed(job, 0, NULL); + } + + static void cancel_job_complete(Job *job, Error **errp) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_delete.patch b/SOURCES/kvm-job-Add-job_delete.patch new file mode 100644 index 0000000..ac747b9 --- /dev/null +++ b/SOURCES/kvm-job-Add-job_delete.patch @@ -0,0 +1,75 @@ +From 5845c03f7bd9ad3759eb63977eecb7412f6250aa Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:02 +0200 +Subject: [PATCH 094/268] job: Add job_delete() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-20-kwolf@redhat.com> +Patchwork-id: 81065 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 19/73] job: Add job_delete() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves freeing the Job object and its fields from block_job_unref() +to job_delete(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit fd61a701f1de8e4c1d89b3716ba9ca749cf5c724) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 3 +-- + include/qemu/job.h | 3 +++ + job.c | 6 ++++++ + 3 files changed, 10 insertions(+), 2 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index ea71ec0..430a67b 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -261,9 +261,8 @@ void block_job_unref(BlockJob *job) + block_job_detach_aio_context, job); + blk_unref(job->blk); + error_free(job->blocker); +- g_free(job->job.id); + assert(!timer_pending(&job->sleep_timer)); +- g_free(job); ++ job_delete(&job->job); + } + } + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 279ce68..43dc2e4 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -62,6 +62,9 @@ struct JobDriver { + */ + void *job_create(const char *job_id, const JobDriver *driver, Error **errp); + ++/** Frees the @job object. */ ++void job_delete(Job *job); ++ + /** Returns the JobType of a given Job. */ + JobType job_type(const Job *job); + +diff --git a/job.c b/job.c +index 83724a4..cfdd008 100644 +--- a/job.c ++++ b/job.c +@@ -56,3 +56,9 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + + return job; + } ++ ++void job_delete(Job *job) ++{ ++ g_free(job->id); ++ g_free(job); ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_dismiss.patch b/SOURCES/kvm-job-Add-job_dismiss.patch new file mode 100644 index 0000000..d567628 --- /dev/null +++ b/SOURCES/kvm-job-Add-job_dismiss.patch @@ -0,0 +1,174 @@ +From 83f845c133441c92d87a0040f4b52c3e2654b995 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:26 +0200 +Subject: [PATCH 118/268] job: Add job_dismiss() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-44-kwolf@redhat.com> +Patchwork-id: 81125 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 43/73] job: Add job_dismiss() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves block_job_dismiss() to the Job layer. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 5f9a6a08e8f65e01746d2485fc65a3a78e74865f) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockdev.c | 10 ++++++---- + blockjob.c | 13 ------------- + include/block/blockjob.h | 9 --------- + include/qemu/job.h | 7 ++++++- + job.c | 15 ++++++++++++++- + tests/test-blockjob.c | 4 ++-- + 6 files changed, 28 insertions(+), 30 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index c768e68..721dc9a 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3916,14 +3916,16 @@ void qmp_block_job_finalize(const char *id, Error **errp) + void qmp_block_job_dismiss(const char *id, Error **errp) + { + AioContext *aio_context; +- BlockJob *job = find_block_job(id, &aio_context, errp); ++ BlockJob *bjob = find_block_job(id, &aio_context, errp); ++ Job *job; + +- if (!job) { ++ if (!bjob) { + return; + } + +- trace_qmp_block_job_dismiss(job); +- block_job_dismiss(&job, errp); ++ trace_qmp_block_job_dismiss(bjob); ++ job = &bjob->job; ++ job_dismiss(&job, errp); + aio_context_release(aio_context); + } + +diff --git a/blockjob.c b/blockjob.c +index f146fe0..3ca009b 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -242,19 +242,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) + return ratelimit_calculate_delay(&job->limit, n); + } + +-void block_job_dismiss(BlockJob **jobptr, Error **errp) +-{ +- BlockJob *job = *jobptr; +- /* similarly to _complete, this is QMP-interface only. */ +- assert(job->job.id); +- if (job_apply_verb(&job->job, JOB_VERB_DISMISS, errp)) { +- return; +- } +- +- job_do_dismiss(&job->job); +- *jobptr = NULL; +-} +- + void block_job_progress_update(BlockJob *job, uint64_t done) + { + job->offset += done; +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index e9ed7b8..5a81afc 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -141,15 +141,6 @@ void block_job_remove_all_bdrv(BlockJob *job); + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); + + /** +- * block_job_dismiss: +- * @job: The job to be dismissed. +- * @errp: Error object. +- * +- * Remove a concluded job from the query list. +- */ +-void block_job_dismiss(BlockJob **job, Error **errp); +- +-/** + * block_job_progress_update: + * @job: The job that has made progress + * @done: How much progress the job made +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 94900ec..1e8050c 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -487,6 +487,12 @@ int job_complete_sync(Job *job, Error **errp); + */ + void job_finalize(Job *job, Error **errp); + ++/** ++ * Remove the concluded @job from the query list and resets the passed pointer ++ * to %NULL. Returns an error if the job is not actually concluded. ++ */ ++void job_dismiss(Job **job, Error **errp); ++ + typedef void JobDeferToMainLoopFn(Job *job, void *opaque); + + /** +@@ -515,6 +521,5 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + + /* TODO To be removed from the public interface */ + void job_state_transition(Job *job, JobStatus s1); +-void job_do_dismiss(Job *job); + + #endif +diff --git a/job.c b/job.c +index eede680..7cd3602 100644 +--- a/job.c ++++ b/job.c +@@ -568,7 +568,7 @@ void job_user_resume(Job *job, Error **errp) + job_resume(job); + } + +-void job_do_dismiss(Job *job) ++static void job_do_dismiss(Job *job) + { + assert(job); + job->busy = false; +@@ -581,6 +581,19 @@ void job_do_dismiss(Job *job) + job_unref(job); + } + ++void job_dismiss(Job **jobptr, Error **errp) ++{ ++ Job *job = *jobptr; ++ /* similarly to _complete, this is QMP-interface only. */ ++ assert(job->id); ++ if (job_apply_verb(job, JOB_VERB_DISMISS, errp)) { ++ return; ++ } ++ ++ job_do_dismiss(job); ++ *jobptr = NULL; ++} ++ + void job_early_fail(Job *job) + { + assert(job->status == JOB_STATUS_CREATED); +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 46a7873..7131cab 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -233,8 +233,8 @@ static void cancel_common(CancelJob *s) + + job_cancel_sync(&job->job); + if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { +- BlockJob *dummy = job; +- block_job_dismiss(&dummy, &error_abort); ++ Job *dummy = &job->job; ++ job_dismiss(&dummy, &error_abort); + } + assert(job->job.status == JOB_STATUS_NULL); + job_unref(&job->job); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_drain.patch b/SOURCES/kvm-job-Add-job_drain.patch new file mode 100644 index 0000000..4ccba89 --- /dev/null +++ b/SOURCES/kvm-job-Add-job_drain.patch @@ -0,0 +1,281 @@ +From ba0f4624b12b27820e7e59b5e2a6a84f9736533d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:18 +0200 +Subject: [PATCH 110/268] job: Add job_drain() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-36-kwolf@redhat.com> +Patchwork-id: 81119 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 35/73] job: Add job_drain() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +block_job_drain() contains a blk_drain() call which cannot be moved to +Job, so add a new JobDriver callback JobDriver.drain which has a common +implementation for all BlockJobs. In addition to this we keep the +existing BlockJobDriver.drain callback that is called by the common +drain implementation for all block jobs. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit b69f777dd9ba992fdd35828a90eefcd88c0ec332) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 1 + + block/commit.c | 1 + + block/mirror.c | 2 ++ + block/stream.c | 1 + + blockjob.c | 20 ++++++++++---------- + include/block/blockjob_int.h | 12 ++++++++++++ + include/qemu/job.h | 13 +++++++++++++ + job.c | 11 +++++++++++ + tests/test-bdrv-drain.c | 1 + + tests/test-blockjob-txn.c | 1 + + tests/test-blockjob.c | 2 ++ + 11 files changed, 55 insertions(+), 10 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index bd31282..ca7d990 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -529,6 +529,7 @@ static const BlockJobDriver backup_job_driver = { + .job_type = JOB_TYPE_BACKUP, + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = backup_run, + .commit = backup_commit, + .abort = backup_abort, +diff --git a/block/commit.c b/block/commit.c +index e53b2d7..02a8af9 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -221,6 +221,7 @@ static const BlockJobDriver commit_job_driver = { + .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = commit_run, + }, + }; +diff --git a/block/mirror.c b/block/mirror.c +index c3951d1..a579bd8 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -992,6 +992,7 @@ static const BlockJobDriver mirror_job_driver = { + .job_type = JOB_TYPE_MIRROR, + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = mirror_run, + .pause = mirror_pause, + }, +@@ -1006,6 +1007,7 @@ static const BlockJobDriver commit_active_job_driver = { + .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = mirror_run, + .pause = mirror_pause, + }, +diff --git a/block/stream.c b/block/stream.c +index eee0253..b996278 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -215,6 +215,7 @@ static const BlockJobDriver stream_job_driver = { + .free = block_job_free, + .start = stream_run, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + }, + }; + +diff --git a/blockjob.c b/blockjob.c +index 4cac367..63e1669 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -169,14 +169,13 @@ static void block_job_attached_aio_context(AioContext *new_context, + job_resume(&job->job); + } + +-static void block_job_drain(BlockJob *job) ++void block_job_drain(Job *job) + { +- /* If job is !job->job.busy this kicks it into the next pause point. */ +- block_job_enter(job); ++ BlockJob *bjob = container_of(job, BlockJob, job); + +- blk_drain(job->blk); +- if (job->driver->drain) { +- job->driver->drain(job); ++ blk_drain(bjob->blk); ++ if (bjob->driver->drain) { ++ bjob->driver->drain(bjob); + } + } + +@@ -190,7 +189,7 @@ static void block_job_detach_aio_context(void *opaque) + job_pause(&job->job); + + while (!job->job.paused && !job_is_completed(&job->job)) { +- block_job_drain(job); ++ job_drain(&job->job); + } + + job->job.aio_context = NULL; +@@ -327,11 +326,11 @@ static int block_job_finish_sync(BlockJob *job, + job_unref(&job->job); + return -EBUSY; + } +- /* block_job_drain calls block_job_enter, and it should be enough to +- * induce progress until the job completes or moves to the main thread. ++ /* job_drain calls job_enter, and it should be enough to induce progress ++ * until the job completes or moves to the main thread. + */ + while (!job->job.deferred_to_main_loop && !job_is_completed(&job->job)) { +- block_job_drain(job); ++ job_drain(&job->job); + } + while (!job_is_completed(&job->job)) { + aio_poll(qemu_get_aio_context(), true); +@@ -713,6 +712,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + assert(is_block_job(&job->job)); + assert(job->job.driver->free == &block_job_free); + assert(job->job.driver->user_resume == &block_job_user_resume); ++ assert(job->job.driver->drain == &block_job_drain); + + job->driver = driver; + job->blk = blk; +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index bf2b762..38fe22d 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -65,6 +65,10 @@ struct BlockJobDriver { + * If the callback is not NULL, it will be invoked when the job has to be + * synchronously cancelled or completed; it should drain BlockDriverStates + * as required to ensure progress. ++ * ++ * Block jobs must use the default implementation for job_driver.drain, ++ * which will in turn call this callback after doing generic block job ++ * stuff. + */ + void (*drain)(BlockJob *job); + }; +@@ -112,6 +116,14 @@ void block_job_free(Job *job); + void block_job_user_resume(Job *job); + + /** ++ * block_job_drain: ++ * Callback to be used for JobDriver.drain in all block jobs. Drains the main ++ * block node associated with the block jobs and calls BlockJobDriver.drain for ++ * job-specific actions. ++ */ ++void block_job_drain(Job *job); ++ ++/** + * block_job_yield: + * @job: The job that calls the function. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 2648c74..aebc195 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -167,6 +167,13 @@ struct JobDriver { + */ + void (*user_resume)(Job *job); + ++ /* ++ * If the callback is not NULL, it will be invoked when the job has to be ++ * synchronously cancelled or completed; it should drain any activities ++ * as required to ensure progress. ++ */ ++ void (*drain)(Job *job); ++ + /** + * If the callback is not NULL, it will be invoked when all the jobs + * belonging to the same transaction complete; or upon this job's +@@ -325,6 +332,12 @@ bool job_user_paused(Job *job); + */ + void job_user_resume(Job *job, Error **errp); + ++/* ++ * Drain any activities as required to ensure progress. This can be called in a ++ * loop to synchronously complete a job. ++ */ ++void job_drain(Job *job); ++ + /** + * Get the next element from the list of block jobs after @job, or the + * first one if @job is %NULL. +diff --git a/job.c b/job.c +index 64b64da..3772a35 100644 +--- a/job.c ++++ b/job.c +@@ -367,6 +367,17 @@ void coroutine_fn job_sleep_ns(Job *job, int64_t ns) + job_pause_point(job); + } + ++void job_drain(Job *job) ++{ ++ /* If job is !busy this kicks it into the next pause point. */ ++ job_enter(job); ++ ++ if (job->driver->drain) { ++ job->driver->drain(job); ++ } ++} ++ ++ + /** + * All jobs must allow a pause point before entering their job proper. This + * ensures that jobs can be paused prior to being started, then resumed later. +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index c993512..58ea566 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -525,6 +525,7 @@ BlockJobDriver test_job_driver = { + .instance_size = sizeof(TestBlockJob), + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = test_job_start, + }, + .complete = test_job_complete, +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 60e9fa2..1572f8d 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -79,6 +79,7 @@ static const BlockJobDriver test_block_job_driver = { + .instance_size = sizeof(TestBlockJob), + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = test_block_job_run, + }, + }; +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 1fe6803..592a136 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -21,6 +21,7 @@ static const BlockJobDriver test_block_job_driver = { + .instance_size = sizeof(BlockJob), + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + }, + }; + +@@ -201,6 +202,7 @@ static const BlockJobDriver test_cancel_driver = { + .instance_size = sizeof(CancelJob), + .free = block_job_free, + .user_resume = block_job_user_resume, ++ .drain = block_job_drain, + .start = cancel_job_start, + }, + .complete = cancel_job_complete, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_event_.patch b/SOURCES/kvm-job-Add-job_event_.patch new file mode 100644 index 0000000..a8d8d16 --- /dev/null +++ b/SOURCES/kvm-job-Add-job_event_.patch @@ -0,0 +1,232 @@ +From 941f553a991d152f7691f5b1e3d8459b4843a51b Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:15 +0200 +Subject: [PATCH 107/268] job: Add job_event_*() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-33-kwolf@redhat.com> +Patchwork-id: 81092 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 32/73] job: Add job_event_*() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Go through the Job layer in order to send QMP events. For the moment, +these functions only call a notifier in the BlockJob layer that sends +the existing commands. + +This uses notifiers rather than JobDriver callbacks because internal +users of jobs won't receive QMP events, but might still be interested +in getting notified for the events. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 139a9f020d49e9f863e0d46fd3d0b440dfb3b9d7) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 41 +++++++++++++++++++++++++++-------------- + include/block/blockjob.h | 9 +++++++++ + include/qemu/job.h | 18 ++++++++++++++++++ + job.c | 19 +++++++++++++++++++ + 4 files changed, 73 insertions(+), 14 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index b4334fb..05d7921 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -36,10 +36,6 @@ + #include "qemu/coroutine.h" + #include "qemu/timer.h" + +-static void block_job_event_cancelled(BlockJob *job); +-static void block_job_event_completed(BlockJob *job, const char *msg); +-static void block_job_event_pending(BlockJob *job); +- + /* Transactional group of block jobs */ + struct BlockJobTxn { + +@@ -352,13 +348,9 @@ static int block_job_finalize_single(BlockJob *job) + /* Emit events only if we actually started */ + if (job_started(&job->job)) { + if (job_is_cancelled(&job->job)) { +- block_job_event_cancelled(job); ++ job_event_cancelled(&job->job); + } else { +- const char *msg = NULL; +- if (job->ret < 0) { +- msg = strerror(-job->ret); +- } +- block_job_event_completed(job, msg); ++ job_event_completed(&job->job); + } + } + +@@ -504,7 +496,7 @@ static int block_job_transition_to_pending(BlockJob *job) + { + job_state_transition(&job->job, JOB_STATUS_PENDING); + if (!job->job.auto_finalize) { +- block_job_event_pending(job); ++ job_event_pending(&job->job); + } + return 0; + } +@@ -712,8 +704,10 @@ static void block_job_iostatus_set_err(BlockJob *job, int error) + } + } + +-static void block_job_event_cancelled(BlockJob *job) ++static void block_job_event_cancelled(Notifier *n, void *opaque) + { ++ BlockJob *job = opaque; ++ + if (block_job_is_internal(job)) { + return; + } +@@ -726,12 +720,19 @@ static void block_job_event_cancelled(BlockJob *job) + &error_abort); + } + +-static void block_job_event_completed(BlockJob *job, const char *msg) ++static void block_job_event_completed(Notifier *n, void *opaque) + { ++ BlockJob *job = opaque; ++ const char *msg = NULL; ++ + if (block_job_is_internal(job)) { + return; + } + ++ if (job->ret < 0) { ++ msg = strerror(-job->ret); ++ } ++ + qapi_event_send_block_job_completed(job_type(&job->job), + job->job.id, + job->len, +@@ -742,8 +743,10 @@ static void block_job_event_completed(BlockJob *job, const char *msg) + &error_abort); + } + +-static void block_job_event_pending(BlockJob *job) ++static void block_job_event_pending(Notifier *n, void *opaque) + { ++ BlockJob *job = opaque; ++ + if (block_job_is_internal(job)) { + return; + } +@@ -799,6 +802,16 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->cb = cb; + job->opaque = opaque; + ++ job->finalize_cancelled_notifier.notify = block_job_event_cancelled; ++ job->finalize_completed_notifier.notify = block_job_event_completed; ++ job->pending_notifier.notify = block_job_event_pending; ++ ++ notifier_list_add(&job->job.on_finalize_cancelled, ++ &job->finalize_cancelled_notifier); ++ notifier_list_add(&job->job.on_finalize_completed, ++ &job->finalize_completed_notifier); ++ notifier_list_add(&job->job.on_pending, &job->pending_notifier); ++ + error_setg(&job->blocker, "block device is in use by block job: %s", + job_type_str(&job->job)); + block_job_add_bdrv(job, "main node", bs, 0, BLK_PERM_ALL, &error_abort); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index f9aaaaa..aef0629 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -82,6 +82,15 @@ typedef struct BlockJob { + /** Block other operations when block job is running */ + Error *blocker; + ++ /** Called when a cancelled job is finalised. */ ++ Notifier finalize_cancelled_notifier; ++ ++ /** Called when a successfully completed job is finalised. */ ++ Notifier finalize_completed_notifier; ++ ++ /** Called when the job transitions to PENDING */ ++ Notifier pending_notifier; ++ + /** BlockDriverStates that are involved in this block job */ + GSList *nodes; + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 9783e40..14d9377 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -105,6 +105,15 @@ typedef struct Job { + /** True if this job should automatically dismiss itself */ + bool auto_dismiss; + ++ /** Notifiers called when a cancelled job is finalised */ ++ NotifierList on_finalize_cancelled; ++ ++ /** Notifiers called when a successfully completed job is finalised */ ++ NotifierList on_finalize_completed; ++ ++ /** Notifiers called when the job transitions to PENDING */ ++ NotifierList on_pending; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + } Job; +@@ -182,6 +191,15 @@ void job_ref(Job *job); + */ + void job_unref(Job *job); + ++/** To be called when a cancelled job is finalised. */ ++void job_event_cancelled(Job *job); ++ ++/** To be called when a successfully completed job is finalised. */ ++void job_event_completed(Job *job); ++ ++/** To be called when the job transitions to PENDING */ ++void job_event_pending(Job *job); ++ + /** + * Conditionally enter the job coroutine if the job is ready to run, not + * already busy and fn() returns true. fn() is called while under the job_lock +diff --git a/job.c b/job.c +index dd46170..817c3b4 100644 +--- a/job.c ++++ b/job.c +@@ -215,6 +215,10 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + job->auto_finalize = !(flags & JOB_MANUAL_FINALIZE); + job->auto_dismiss = !(flags & JOB_MANUAL_DISMISS); + ++ notifier_list_init(&job->on_finalize_cancelled); ++ notifier_list_init(&job->on_finalize_completed); ++ notifier_list_init(&job->on_pending); ++ + job_state_transition(job, JOB_STATUS_CREATED); + aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, + QEMU_CLOCK_REALTIME, SCALE_NS, +@@ -247,6 +251,21 @@ void job_unref(Job *job) + } + } + ++void job_event_cancelled(Job *job) ++{ ++ notifier_list_notify(&job->on_finalize_cancelled, job); ++} ++ ++void job_event_completed(Job *job) ++{ ++ notifier_list_notify(&job->on_finalize_completed, job); ++} ++ ++void job_event_pending(Job *job) ++{ ++ notifier_list_notify(&job->on_pending, job); ++} ++ + void job_enter_cond(Job *job, bool(*fn)(Job *job)) + { + if (!job_started(job)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_is_ready.patch b/SOURCES/kvm-job-Add-job_is_ready.patch new file mode 100644 index 0000000..c542564 --- /dev/null +++ b/SOURCES/kvm-job-Add-job_is_ready.patch @@ -0,0 +1,155 @@ +From 15f474087cf25ad960213ec63a7376c639fceb05 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:27 +0200 +Subject: [PATCH 119/268] job: Add job_is_ready() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-45-kwolf@redhat.com> +Patchwork-id: 81128 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 44/73] job: Add job_is_ready() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Instead of having a 'bool ready' in BlockJob, add a function that +derives its value from the job status. + +At the same time, this fixes the behaviour to match what the QAPI +documentation promises for query-block-job: 'true if the job may be +completed'. When the ready flag was introduced in commit ef6dbf1e46e, +the flag never had to be reset to match the description because after +being ready, the jobs would immediately complete and disappear. + +Job transactions and manual job finalisation were introduced only later. +With these changes, jobs may stay around even after having completed +(and they are not ready to be completed a second time), however their +patches forgot to reset the ready flag. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit df956ae2014340bf7de0190edb1d09be55d9eadf) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 3 +-- + include/block/blockjob.h | 5 ----- + include/qemu/job.h | 3 +++ + job.c | 22 ++++++++++++++++++++++ + qemu-img.c | 2 +- + tests/test-blockjob.c | 2 +- + 6 files changed, 28 insertions(+), 9 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 3ca009b..38f18e9 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -269,7 +269,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info->offset = job->offset; + info->speed = job->speed; + info->io_status = job->iostatus; +- info->ready = job->ready; ++ info->ready = job_is_ready(&job->job), + info->status = job->job.status; + info->auto_finalize = job->job.auto_finalize; + info->auto_dismiss = job->job.auto_dismiss; +@@ -436,7 +436,6 @@ void block_job_user_resume(Job *job) + void block_job_event_ready(BlockJob *job) + { + job_state_transition(&job->job, JOB_STATUS_READY); +- job->ready = true; + + if (block_job_is_internal(job)) { + return; +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 5a81afc..8e1e1ee 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -49,11 +49,6 @@ typedef struct BlockJob { + /** The block device on which the job is operating. */ + BlockBackend *blk; + +- /** +- * Set to true when the job is ready to be completed. +- */ +- bool ready; +- + /** Status that is published by the query-block-jobs QMP API */ + BlockDeviceIoStatus iostatus; + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 1e8050c..487f9d9 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -367,6 +367,9 @@ bool job_is_cancelled(Job *job); + /** Returns whether the job is in a completed state. */ + bool job_is_completed(Job *job); + ++/** Returns whether the job is ready to be completed. */ ++bool job_is_ready(Job *job); ++ + /** + * Request @job to pause at the next pause point. Must be paired with + * job_resume(). If the job is supposed to be resumed by user action, call +diff --git a/job.c b/job.c +index 7cd3602..aa4c746 100644 +--- a/job.c ++++ b/job.c +@@ -199,6 +199,28 @@ bool job_is_cancelled(Job *job) + return job->cancelled; + } + ++bool job_is_ready(Job *job) ++{ ++ switch (job->status) { ++ case JOB_STATUS_UNDEFINED: ++ case JOB_STATUS_CREATED: ++ case JOB_STATUS_RUNNING: ++ case JOB_STATUS_PAUSED: ++ case JOB_STATUS_WAITING: ++ case JOB_STATUS_PENDING: ++ case JOB_STATUS_ABORTING: ++ case JOB_STATUS_CONCLUDED: ++ case JOB_STATUS_NULL: ++ return false; ++ case JOB_STATUS_READY: ++ case JOB_STATUS_STANDBY: ++ return true; ++ default: ++ g_assert_not_reached(); ++ } ++ return false; ++} ++ + bool job_is_completed(Job *job) + { + switch (job->status) { +diff --git a/qemu-img.c b/qemu-img.c +index 734ea94..3c449a2 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -878,7 +878,7 @@ static void run_block_job(BlockJob *job, Error **errp) + aio_poll(aio_context, true); + qemu_progress_print(job->len ? + ((float)job->offset / job->len * 100.f) : 0.0f, 0); +- } while (!job->ready && !job_is_completed(&job->job)); ++ } while (!job_is_ready(&job->job) && !job_is_completed(&job->job)); + + if (!job_is_completed(&job->job)) { + ret = job_complete_sync(&job->job, errp); +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 7131cab..8180d03 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -185,7 +185,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + goto defer; + } + +- if (!s->common.ready && s->should_converge) { ++ if (!job_is_ready(&s->common.job) && s->should_converge) { + block_job_event_ready(&s->common); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_sleep_ns.patch b/SOURCES/kvm-job-Add-job_sleep_ns.patch new file mode 100644 index 0000000..04d626f --- /dev/null +++ b/SOURCES/kvm-job-Add-job_sleep_ns.patch @@ -0,0 +1,363 @@ +From 5da39ce2527a4c7e64543f1b0a3733a30cedbc9d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:10 +0200 +Subject: [PATCH 102/268] job: Add job_sleep_ns() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-28-kwolf@redhat.com> +Patchwork-id: 81117 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 27/73] job: Add job_sleep_ns() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +There is nothing block layer specific about block_job_sleep_ns(), so +move the function to Job. + +Signed-off-by: Kevin Wolf +Reviewed-by: John Snow +Reviewed-by: Max Reitz +(cherry picked from commit 5d43e86e11f488fda7956b13160e0c0105a84845) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + block/commit.c | 2 +- + block/mirror.c | 4 ++-- + block/stream.c | 2 +- + blockjob.c | 27 --------------------------- + include/block/blockjob_int.h | 11 ----------- + include/qemu/job.h | 19 ++++++++++++++++++- + job.c | 32 ++++++++++++++++++++++++++++++++ + tests/test-bdrv-drain.c | 8 ++++---- + tests/test-blockjob-txn.c | 2 +- + tests/test-blockjob.c | 2 +- + 11 files changed, 61 insertions(+), 50 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 7d9aad9..f3a4f7c 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -338,7 +338,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) + * return. Without a yield, the VM would not reboot. */ + delay_ns = block_job_ratelimit_get_delay(&job->common, job->bytes_read); + job->bytes_read = 0; +- block_job_sleep_ns(&job->common, delay_ns); ++ job_sleep_ns(&job->common.job, delay_ns); + + if (job_is_cancelled(&job->common.job)) { + return true; +diff --git a/block/commit.c b/block/commit.c +index 2fbc310..1c6cb6c 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -172,7 +172,7 @@ static void coroutine_fn commit_run(void *opaque) + /* Note that even when no rate limit is applied we need to yield + * with no pending I/O here so that bdrv_drain_all() returns. + */ +- block_job_sleep_ns(&s->common, delay_ns); ++ job_sleep_ns(&s->common.job, delay_ns); + if (job_is_cancelled(&s->common.job)) { + break; + } +diff --git a/block/mirror.c b/block/mirror.c +index 95fc807..5d8f75c 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -595,7 +595,7 @@ static void mirror_throttle(MirrorBlockJob *s) + + if (now - s->last_pause_ns > BLOCK_JOB_SLICE_TIME) { + s->last_pause_ns = now; +- block_job_sleep_ns(&s->common, 0); ++ job_sleep_ns(&s->common.job, 0); + } else { + job_pause_point(&s->common.job); + } +@@ -869,7 +869,7 @@ static void coroutine_fn mirror_run(void *opaque) + cnt == 0 ? BLOCK_JOB_SLICE_TIME : 0); + } + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); +- block_job_sleep_ns(&s->common, delay_ns); ++ job_sleep_ns(&s->common.job, delay_ns); + if (job_is_cancelled(&s->common.job) && + (!s->synced || s->common.force)) + { +diff --git a/block/stream.c b/block/stream.c +index 6d8b7b6..1faab02 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -140,7 +140,7 @@ static void coroutine_fn stream_run(void *opaque) + /* Note that even when no rate limit is applied we need to yield + * with no pending I/O here so that bdrv_drain_all() returns. + */ +- block_job_sleep_ns(&s->common, delay_ns); ++ job_sleep_ns(&s->common.job, delay_ns); + if (job_is_cancelled(&s->common.job)) { + break; + } +diff --git a/blockjob.c b/blockjob.c +index 313b1ff..4dc360c 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -181,7 +181,6 @@ void block_job_free(Job *job) + block_job_detach_aio_context, bjob); + blk_unref(bjob->blk); + error_free(bjob->blocker); +- assert(!timer_pending(&bjob->job.sleep_timer)); + } + + static void block_job_attached_aio_context(AioContext *new_context, +@@ -290,13 +289,6 @@ const BlockJobDriver *block_job_driver(BlockJob *job) + return job->driver; + } + +-static void block_job_sleep_timer_cb(void *opaque) +-{ +- BlockJob *job = opaque; +- +- block_job_enter(job); +-} +- + static void block_job_decommission(BlockJob *job) + { + assert(job); +@@ -866,9 +858,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->opaque = opaque; + job->auto_finalize = !(flags & BLOCK_JOB_MANUAL_FINALIZE); + job->auto_dismiss = !(flags & BLOCK_JOB_MANUAL_DISMISS); +- aio_timer_init(qemu_get_aio_context(), &job->job.sleep_timer, +- QEMU_CLOCK_REALTIME, SCALE_NS, +- block_job_sleep_timer_cb, job); + + error_setg(&job->blocker, "block device is in use by block job: %s", + job_type_str(&job->job)); +@@ -931,22 +920,6 @@ void block_job_enter(BlockJob *job) + job_enter_cond(&job->job, NULL); + } + +-void block_job_sleep_ns(BlockJob *job, int64_t ns) +-{ +- assert(job->job.busy); +- +- /* Check cancellation *before* setting busy = false, too! */ +- if (job_is_cancelled(&job->job)) { +- return; +- } +- +- if (!job_should_pause(&job->job)) { +- job_do_yield(&job->job, qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + ns); +- } +- +- job_pause_point(&job->job); +-} +- + void block_job_yield(BlockJob *job) + { + assert(job->job.busy); +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 0a614a8..8937f5b 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -134,17 +134,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + void block_job_free(Job *job); + + /** +- * block_job_sleep_ns: +- * @job: The job that calls the function. +- * @ns: How many nanoseconds to stop for. +- * +- * Put the job to sleep (assuming that it wasn't canceled) for @ns +- * %QEMU_CLOCK_REALTIME nanoseconds. Canceling the job will immediately +- * interrupt the wait. +- */ +-void block_job_sleep_ns(BlockJob *job, int64_t ns); +- +-/** + * block_job_yield: + * @job: The job that calls the function. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 9dcff12..509408f 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -58,7 +58,7 @@ typedef struct Job { + Coroutine *co; + + /** +- * Timer that is used by @block_job_sleep_ns. Accessed under job_mutex (in ++ * Timer that is used by @job_sleep_ns. Accessed under job_mutex (in + * job.c). + */ + QEMUTimer sleep_timer; +@@ -168,6 +168,13 @@ void job_enter_cond(Job *job, bool(*fn)(Job *job)); + void job_start(Job *job); + + /** ++ * @job: The job to enter. ++ * ++ * Continue the specified job by entering the coroutine. ++ */ ++void job_enter(Job *job); ++ ++/** + * @job: The job that is ready to pause. + * + * Pause now if job_pause() has been called. Jobs that perform lots of I/O +@@ -175,6 +182,16 @@ void job_start(Job *job); + */ + void coroutine_fn job_pause_point(Job *job); + ++/** ++ * @job: The job that calls the function. ++ * @ns: How many nanoseconds to stop for. ++ * ++ * Put the job to sleep (assuming that it wasn't canceled) for @ns ++ * %QEMU_CLOCK_REALTIME nanoseconds. Canceling the job will immediately ++ * interrupt the wait. ++ */ ++void coroutine_fn job_sleep_ns(Job *job, int64_t ns); ++ + + /** Returns the JobType of a given Job. */ + JobType job_type(const Job *job); +diff --git a/job.c b/job.c +index 78497fd..1b8cba1 100644 +--- a/job.c ++++ b/job.c +@@ -152,6 +152,13 @@ Job *job_get(const char *id) + return NULL; + } + ++static void job_sleep_timer_cb(void *opaque) ++{ ++ Job *job = opaque; ++ ++ job_enter(job); ++} ++ + void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + Error **errp) + { +@@ -178,6 +185,9 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + job->pause_count = 1; + + job_state_transition(job, JOB_STATUS_CREATED); ++ aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, ++ QEMU_CLOCK_REALTIME, SCALE_NS, ++ job_sleep_timer_cb, job); + + QLIST_INSERT_HEAD(&jobs, job, job_list); + +@@ -193,6 +203,7 @@ void job_unref(Job *job) + { + if (--job->refcnt == 0) { + assert(job->status == JOB_STATUS_NULL); ++ assert(!timer_pending(&job->sleep_timer)); + + if (job->driver->free) { + job->driver->free(job); +@@ -232,6 +243,11 @@ void job_enter_cond(Job *job, bool(*fn)(Job *job)) + aio_co_wake(job->co); + } + ++void job_enter(Job *job) ++{ ++ job_enter_cond(job, NULL); ++} ++ + /* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. + * Reentering the job coroutine with block_job_enter() before the timer has + * expired is allowed and cancels the timer. +@@ -283,6 +299,22 @@ void coroutine_fn job_pause_point(Job *job) + } + } + ++void coroutine_fn job_sleep_ns(Job *job, int64_t ns) ++{ ++ assert(job->busy); ++ ++ /* Check cancellation *before* setting busy = false, too! */ ++ if (job_is_cancelled(job)) { ++ return; ++ } ++ ++ if (!job_should_pause(job)) { ++ job_do_yield(job, qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + ns); ++ } ++ ++ job_pause_point(job); ++} ++ + /** + * All jobs must allow a pause point before entering their job proper. This + * ensures that jobs can be paused prior to being started, then resumed later. +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index c9f2f9b..50232f5 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -508,7 +508,7 @@ static void coroutine_fn test_job_start(void *opaque) + + block_job_event_ready(&s->common); + while (!s->should_complete) { +- block_job_sleep_ns(&s->common, 100000); ++ job_sleep_ns(&s->common.job, 100000); + } + + job_defer_to_main_loop(&s->common.job, test_job_completed, NULL); +@@ -553,7 +553,7 @@ static void test_blockjob_common(enum drain_type drain_type) + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_false(job->job.busy); /* We're in block_job_sleep_ns() */ ++ g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ + + do_drain_begin(drain_type, src); + +@@ -571,7 +571,7 @@ static void test_blockjob_common(enum drain_type drain_type) + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_false(job->job.busy); /* We're in block_job_sleep_ns() */ ++ g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ + + do_drain_begin(drain_type, target); + +@@ -589,7 +589,7 @@ static void test_blockjob_common(enum drain_type drain_type) + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_false(job->job.busy); /* We're in block_job_sleep_ns() */ ++ g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ + + ret = block_job_complete_sync(job, &error_abort); + g_assert_cmpint(ret, ==, 0); +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 323e154..0e6162b 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -45,7 +45,7 @@ static void coroutine_fn test_block_job_run(void *opaque) + + while (s->iterations--) { + if (s->use_timer) { +- block_job_sleep_ns(job, 0); ++ job_sleep_ns(&job->job, 0); + } else { + block_job_yield(job); + } +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 1d18325..b329bd5 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -188,7 +188,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + block_job_event_ready(&s->common); + } + +- block_job_sleep_ns(&s->common, 100000); ++ job_sleep_ns(&s->common.job, 100000); + } + + defer: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_transition_to_ready.patch b/SOURCES/kvm-job-Add-job_transition_to_ready.patch new file mode 100644 index 0000000..62e9ab1 --- /dev/null +++ b/SOURCES/kvm-job-Add-job_transition_to_ready.patch @@ -0,0 +1,267 @@ +From 603278c7c9f5e89f7db4c98b81d2b3d64a6bbe6b Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:28 +0200 +Subject: [PATCH 120/268] job: Add job_transition_to_ready() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-46-kwolf@redhat.com> +Patchwork-id: 81114 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 45/73] job: Add job_transition_to_ready() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +The transition to the READY state was still performed in the BlockJob +layer, in the same function that sent the BLOCK_JOB_READY QMP event. + +This patch brings the state transition to the Job layer and implements +the QMP event using a notifier called from the Job layer, like we +already do for other events related to state transitions. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 2e1795b58131427719c7cd11f8b9b6984b3f24f8) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/mirror.c | 6 +++--- + blockjob.c | 33 ++++++++++++++++++--------------- + include/block/blockjob.h | 3 +++ + include/block/blockjob_int.h | 8 -------- + include/qemu/job.h | 9 ++++++--- + job.c | 16 +++++++++++++--- + tests/test-bdrv-drain.c | 2 +- + tests/test-blockjob.c | 2 +- + 8 files changed, 45 insertions(+), 34 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 687f955..bdc1b5b 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -727,8 +727,8 @@ static void coroutine_fn mirror_run(void *opaque) + } + + if (s->bdev_length == 0) { +- /* Report BLOCK_JOB_READY and wait for complete. */ +- block_job_event_ready(&s->common); ++ /* Transition to the READY state and wait for complete. */ ++ job_transition_to_ready(&s->common.job); + s->synced = true; + while (!job_is_cancelled(&s->common.job) && !s->should_complete) { + job_yield(&s->common.job); +@@ -824,7 +824,7 @@ static void coroutine_fn mirror_run(void *opaque) + * report completion. This way, block-job-cancel will leave + * the target in a consistent state. + */ +- block_job_event_ready(&s->common); ++ job_transition_to_ready(&s->common.job); + s->synced = true; + } + +diff --git a/blockjob.c b/blockjob.c +index 38f18e9..da11b3b 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -338,6 +338,22 @@ static void block_job_event_pending(Notifier *n, void *opaque) + &error_abort); + } + ++static void block_job_event_ready(Notifier *n, void *opaque) ++{ ++ BlockJob *job = opaque; ++ ++ if (block_job_is_internal(job)) { ++ return; ++ } ++ ++ qapi_event_send_block_job_ready(job_type(&job->job), ++ job->job.id, ++ job->len, ++ job->offset, ++ job->speed, &error_abort); ++} ++ ++ + /* + * API for block job drivers and the block layer. These functions are + * declared in blockjob_int.h. +@@ -386,12 +402,14 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->finalize_cancelled_notifier.notify = block_job_event_cancelled; + job->finalize_completed_notifier.notify = block_job_event_completed; + job->pending_notifier.notify = block_job_event_pending; ++ job->ready_notifier.notify = block_job_event_ready; + + notifier_list_add(&job->job.on_finalize_cancelled, + &job->finalize_cancelled_notifier); + notifier_list_add(&job->job.on_finalize_completed, + &job->finalize_completed_notifier); + notifier_list_add(&job->job.on_pending, &job->pending_notifier); ++ notifier_list_add(&job->job.on_ready, &job->ready_notifier); + + error_setg(&job->blocker, "block device is in use by block job: %s", + job_type_str(&job->job)); +@@ -433,21 +451,6 @@ void block_job_user_resume(Job *job) + block_job_iostatus_reset(bjob); + } + +-void block_job_event_ready(BlockJob *job) +-{ +- job_state_transition(&job->job, JOB_STATUS_READY); +- +- if (block_job_is_internal(job)) { +- return; +- } +- +- qapi_event_send_block_job_ready(job_type(&job->job), +- job->job.id, +- job->len, +- job->offset, +- job->speed, &error_abort); +-} +- + BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, + int is_read, int error) + { +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 8e1e1ee..4fca45f 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -76,6 +76,9 @@ typedef struct BlockJob { + /** Called when the job transitions to PENDING */ + Notifier pending_notifier; + ++ /** Called when the job transitions to READY */ ++ Notifier ready_notifier; ++ + /** BlockDriverStates that are involved in this block job */ + GSList *nodes; + } BlockJob; +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 806ac64..5cd50c6 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -116,14 +116,6 @@ void block_job_drain(Job *job); + int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n); + + /** +- * block_job_event_ready: +- * @job: The job which is now ready to be completed. +- * +- * Send a BLOCK_JOB_READY event for the specified job. +- */ +-void block_job_event_ready(BlockJob *job); +- +-/** + * block_job_error_action: + * @job: The job to signal an error for. + * @on_err: The error action setting. +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 487f9d9..bfc2bc5 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -132,6 +132,9 @@ typedef struct Job { + /** Notifiers called when the job transitions to PENDING */ + NotifierList on_pending; + ++ /** Notifiers called when the job transitions to READY */ ++ NotifierList on_ready; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + +@@ -426,6 +429,9 @@ int job_apply_verb(Job *job, JobVerb verb, Error **errp); + /** The @job could not be started, free it. */ + void job_early_fail(Job *job); + ++/** Moves the @job from RUNNING to READY */ ++void job_transition_to_ready(Job *job); ++ + /** + * @job: The job being completed. + * @ret: The status code. +@@ -522,7 +528,4 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque); + */ + int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp); + +-/* TODO To be removed from the public interface */ +-void job_state_transition(Job *job, JobStatus s1); +- + #endif +diff --git a/job.c b/job.c +index aa4c746..b5bd51b 100644 +--- a/job.c ++++ b/job.c +@@ -157,9 +157,7 @@ static int job_txn_apply(JobTxn *txn, int fn(Job *), bool lock) + return rc; + } + +- +-/* TODO Make static once the whole state machine is in job.c */ +-void job_state_transition(Job *job, JobStatus s1) ++static void job_state_transition(Job *job, JobStatus s1) + { + JobStatus s0 = job->status; + assert(s1 >= 0 && s1 <= JOB_STATUS__MAX); +@@ -321,6 +319,7 @@ void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, + notifier_list_init(&job->on_finalize_cancelled); + notifier_list_init(&job->on_finalize_completed); + notifier_list_init(&job->on_pending); ++ notifier_list_init(&job->on_ready); + + job_state_transition(job, JOB_STATUS_CREATED); + aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, +@@ -380,6 +379,11 @@ static void job_event_pending(Job *job) + notifier_list_notify(&job->on_pending, job); + } + ++static void job_event_ready(Job *job) ++{ ++ notifier_list_notify(&job->on_ready, job); ++} ++ + void job_enter_cond(Job *job, bool(*fn)(Job *job)) + { + if (!job_started(job)) { +@@ -799,6 +803,12 @@ static int job_transition_to_pending(Job *job) + return 0; + } + ++void job_transition_to_ready(Job *job) ++{ ++ job_state_transition(job, JOB_STATUS_READY); ++ job_event_ready(job); ++} ++ + static void job_completed_txn_success(Job *job) + { + JobTxn *txn = job->txn; +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 3600ffd..2cba63b 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -505,7 +505,7 @@ static void coroutine_fn test_job_start(void *opaque) + { + TestBlockJob *s = opaque; + +- block_job_event_ready(&s->common); ++ job_transition_to_ready(&s->common.job); + while (!s->should_complete) { + job_sleep_ns(&s->common.job, 100000); + } +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 8180d03..e408d52 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -186,7 +186,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + } + + if (!job_is_ready(&s->common.job) && s->should_converge) { +- block_job_event_ready(&s->common); ++ job_transition_to_ready(&s->common.job); + } + + job_sleep_ns(&s->common.job, 100000); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-job_yield.patch b/SOURCES/kvm-job-Add-job_yield.patch new file mode 100644 index 0000000..5390b3b --- /dev/null +++ b/SOURCES/kvm-job-Add-job_yield.patch @@ -0,0 +1,191 @@ +From cb84a9e38278854566813e8e5d275de77a0a9019 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:25 +0200 +Subject: [PATCH 117/268] job: Add job_yield() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-43-kwolf@redhat.com> +Patchwork-id: 81120 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 42/73] job: Add job_yield() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves block_job_yield() to the Job layer. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 198c49cc8d81e8eb0df3749d395599895c3a3a76) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + block/mirror.c | 2 +- + blockjob.c | 16 ---------------- + include/block/blockjob_int.h | 8 -------- + include/qemu/job.h | 9 +++++++-- + job.c | 20 ++++++++++++++++++-- + tests/test-blockjob-txn.c | 2 +- + 7 files changed, 28 insertions(+), 31 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index b13f91d..6f4f3df 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -444,7 +444,7 @@ static void coroutine_fn backup_run(void *opaque) + while (!job_is_cancelled(&job->common.job)) { + /* Yield until the job is cancelled. We just let our before_write + * notify callback service CoW requests. */ +- block_job_yield(&job->common); ++ job_yield(&job->common.job); + } + } else if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { + ret = backup_run_incremental(job); +diff --git a/block/mirror.c b/block/mirror.c +index c63cf7c..687f955 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -731,7 +731,7 @@ static void coroutine_fn mirror_run(void *opaque) + block_job_event_ready(&s->common); + s->synced = true; + while (!job_is_cancelled(&s->common.job) && !s->should_complete) { +- block_job_yield(&s->common); ++ job_yield(&s->common.job); + } + s->common.job.cancelled = false; + goto immediate_exit; +diff --git a/blockjob.c b/blockjob.c +index 438baa1..f146fe0 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -431,22 +431,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return job; + } + +-void block_job_yield(BlockJob *job) +-{ +- assert(job->job.busy); +- +- /* Check cancellation *before* setting busy = false, too! */ +- if (job_is_cancelled(&job->job)) { +- return; +- } +- +- if (!job_should_pause(&job->job)) { +- job_do_yield(&job->job, -1); +- } +- +- job_pause_point(&job->job); +-} +- + void block_job_iostatus_reset(BlockJob *job) + { + if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) { +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 7df07b2..806ac64 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -108,14 +108,6 @@ void block_job_user_resume(Job *job); + void block_job_drain(Job *job); + + /** +- * block_job_yield: +- * @job: The job that calls the function. +- * +- * Yield the block job coroutine. +- */ +-void block_job_yield(BlockJob *job); +- +-/** + * block_job_ratelimit_get_delay: + * + * Calculate and return delay for the next request in ns. See the documentation +diff --git a/include/qemu/job.h b/include/qemu/job.h +index bbe1b0c..94900ec 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -339,6 +339,13 @@ void coroutine_fn job_pause_point(Job *job); + + /** + * @job: The job that calls the function. ++ * ++ * Yield the job coroutine. ++ */ ++void job_yield(Job *job); ++ ++/** ++ * @job: The job that calls the function. + * @ns: How many nanoseconds to stop for. + * + * Put the job to sleep (assuming that it wasn't canceled) for @ns +@@ -508,8 +515,6 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + + /* TODO To be removed from the public interface */ + void job_state_transition(Job *job, JobStatus s1); +-void coroutine_fn job_do_yield(Job *job, uint64_t ns); +-bool job_should_pause(Job *job); + void job_do_dismiss(Job *job); + + #endif +diff --git a/job.c b/job.c +index 2e453f6..eede680 100644 +--- a/job.c ++++ b/job.c +@@ -226,7 +226,7 @@ static bool job_started(Job *job) + return job->co; + } + +-bool job_should_pause(Job *job) ++static bool job_should_pause(Job *job) + { + return job->pause_count > 0; + } +@@ -396,7 +396,7 @@ void job_enter(Job *job) + * + * If @ns is (uint64_t) -1, no timer is scheduled and job_enter() must be + * called explicitly. */ +-void coroutine_fn job_do_yield(Job *job, uint64_t ns) ++static void coroutine_fn job_do_yield(Job *job, uint64_t ns) + { + job_lock(); + if (ns != -1) { +@@ -441,6 +441,22 @@ void coroutine_fn job_pause_point(Job *job) + } + } + ++void job_yield(Job *job) ++{ ++ assert(job->busy); ++ ++ /* Check cancellation *before* setting busy = false, too! */ ++ if (job_is_cancelled(job)) { ++ return; ++ } ++ ++ if (!job_should_pause(job)) { ++ job_do_yield(job, -1); ++ } ++ ++ job_pause_point(job); ++} ++ + void coroutine_fn job_sleep_ns(Job *job, int64_t ns) + { + assert(job->busy); +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 34ee179..fce8366 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -47,7 +47,7 @@ static void coroutine_fn test_block_job_run(void *opaque) + if (s->use_timer) { + job_sleep_ns(&job->job, 0); + } else { +- block_job_yield(job); ++ job_yield(&job->job); + } + + if (job_is_cancelled(&job->job)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-lifecycle-QMP-commands.patch b/SOURCES/kvm-job-Add-lifecycle-QMP-commands.patch new file mode 100644 index 0000000..2cae905 --- /dev/null +++ b/SOURCES/kvm-job-Add-lifecycle-QMP-commands.patch @@ -0,0 +1,324 @@ +From d960ab86c5ad6eafff3d1d550a80226209b24062 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:32 +0200 +Subject: [PATCH 124/268] job: Add lifecycle QMP commands + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-50-kwolf@redhat.com> +Patchwork-id: 81129 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 49/73] job: Add lifecycle QMP commands +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds QMP commands that control the transition between states of the +job lifecycle. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 1a90bc8128ee7d16ce4abb131961e37084d75b16) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + MAINTAINERS | 1 + + Makefile.objs | 1 + + job-qmp.c | 134 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + qapi/job.json | 99 +++++++++++++++++++++++++++++++++++++++++++ + trace-events | 9 ++++ + 5 files changed, 244 insertions(+) + create mode 100644 job-qmp.c + +diff --git a/MAINTAINERS b/MAINTAINERS +index 5aaf264..a783c92 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1372,6 +1372,7 @@ S: Supported + F: blockjob.c + F: include/block/blockjob.h + F: job.c ++F: job-qmp.c + F: include/block/job.h + F: block/backup.c + F: block/commit.c +diff --git a/Makefile.objs b/Makefile.objs +index 3df8d58..c6c3554 100644 +--- a/Makefile.objs ++++ b/Makefile.objs +@@ -97,6 +97,7 @@ io-obj-y = io/ + ifeq ($(CONFIG_SOFTMMU),y) + common-obj-y = blockdev.o blockdev-nbd.o block/ + common-obj-y += bootdevice.o iothread.o ++common-obj-y += job-qmp.o + common-obj-y += net/ + common-obj-y += qdev-monitor.o device-hotplug.o + common-obj-$(CONFIG_WIN32) += os-win32.o +diff --git a/job-qmp.c b/job-qmp.c +new file mode 100644 +index 0000000..b2e18cf +--- /dev/null ++++ b/job-qmp.c +@@ -0,0 +1,134 @@ ++/* ++ * QMP interface for background jobs ++ * ++ * Copyright (c) 2011 IBM Corp. ++ * Copyright (c) 2012, 2018 Red Hat, Inc. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ++ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ */ ++ ++#include "qemu/osdep.h" ++#include "qemu-common.h" ++#include "qemu/job.h" ++#include "qapi/qapi-commands-job.h" ++#include "qapi/error.h" ++#include "trace-root.h" ++ ++/* Get a job using its ID and acquire its AioContext */ ++static Job *find_job(const char *id, AioContext **aio_context, Error **errp) ++{ ++ Job *job; ++ ++ *aio_context = NULL; ++ ++ job = job_get(id); ++ if (!job) { ++ error_setg(errp, "Job not found"); ++ return NULL; ++ } ++ ++ *aio_context = job->aio_context; ++ aio_context_acquire(*aio_context); ++ ++ return job; ++} ++ ++void qmp_job_cancel(const char *id, Error **errp) ++{ ++ AioContext *aio_context; ++ Job *job = find_job(id, &aio_context, errp); ++ ++ if (!job) { ++ return; ++ } ++ ++ trace_qmp_job_cancel(job); ++ job_user_cancel(job, true, errp); ++ aio_context_release(aio_context); ++} ++ ++void qmp_job_pause(const char *id, Error **errp) ++{ ++ AioContext *aio_context; ++ Job *job = find_job(id, &aio_context, errp); ++ ++ if (!job) { ++ return; ++ } ++ ++ trace_qmp_job_pause(job); ++ job_user_pause(job, errp); ++ aio_context_release(aio_context); ++} ++ ++void qmp_job_resume(const char *id, Error **errp) ++{ ++ AioContext *aio_context; ++ Job *job = find_job(id, &aio_context, errp); ++ ++ if (!job) { ++ return; ++ } ++ ++ trace_qmp_job_resume(job); ++ job_user_resume(job, errp); ++ aio_context_release(aio_context); ++} ++ ++void qmp_job_complete(const char *id, Error **errp) ++{ ++ AioContext *aio_context; ++ Job *job = find_job(id, &aio_context, errp); ++ ++ if (!job) { ++ return; ++ } ++ ++ trace_qmp_job_complete(job); ++ job_complete(job, errp); ++ aio_context_release(aio_context); ++} ++ ++void qmp_job_finalize(const char *id, Error **errp) ++{ ++ AioContext *aio_context; ++ Job *job = find_job(id, &aio_context, errp); ++ ++ if (!job) { ++ return; ++ } ++ ++ trace_qmp_job_finalize(job); ++ job_finalize(job, errp); ++ aio_context_release(aio_context); ++} ++ ++void qmp_job_dismiss(const char *id, Error **errp) ++{ ++ AioContext *aio_context; ++ Job *job = find_job(id, &aio_context, errp); ++ ++ if (!job) { ++ return; ++ } ++ ++ trace_qmp_job_dismiss(job); ++ job_dismiss(&job, errp); ++ aio_context_release(aio_context); ++} +diff --git a/qapi/job.json b/qapi/job.json +index 9fbdd0c..b84dc6c 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -106,3 +106,102 @@ + { 'event': 'JOB_STATUS_CHANGE', + 'data': { 'id': 'str', + 'status': 'JobStatus' } } ++ ++## ++# @job-pause: ++# ++# Pause an active job. ++# ++# This command returns immediately after marking the active job for pausing. ++# Pausing an already paused job is an error. ++# ++# The job will pause as soon as possible, which means transitioning into the ++# PAUSED state if it was RUNNING, or into STANDBY if it was READY. The ++# corresponding JOB_STATUS_CHANGE event will be emitted. ++# ++# Cancelling a paused job automatically resumes it. ++# ++# @id: The job identifier. ++# ++# Since: 2.13 ++## ++{ 'command': 'job-pause', 'data': { 'id': 'str' } } ++ ++## ++# @job-resume: ++# ++# Resume a paused job. ++# ++# This command returns immediately after resuming a paused job. Resuming an ++# already running job is an error. ++# ++# @id : The job identifier. ++# ++# Since: 2.13 ++## ++{ 'command': 'job-resume', 'data': { 'id': 'str' } } ++ ++## ++# @job-cancel: ++# ++# Instruct an active background job to cancel at the next opportunity. ++# This command returns immediately after marking the active job for ++# cancellation. ++# ++# The job will cancel as soon as possible and then emit a JOB_STATUS_CHANGE ++# event. Usually, the status will change to ABORTING, but it is possible that ++# a job successfully completes (e.g. because it was almost done and there was ++# no opportunity to cancel earlier than completing the job) and transitions to ++# PENDING instead. ++# ++# @id: The job identifier. ++# ++# Since: 2.13 ++## ++{ 'command': 'job-cancel', 'data': { 'id': 'str' } } ++ ++ ++## ++# @job-complete: ++# ++# Manually trigger completion of an active job in the READY state. ++# ++# @id: The job identifier. ++# ++# Since: 2.13 ++## ++{ 'command': 'job-complete', 'data': { 'id': 'str' } } ++ ++## ++# @job-dismiss: ++# ++# Deletes a job that is in the CONCLUDED state. This command only needs to be ++# run explicitly for jobs that don't have automatic dismiss enabled. ++# ++# This command will refuse to operate on any job that has not yet reached its ++# terminal state, JOB_STATUS_CONCLUDED. For jobs that make use of JOB_READY ++# event, job-cancel or job-complete will still need to be used as appropriate. ++# ++# @id: The job identifier. ++# ++# Since: 2.13 ++## ++{ 'command': 'job-dismiss', 'data': { 'id': 'str' } } ++ ++## ++# @job-finalize: ++# ++# Instructs all jobs in a transaction (or a single job if it is not part of any ++# transaction) to finalize any graph changes and do any necessary cleanup. This ++# command requires that all involved jobs are in the PENDING state. ++# ++# For jobs in a transaction, instructing one job to finalize will force ++# ALL jobs in the transaction to finalize, so it is only necessary to instruct ++# a single member job to finalize. ++# ++# @id: The identifier of any job in the transaction, or of a job that is not ++# part of any transaction. ++# ++# Since: 2.13 ++## ++{ 'command': 'job-finalize', 'data': { 'id': 'str' } } +diff --git a/trace-events b/trace-events +index ef7579a..c445f54 100644 +--- a/trace-events ++++ b/trace-events +@@ -109,6 +109,15 @@ job_state_transition(void *job, int ret, const char *legal, const char *s0, con + job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" + job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" + ++# job-qmp.c ++qmp_job_cancel(void *job) "job %p" ++qmp_job_pause(void *job) "job %p" ++qmp_job_resume(void *job) "job %p" ++qmp_job_complete(void *job) "job %p" ++qmp_job_finalize(void *job) "job %p" ++qmp_job_dismiss(void *job) "job %p" ++ ++ + ### Guest events, keep at bottom + + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-query-jobs-QMP-command.patch b/SOURCES/kvm-job-Add-query-jobs-QMP-command.patch new file mode 100644 index 0000000..981074a --- /dev/null +++ b/SOURCES/kvm-job-Add-query-jobs-QMP-command.patch @@ -0,0 +1,175 @@ +From a6b5a44e6aeb7b7baf1e9dd7796511850a15cbd1 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:33 +0200 +Subject: [PATCH 125/268] job: Add query-jobs QMP command + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-51-kwolf@redhat.com> +Patchwork-id: 81095 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 50/73] job: Add query-jobs QMP command +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a minimal query-jobs implementation that shouldn't pose many +design questions. It can later be extended to expose more information, +and especially job-specific information. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 456273b02474780537e2bb52a72213f63bb5227a) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + include/qemu/job.h | 3 +++ + job-qmp.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ + job.c | 2 +- + qapi/job.json | 46 ++++++++++++++++++++++++++++++++++++++++++++++ + 4 files changed, 104 insertions(+), 1 deletion(-) + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 92d1d24..8c8badf 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -392,6 +392,9 @@ JobType job_type(const Job *job); + /** Returns the enum string for the JobType of a given Job. */ + const char *job_type_str(const Job *job); + ++/** Returns true if the job should not be visible to the management layer. */ ++bool job_is_internal(Job *job); ++ + /** Returns whether the job is scheduled for cancellation. */ + bool job_is_cancelled(Job *job); + +diff --git a/job-qmp.c b/job-qmp.c +index b2e18cf..7f38f63 100644 +--- a/job-qmp.c ++++ b/job-qmp.c +@@ -132,3 +132,57 @@ void qmp_job_dismiss(const char *id, Error **errp) + job_dismiss(&job, errp); + aio_context_release(aio_context); + } ++ ++static JobInfo *job_query_single(Job *job, Error **errp) ++{ ++ JobInfo *info; ++ const char *errmsg = NULL; ++ ++ assert(!job_is_internal(job)); ++ ++ if (job->ret < 0) { ++ errmsg = strerror(-job->ret); ++ } ++ ++ info = g_new(JobInfo, 1); ++ *info = (JobInfo) { ++ .id = g_strdup(job->id), ++ .type = job_type(job), ++ .status = job->status, ++ .current_progress = job->progress_current, ++ .total_progress = job->progress_total, ++ .has_error = !!errmsg, ++ .error = g_strdup(errmsg), ++ }; ++ ++ return info; ++} ++ ++JobInfoList *qmp_query_jobs(Error **errp) ++{ ++ JobInfoList *head = NULL, **p_next = &head; ++ Job *job; ++ ++ for (job = job_next(NULL); job; job = job_next(job)) { ++ JobInfoList *elem; ++ AioContext *aio_context; ++ ++ if (job_is_internal(job)) { ++ continue; ++ } ++ elem = g_new0(JobInfoList, 1); ++ aio_context = job->aio_context; ++ aio_context_acquire(aio_context); ++ elem->value = job_query_single(job, errp); ++ aio_context_release(aio_context); ++ if (!elem->value) { ++ g_free(elem); ++ qapi_free_JobInfoList(head); ++ return NULL; ++ } ++ *p_next = elem; ++ p_next = &elem->next; ++ } ++ ++ return head; ++} +diff --git a/job.c b/job.c +index 599a104..f026661 100644 +--- a/job.c ++++ b/job.c +@@ -158,7 +158,7 @@ static int job_txn_apply(JobTxn *txn, int fn(Job *), bool lock) + return rc; + } + +-static bool job_is_internal(Job *job) ++bool job_is_internal(Job *job) + { + return (job->id == NULL); + } +diff --git a/qapi/job.json b/qapi/job.json +index b84dc6c..970124d 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -205,3 +205,49 @@ + # Since: 2.13 + ## + { 'command': 'job-finalize', 'data': { 'id': 'str' } } ++ ++## ++# @JobInfo: ++# ++# Information about a job. ++# ++# @id: The job identifier ++# ++# @type: The kind of job that is being performed ++# ++# @status: Current job state/status ++# ++# @current-progress: Progress made until now. The unit is arbitrary and the ++# value can only meaningfully be used for the ratio of ++# @current-progress to @total-progress. The value is ++# monotonically increasing. ++# ++# @total-progress: Estimated @current-progress value at the completion of ++# the job. This value can arbitrarily change while the ++# job is running, in both directions. ++# ++# @error: If this field is present, the job failed; if it is ++# still missing in the CONCLUDED state, this indicates ++# successful completion. ++# ++# The value is a human-readable error message to describe ++# the reason for the job failure. It should not be parsed ++# by applications. ++# ++# Since: 2.13 ++## ++{ 'struct': 'JobInfo', ++ 'data': { 'id': 'str', 'type': 'JobType', 'status': 'JobStatus', ++ 'current-progress': 'int', 'total-progress': 'int', ++ '*error': 'str' } } ++ ++## ++# @query-jobs: ++# ++# Return information about jobs. ++# ++# Returns: a list with a @JobInfo for each active job ++# ++# Since: 2.13 ++## ++{ 'command': 'query-jobs', 'returns': ['JobInfo'] } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Add-reference-counting.patch b/SOURCES/kvm-job-Add-reference-counting.patch new file mode 100644 index 0000000..7ec9f35 --- /dev/null +++ b/SOURCES/kvm-job-Add-reference-counting.patch @@ -0,0 +1,446 @@ +From 2f7ae3f2886ec34dfb0a51e75d0f62998213928a Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:05 +0200 +Subject: [PATCH 097/268] job: Add reference counting + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-23-kwolf@redhat.com> +Patchwork-id: 81073 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 22/73] job: Add reference counting +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves reference counting from BlockJob to Job. + +In order to keep calling the BlockJob cleanup code when the job is +deleted via job_unref(), introduce a new JobDriver.free callback. Every +block job must use block_job_free() for this callback, this is asserted +in block_job_create(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 80fa2c756b3241f24015a7503a01f7999d4a942d) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 1 + + block/commit.c | 1 + + block/mirror.c | 2 ++ + block/stream.c | 1 + + blockjob.c | 48 +++++++++++++++++++------------------------- + include/block/blockjob.h | 21 ------------------- + include/block/blockjob_int.h | 7 +++++++ + include/qemu/job.h | 19 ++++++++++++++++-- + job.c | 22 ++++++++++++++++---- + qemu-img.c | 4 ++-- + tests/test-bdrv-drain.c | 1 + + tests/test-blockjob-txn.c | 1 + + tests/test-blockjob.c | 6 ++++-- + 13 files changed, 76 insertions(+), 58 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index baf8d43..cfdb89d 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -526,6 +526,7 @@ static const BlockJobDriver backup_job_driver = { + .job_driver = { + .instance_size = sizeof(BackupBlockJob), + .job_type = JOB_TYPE_BACKUP, ++ .free = block_job_free, + }, + .start = backup_run, + .commit = backup_commit, +diff --git a/block/commit.c b/block/commit.c +index 32d29c8..925c96a 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -218,6 +218,7 @@ static const BlockJobDriver commit_job_driver = { + .job_driver = { + .instance_size = sizeof(CommitBlockJob), + .job_type = JOB_TYPE_COMMIT, ++ .free = block_job_free, + }, + .start = commit_run, + }; +diff --git a/block/mirror.c b/block/mirror.c +index 35fcc1f..0df4f70 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -989,6 +989,7 @@ static const BlockJobDriver mirror_job_driver = { + .job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = JOB_TYPE_MIRROR, ++ .free = block_job_free, + }, + .start = mirror_run, + .complete = mirror_complete, +@@ -1001,6 +1002,7 @@ static const BlockJobDriver commit_active_job_driver = { + .job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = JOB_TYPE_COMMIT, ++ .free = block_job_free, + }, + .start = mirror_run, + .complete = mirror_complete, +diff --git a/block/stream.c b/block/stream.c +index cb723f1..7273d22 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -212,6 +212,7 @@ static const BlockJobDriver stream_job_driver = { + .job_driver = { + .instance_size = sizeof(StreamBlockJob), + .job_type = JOB_TYPE_STREAM, ++ .free = block_job_free, + }, + .start = stream_run, + }; +diff --git a/blockjob.c b/blockjob.c +index 0fba01e..0bf0a26 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -190,31 +190,25 @@ static void block_job_resume(BlockJob *job) + block_job_enter_cond(job, block_job_timer_not_pending); + } + +-void block_job_ref(BlockJob *job) +-{ +- ++job->refcnt; +-} +- + static void block_job_attached_aio_context(AioContext *new_context, + void *opaque); + static void block_job_detach_aio_context(void *opaque); + +-void block_job_unref(BlockJob *job) ++void block_job_free(Job *job) + { +- if (--job->refcnt == 0) { +- assert(job->job.status == JOB_STATUS_NULL); +- assert(!job->txn); +- BlockDriverState *bs = blk_bs(job->blk); +- bs->job = NULL; +- block_job_remove_all_bdrv(job); +- blk_remove_aio_context_notifier(job->blk, +- block_job_attached_aio_context, +- block_job_detach_aio_context, job); +- blk_unref(job->blk); +- error_free(job->blocker); +- assert(!timer_pending(&job->sleep_timer)); +- job_delete(&job->job); +- } ++ BlockJob *bjob = container_of(job, BlockJob, job); ++ BlockDriverState *bs = blk_bs(bjob->blk); ++ ++ assert(!bjob->txn); ++ ++ bs->job = NULL; ++ block_job_remove_all_bdrv(bjob); ++ blk_remove_aio_context_notifier(bjob->blk, ++ block_job_attached_aio_context, ++ block_job_detach_aio_context, bjob); ++ blk_unref(bjob->blk); ++ error_free(bjob->blocker); ++ assert(!timer_pending(&bjob->sleep_timer)); + } + + static void block_job_attached_aio_context(AioContext *new_context, +@@ -245,7 +239,7 @@ static void block_job_detach_aio_context(void *opaque) + BlockJob *job = opaque; + + /* In case the job terminates during aio_poll()... */ +- block_job_ref(job); ++ job_ref(&job->job); + + block_job_pause(job); + +@@ -253,7 +247,7 @@ static void block_job_detach_aio_context(void *opaque) + block_job_drain(job); + } + +- block_job_unref(job); ++ job_unref(&job->job); + } + + static char *child_job_get_parent_desc(BdrvChild *c) +@@ -367,7 +361,7 @@ static void block_job_decommission(BlockJob *job) + job->deferred_to_main_loop = true; + block_job_txn_del_job(job); + job_state_transition(&job->job, JOB_STATUS_NULL); +- block_job_unref(job); ++ job_unref(&job->job); + } + + static void block_job_do_dismiss(BlockJob *job) +@@ -506,14 +500,14 @@ static int block_job_finish_sync(BlockJob *job, + + assert(blk_bs(job->blk)->job == job); + +- block_job_ref(job); ++ job_ref(&job->job); + + if (finish) { + finish(job, &local_err); + } + if (local_err) { + error_propagate(errp, local_err); +- block_job_unref(job); ++ job_unref(&job->job); + return -EBUSY; + } + /* block_job_drain calls block_job_enter, and it should be enough to +@@ -526,7 +520,7 @@ static int block_job_finish_sync(BlockJob *job, + aio_poll(qemu_get_aio_context(), true); + } + ret = (job->cancelled && job->ret == 0) ? -ECANCELED : job->ret; +- block_job_unref(job); ++ job_unref(&job->job); + return ret; + } + +@@ -909,6 +903,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + } + + assert(is_block_job(&job->job)); ++ assert(job->job.driver->free == &block_job_free); + + job->driver = driver; + job->blk = blk; +@@ -917,7 +912,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->busy = false; + job->paused = true; + job->pause_count = 1; +- job->refcnt = 1; + job->auto_finalize = !(flags & BLOCK_JOB_MANUAL_FINALIZE); + job->auto_dismiss = !(flags & BLOCK_JOB_MANUAL_DISMISS); + aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 01cdee6..087e782 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -132,9 +132,6 @@ typedef struct BlockJob { + /** The opaque value that is passed to the completion function. */ + void *opaque; + +- /** Reference count of the block job */ +- int refcnt; +- + /** True when job has reported completion by calling block_job_completed. */ + bool completed; + +@@ -400,24 +397,6 @@ void block_job_iostatus_reset(BlockJob *job); + BlockJobTxn *block_job_txn_new(void); + + /** +- * block_job_ref: +- * +- * Add a reference to BlockJob refcnt, it will be decreased with +- * block_job_unref, and then be freed if it comes to be the last +- * reference. +- */ +-void block_job_ref(BlockJob *job); +- +-/** +- * block_job_unref: +- * +- * Release a reference that was previously acquired with block_job_ref +- * or block_job_create. If it's the last reference to the object, it will be +- * freed. +- */ +-void block_job_unref(BlockJob *job); +- +-/** + * block_job_txn_unref: + * + * Release a reference that was previously acquired with block_job_txn_add_job +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 1e62d6d..6f0fe3c 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -144,6 +144,13 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + BlockCompletionFunc *cb, void *opaque, Error **errp); + + /** ++ * block_job_free: ++ * Callback to be used for JobDriver.free in all block jobs. Frees block job ++ * specific resources in @job. ++ */ ++void block_job_free(Job *job); ++ ++/** + * block_job_sleep_ns: + * @job: The job that calls the function. + * @ns: How many nanoseconds to stop for. +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 0b78778..0751e2a 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -41,6 +41,9 @@ typedef struct Job { + /** The type of this job. */ + const JobDriver *driver; + ++ /** Reference count of the block job */ ++ int refcnt; ++ + /** Current state; See @JobStatus for details. */ + JobStatus status; + +@@ -57,6 +60,9 @@ struct JobDriver { + + /** Enum describing the operation */ + JobType job_type; ++ ++ /** Called when the job is freed */ ++ void (*free)(Job *job); + }; + + +@@ -69,8 +75,17 @@ struct JobDriver { + */ + void *job_create(const char *job_id, const JobDriver *driver, Error **errp); + +-/** Frees the @job object. */ +-void job_delete(Job *job); ++/** ++ * Add a reference to Job refcnt, it will be decreased with job_unref, and then ++ * be freed if it comes to be the last reference. ++ */ ++void job_ref(Job *job); ++ ++/** ++ * Release a reference that was previously acquired with job_ref() or ++ * job_create(). If it's the last reference to the object, it will be freed. ++ */ ++void job_unref(Job *job); + + /** Returns the JobType of a given Job. */ + JobType job_type(const Job *job); +diff --git a/job.c b/job.c +index b049a32..926f1de 100644 +--- a/job.c ++++ b/job.c +@@ -134,6 +134,7 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + job = g_malloc0(driver->instance_size); + job->driver = driver; + job->id = g_strdup(job_id); ++ job->refcnt = 1; + + job_state_transition(job, JOB_STATUS_CREATED); + +@@ -142,10 +143,23 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + return job; + } + +-void job_delete(Job *job) ++void job_ref(Job *job) + { +- QLIST_REMOVE(job, job_list); ++ ++job->refcnt; ++} ++ ++void job_unref(Job *job) ++{ ++ if (--job->refcnt == 0) { ++ assert(job->status == JOB_STATUS_NULL); + +- g_free(job->id); +- g_free(job); ++ if (job->driver->free) { ++ job->driver->free(job); ++ } ++ ++ QLIST_REMOVE(job, job_list); ++ ++ g_free(job->id); ++ g_free(job); ++ } + } +diff --git a/qemu-img.c b/qemu-img.c +index e2395b9..f745919 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -873,7 +873,7 @@ static void run_block_job(BlockJob *job, Error **errp) + int ret = 0; + + aio_context_acquire(aio_context); +- block_job_ref(job); ++ job_ref(&job->job); + do { + aio_poll(aio_context, true); + qemu_progress_print(job->len ? +@@ -885,7 +885,7 @@ static void run_block_job(BlockJob *job, Error **errp) + } else { + ret = job->ret; + } +- block_job_unref(job); ++ job_unref(&job->job); + aio_context_release(aio_context); + + /* publish completion progress only when success */ +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index fe9f412..f9e37d4 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -522,6 +522,7 @@ static void test_job_complete(BlockJob *job, Error **errp) + BlockJobDriver test_job_driver = { + .job_driver = { + .instance_size = sizeof(TestBlockJob), ++ .free = block_job_free, + }, + .start = test_job_start, + .complete = test_job_complete, +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 48b12d1..b49b28c 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -76,6 +76,7 @@ static void test_block_job_cb(void *opaque, int ret) + static const BlockJobDriver test_block_job_driver = { + .job_driver = { + .instance_size = sizeof(TestBlockJob), ++ .free = block_job_free, + }, + .start = test_block_job_run, + }; +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 6ccd585..e24fc3f 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -19,6 +19,7 @@ + static const BlockJobDriver test_block_job_driver = { + .job_driver = { + .instance_size = sizeof(BlockJob), ++ .free = block_job_free, + }, + }; + +@@ -196,6 +197,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + static const BlockJobDriver test_cancel_driver = { + .job_driver = { + .instance_size = sizeof(CancelJob), ++ .free = block_job_free, + }, + .start = cancel_job_start, + .complete = cancel_job_complete, +@@ -210,7 +212,7 @@ static CancelJob *create_common(BlockJob **pjob) + blk = create_blk(NULL); + job = mk_job(blk, "Steve", &test_cancel_driver, true, + BLOCK_JOB_MANUAL_FINALIZE | BLOCK_JOB_MANUAL_DISMISS); +- block_job_ref(job); ++ job_ref(&job->job); + assert(job->job.status == JOB_STATUS_CREATED); + s = container_of(job, CancelJob, common); + s->blk = blk; +@@ -231,7 +233,7 @@ static void cancel_common(CancelJob *s) + block_job_dismiss(&dummy, &error_abort); + } + assert(job->job.status == JOB_STATUS_NULL); +- block_job_unref(job); ++ job_unref(&job->job); + destroy_blk(blk); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Avoid-deadlocks-in-job_completed_txn_abort.patch b/SOURCES/kvm-job-Avoid-deadlocks-in-job_completed_txn_abort.patch new file mode 100644 index 0000000..961f346 --- /dev/null +++ b/SOURCES/kvm-job-Avoid-deadlocks-in-job_completed_txn_abort.patch @@ -0,0 +1,85 @@ +From 42e244782774dc971c83c4fabc16f46c57d86f21 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:09 +0100 +Subject: [PATCH 43/49] job: Avoid deadlocks in job_completed_txn_abort() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-31-kwolf@redhat.com> +Patchwork-id: 82622 +O-Subject: [RHEL-8 qemu-kvm PATCH 40/44] job: Avoid deadlocks in job_completed_txn_abort() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Amongst others, job_finalize_single() calls the .prepare/.commit/.abort +callbacks of the individual job driver. Recently, their use was adapted +for all block jobs so that they involve code calling AIO_WAIT_WHILE() +now. Such code must be called under the AioContext lock for the +respective job, but without holding any other AioContext lock. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 644f3a29bd4974aefd46d2adb5062d86063c8a50) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + job.c | 16 +++++++++++----- + 1 file changed, 11 insertions(+), 5 deletions(-) + +diff --git a/job.c b/job.c +index 42af9e2..5b53e43 100644 +--- a/job.c ++++ b/job.c +@@ -713,6 +713,7 @@ static void job_cancel_async(Job *job, bool force) + + static void job_completed_txn_abort(Job *job) + { ++ AioContext *outer_ctx = job->aio_context; + AioContext *ctx; + JobTxn *txn = job->txn; + Job *other_job; +@@ -726,23 +727,26 @@ static void job_completed_txn_abort(Job *job) + txn->aborting = true; + job_txn_ref(txn); + +- /* We are the first failed job. Cancel other jobs. */ +- QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- ctx = other_job->aio_context; +- aio_context_acquire(ctx); +- } ++ /* We can only hold the single job's AioContext lock while calling ++ * job_finalize_single() because the finalization callbacks can involve ++ * calls of AIO_WAIT_WHILE(), which could deadlock otherwise. */ ++ aio_context_release(outer_ctx); + + /* Other jobs are effectively cancelled by us, set the status for + * them; this job, however, may or may not be cancelled, depending + * on the caller, so leave it. */ + QLIST_FOREACH(other_job, &txn->jobs, txn_list) { + if (other_job != job) { ++ ctx = other_job->aio_context; ++ aio_context_acquire(ctx); + job_cancel_async(other_job, false); ++ aio_context_release(ctx); + } + } + while (!QLIST_EMPTY(&txn->jobs)) { + other_job = QLIST_FIRST(&txn->jobs); + ctx = other_job->aio_context; ++ aio_context_acquire(ctx); + if (!job_is_completed(other_job)) { + assert(job_is_cancelled(other_job)); + job_finish_sync(other_job, NULL, NULL); +@@ -751,6 +755,8 @@ static void job_completed_txn_abort(Job *job) + aio_context_release(ctx); + } + ++ aio_context_acquire(outer_ctx); ++ + job_txn_unref(txn); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Convert-block_job_cancel_async-to-Job.patch b/SOURCES/kvm-job-Convert-block_job_cancel_async-to-Job.patch new file mode 100644 index 0000000..35e7ceb --- /dev/null +++ b/SOURCES/kvm-job-Convert-block_job_cancel_async-to-Job.patch @@ -0,0 +1,159 @@ +From 949b0b3562c3c98ce64dc15825f30e2c82260c01 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:17 +0200 +Subject: [PATCH 109/268] job: Convert block_job_cancel_async() to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-35-kwolf@redhat.com> +Patchwork-id: 81072 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 34/73] job: Convert block_job_cancel_async() to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +block_job_cancel_async() did two things that were still block job +specific: + +* Setting job->force. This field makes sense on the Job level, so we can + just move it. While at it, rename it to job->force_cancel to make its + purpose more obvious. + +* Resetting the I/O status. This can't be moved because generic Jobs + don't have an I/O status. What the function really implements is a + user resume, except without entering the coroutine. Consequently, it + makes sense to call the .user_resume driver callback here which + already resets the I/O status. + + The old block_job_cancel_async() has two separate if statements that + check job->iostatus != BLOCK_DEVICE_IO_STATUS_OK and job->user_paused. + However, the former condition always implies the latter (as is + asserted in block_job_iostatus_reset()), so changing the explicit call + of block_job_iostatus_reset() on the former condition with the + .user_resume callback on the latter condition is equivalent and + doesn't need to access any BlockJob specific state. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 004e95df98266da33e08c9f1731aca71b6d6d7c4) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/mirror.c | 4 ++-- + blockjob.c | 25 +++++++++++++------------ + include/block/blockjob.h | 6 ------ + include/qemu/job.h | 6 ++++++ + 4 files changed, 21 insertions(+), 20 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index e9a90ea..c3951d1 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -871,7 +871,7 @@ static void coroutine_fn mirror_run(void *opaque) + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); + job_sleep_ns(&s->common.job, delay_ns); + if (job_is_cancelled(&s->common.job) && +- (!s->synced || s->common.force)) ++ (!s->synced || s->common.job.force_cancel)) + { + break; + } +@@ -884,7 +884,7 @@ immediate_exit: + * or it was cancelled prematurely so that we do not guarantee that + * the target is a copy of the source. + */ +- assert(ret < 0 || ((s->common.force || !s->synced) && ++ assert(ret < 0 || ((s->common.job.force_cancel || !s->synced) && + job_is_cancelled(&s->common.job))); + assert(need_drain); + mirror_wait_for_all_io(s); +diff --git a/blockjob.c b/blockjob.c +index 34c57da..4cac367 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -270,19 +270,20 @@ static int block_job_prepare(BlockJob *job) + return job->job.ret; + } + +-static void block_job_cancel_async(BlockJob *job, bool force) ++static void job_cancel_async(Job *job, bool force) + { +- if (job->iostatus != BLOCK_DEVICE_IO_STATUS_OK) { +- block_job_iostatus_reset(job); +- } +- if (job->job.user_paused) { +- /* Do not call block_job_enter here, the caller will handle it. */ +- job->job.user_paused = false; +- job->job.pause_count--; ++ if (job->user_paused) { ++ /* Do not call job_enter here, the caller will handle it. */ ++ job->user_paused = false; ++ if (job->driver->user_resume) { ++ job->driver->user_resume(job); ++ } ++ assert(job->pause_count > 0); ++ job->pause_count--; + } +- job->job.cancelled = true; ++ job->cancelled = true; + /* To prevent 'force == false' overriding a previous 'force == true' */ +- job->force |= force; ++ job->force_cancel |= force; + } + + static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *), bool lock) +@@ -367,7 +368,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + * on the caller, so leave it. */ + QLIST_FOREACH(other_job, &txn->jobs, txn_list) { + if (other_job != job) { +- block_job_cancel_async(other_job, false); ++ job_cancel_async(&other_job->job, false); + } + } + while (!QLIST_EMPTY(&txn->jobs)) { +@@ -527,7 +528,7 @@ void block_job_cancel(BlockJob *job, bool force) + job_do_dismiss(&job->job); + return; + } +- block_job_cancel_async(job, force); ++ job_cancel_async(&job->job, force); + if (!job_started(&job->job)) { + block_job_completed(job, -ECANCELED); + } else if (job->job.deferred_to_main_loop) { +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 3f405d1..d975efe 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -51,12 +51,6 @@ typedef struct BlockJob { + BlockBackend *blk; + + /** +- * Set to true if the job should abort immediately without waiting +- * for data to be in sync. +- */ +- bool force; +- +- /** + * Set to true when the job is ready to be completed. + */ + bool ready; +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 3e817be..2648c74 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -97,6 +97,12 @@ typedef struct Job { + */ + bool cancelled; + ++ /** ++ * Set to true if the job should abort immediately without waiting ++ * for data to be in sync. ++ */ ++ bool force_cancel; ++ + /** Set to true when the job has deferred work to the main loop. */ + bool deferred_to_main_loop; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Create-Job-JobDriver-and-job_create.patch b/SOURCES/kvm-job-Create-Job-JobDriver-and-job_create.patch new file mode 100644 index 0000000..1dcba98 --- /dev/null +++ b/SOURCES/kvm-job-Create-Job-JobDriver-and-job_create.patch @@ -0,0 +1,571 @@ +From c0a6a1f61a194ab1a96b0f722b9ddca7398f7dde Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:59 +0200 +Subject: [PATCH 091/268] job: Create Job, JobDriver and job_create() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-17-kwolf@redhat.com> +Patchwork-id: 81124 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 16/73] job: Create Job, JobDriver and job_create() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This is the first step towards creating an infrastructure for generic +background jobs that aren't tied to a block device. For now, Job only +stores its ID and JobDriver, the rest stays in BlockJob. + +The following patches will move over more parts of BlockJob to Job if +they are meaningful outside the context of a block job. + +BlockJob.driver is now redundant, but this patch leaves it around to +avoid unnecessary churn. The next patches will get rid of almost all of +its uses anyway so that it can be removed later with much less churn. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 33e9e9bd62d9ae00880d9e12ad8a5b7d2c00e8a5) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + MAINTAINERS | 2 ++ + Makefile.objs | 2 +- + block/backup.c | 4 ++- + block/commit.c | 4 ++- + block/mirror.c | 10 +++++--- + block/stream.c | 4 ++- + blockjob.c | 46 ++++++++++++++++----------------- + include/block/blockjob.h | 9 +++---- + include/block/blockjob_int.h | 4 +-- + include/qemu/job.h | 60 ++++++++++++++++++++++++++++++++++++++++++++ + job.c | 48 +++++++++++++++++++++++++++++++++++ + tests/test-bdrv-drain.c | 4 ++- + tests/test-blockjob-txn.c | 4 ++- + tests/test-blockjob.c | 12 ++++++--- + 14 files changed, 169 insertions(+), 44 deletions(-) + create mode 100644 include/qemu/job.h + create mode 100644 job.c + +diff --git a/MAINTAINERS b/MAINTAINERS +index ad5edc8..c5e3dfb 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1371,6 +1371,8 @@ L: qemu-block@nongnu.org + S: Supported + F: blockjob.c + F: include/block/blockjob.h ++F: job.c ++F: include/block/job.h + F: block/backup.c + F: block/commit.c + F: block/stream.c +diff --git a/Makefile.objs b/Makefile.objs +index c6c9b8f..92b73fc 100644 +--- a/Makefile.objs ++++ b/Makefile.objs +@@ -63,7 +63,7 @@ chardev-obj-y = chardev/ + # block-obj-y is code used by both qemu system emulation and qemu-img + + block-obj-y += nbd/ +-block-obj-y += block.o blockjob.o ++block-obj-y += block.o blockjob.o job.o + block-obj-y += block/ scsi/ + block-obj-y += qemu-io-cmds.o + block-obj-$(CONFIG_REPLICATION) += replication.o +diff --git a/block/backup.c b/block/backup.c +index e14d995..9e672bb 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -523,7 +523,9 @@ static void coroutine_fn backup_run(void *opaque) + } + + static const BlockJobDriver backup_job_driver = { +- .instance_size = sizeof(BackupBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(BackupBlockJob), ++ }, + .job_type = BLOCK_JOB_TYPE_BACKUP, + .start = backup_run, + .commit = backup_commit, +diff --git a/block/commit.c b/block/commit.c +index ba5df6a..18cbb2f 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -215,7 +215,9 @@ out: + } + + static const BlockJobDriver commit_job_driver = { +- .instance_size = sizeof(CommitBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(CommitBlockJob), ++ }, + .job_type = BLOCK_JOB_TYPE_COMMIT, + .start = commit_run, + }; +diff --git a/block/mirror.c b/block/mirror.c +index a4197bb..aa1d6b7 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -913,7 +913,7 @@ static void mirror_complete(BlockJob *job, Error **errp) + + if (!s->synced) { + error_setg(errp, "The active block job '%s' cannot be completed", +- job->id); ++ job->job.id); + return; + } + +@@ -986,7 +986,9 @@ static void mirror_drain(BlockJob *job) + } + + static const BlockJobDriver mirror_job_driver = { +- .instance_size = sizeof(MirrorBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(MirrorBlockJob), ++ }, + .job_type = BLOCK_JOB_TYPE_MIRROR, + .start = mirror_run, + .complete = mirror_complete, +@@ -996,7 +998,9 @@ static const BlockJobDriver mirror_job_driver = { + }; + + static const BlockJobDriver commit_active_job_driver = { +- .instance_size = sizeof(MirrorBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(MirrorBlockJob), ++ }, + .job_type = BLOCK_JOB_TYPE_COMMIT, + .start = mirror_run, + .complete = mirror_complete, +diff --git a/block/stream.c b/block/stream.c +index df9660d..f88fc75 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -209,7 +209,9 @@ out: + } + + static const BlockJobDriver stream_job_driver = { +- .instance_size = sizeof(StreamBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(StreamBlockJob), ++ }, + .job_type = BLOCK_JOB_TYPE_STREAM, + .start = stream_run, + }; +diff --git a/blockjob.c b/blockjob.c +index 112672a..1464856 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -34,7 +34,6 @@ + #include "qapi/qapi-events-block-core.h" + #include "qapi/qmp/qerror.h" + #include "qemu/coroutine.h" +-#include "qemu/id.h" + #include "qemu/timer.h" + + /* Right now, this mutex is only needed to synchronize accesses to job->busy +@@ -92,7 +91,8 @@ static int block_job_apply_verb(BlockJob *job, BlockJobVerb bv, Error **errp) + return 0; + } + error_setg(errp, "Job '%s' in state '%s' cannot accept command verb '%s'", +- job->id, BlockJobStatus_str(job->status), BlockJobVerb_str(bv)); ++ job->job.id, BlockJobStatus_str(job->status), ++ BlockJobVerb_str(bv)); + return -EPERM; + } + +@@ -159,7 +159,7 @@ BlockJob *block_job_get(const char *id) + BlockJob *job; + + QLIST_FOREACH(job, &block_jobs, job_list) { +- if (job->id && !strcmp(id, job->id)) { ++ if (job->job.id && !strcmp(id, job->job.id)) { + return job; + } + } +@@ -261,7 +261,7 @@ void block_job_unref(BlockJob *job) + block_job_detach_aio_context, job); + blk_unref(job->blk); + error_free(job->blocker); +- g_free(job->id); ++ g_free(job->job.id); + assert(!timer_pending(&job->sleep_timer)); + g_free(job); + } +@@ -311,7 +311,7 @@ static char *child_job_get_parent_desc(BdrvChild *c) + BlockJob *job = c->opaque; + return g_strdup_printf("%s job '%s'", + BlockJobType_str(job->driver->job_type), +- job->id); ++ job->job.id); + } + + static void child_job_drained_begin(BdrvChild *c) +@@ -365,7 +365,7 @@ int block_job_add_bdrv(BlockJob *job, const char *name, BlockDriverState *bs, + + bool block_job_is_internal(BlockJob *job) + { +- return (job->id == NULL); ++ return (job->job.id == NULL); + } + + static bool block_job_started(BlockJob *job) +@@ -705,13 +705,13 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) + void block_job_complete(BlockJob *job, Error **errp) + { + /* Should not be reachable via external interface for internal jobs */ +- assert(job->id); ++ assert(job->job.id); + if (block_job_apply_verb(job, BLOCK_JOB_VERB_COMPLETE, errp)) { + return; + } + if (job->pause_count || job->cancelled || !job->driver->complete) { + error_setg(errp, "The active block job '%s' cannot be completed", +- job->id); ++ job->job.id); + return; + } + +@@ -720,7 +720,7 @@ void block_job_complete(BlockJob *job, Error **errp) + + void block_job_finalize(BlockJob *job, Error **errp) + { +- assert(job && job->id); ++ assert(job && job->job.id); + if (block_job_apply_verb(job, BLOCK_JOB_VERB_FINALIZE, errp)) { + return; + } +@@ -731,7 +731,7 @@ void block_job_dismiss(BlockJob **jobptr, Error **errp) + { + BlockJob *job = *jobptr; + /* similarly to _complete, this is QMP-interface only. */ +- assert(job->id); ++ assert(job->job.id); + if (block_job_apply_verb(job, BLOCK_JOB_VERB_DISMISS, errp)) { + return; + } +@@ -848,7 +848,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + } + info = g_new0(BlockJobInfo, 1); + info->type = g_strdup(BlockJobType_str(job->driver->job_type)); +- info->device = g_strdup(job->id); ++ info->device = g_strdup(job->job.id); + info->len = job->len; + info->busy = atomic_read(&job->busy); + info->paused = job->pause_count > 0; +@@ -879,7 +879,7 @@ static void block_job_event_cancelled(BlockJob *job) + } + + qapi_event_send_block_job_cancelled(job->driver->job_type, +- job->id, ++ job->job.id, + job->len, + job->offset, + job->speed, +@@ -893,7 +893,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg) + } + + qapi_event_send_block_job_completed(job->driver->job_type, +- job->id, ++ job->job.id, + job->len, + job->offset, + job->speed, +@@ -907,7 +907,7 @@ static int block_job_event_pending(BlockJob *job) + block_job_state_transition(job, BLOCK_JOB_STATUS_PENDING); + if (!job->auto_finalize && !block_job_is_internal(job)) { + qapi_event_send_block_job_pending(job->driver->job_type, +- job->id, ++ job->job.id, + &error_abort); + } + return 0; +@@ -945,12 +945,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + error_setg(errp, "Cannot specify job ID for internal block job"); + return NULL; + } +- +- if (!id_wellformed(job_id)) { +- error_setg(errp, "Invalid job ID '%s'", job_id); +- return NULL; +- } +- + if (block_job_get(job_id)) { + error_setg(errp, "Job ID '%s' already in use", job_id); + return NULL; +@@ -964,9 +958,13 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return NULL; + } + +- job = g_malloc0(driver->instance_size); ++ job = job_create(job_id, &driver->job_driver, errp); ++ if (job == NULL) { ++ blk_unref(blk); ++ return NULL; ++ } ++ + job->driver = driver; +- job->id = g_strdup(job_id); + job->blk = blk; + job->cb = cb; + job->opaque = opaque; +@@ -1187,7 +1185,7 @@ void block_job_event_ready(BlockJob *job) + } + + qapi_event_send_block_job_ready(job->driver->job_type, +- job->id, ++ job->job.id, + job->len, + job->offset, + job->speed, &error_abort); +@@ -1217,7 +1215,7 @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, + abort(); + } + if (!block_job_is_internal(job)) { +- qapi_event_send_block_job_error(job->id, ++ qapi_event_send_block_job_error(job->job.id, + is_read ? IO_OPERATION_TYPE_READ : + IO_OPERATION_TYPE_WRITE, + action, &error_abort); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 0f56f72..640e649 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -26,6 +26,7 @@ + #ifndef BLOCKJOB_H + #define BLOCKJOB_H + ++#include "qemu/job.h" + #include "block/block.h" + #include "qemu/ratelimit.h" + +@@ -40,6 +41,9 @@ typedef struct BlockJobTxn BlockJobTxn; + * Long-running operation on a BlockDriverState. + */ + typedef struct BlockJob { ++ /** Data belonging to the generic Job infrastructure */ ++ Job job; ++ + /** The job type, including the job vtable. */ + const BlockJobDriver *driver; + +@@ -47,11 +51,6 @@ typedef struct BlockJob { + BlockBackend *blk; + + /** +- * The ID of the block job. May be NULL for internal jobs. +- */ +- char *id; +- +- /** + * The coroutine that executes the job. If not NULL, it is + * reentered when busy is false and the job is cancelled. + */ +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 62ec964..e8eca44 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -35,8 +35,8 @@ + * A class type for block job driver. + */ + struct BlockJobDriver { +- /** Derived BlockJob struct size */ +- size_t instance_size; ++ /** Generic JobDriver callbacks and settings */ ++ JobDriver job_driver; + + /** String describing the operation, part of query-block-jobs QMP API */ + BlockJobType job_type; +diff --git a/include/qemu/job.h b/include/qemu/job.h +new file mode 100644 +index 0000000..b4b49f1 +--- /dev/null ++++ b/include/qemu/job.h +@@ -0,0 +1,60 @@ ++/* ++ * Declarations for background jobs ++ * ++ * Copyright (c) 2011 IBM Corp. ++ * Copyright (c) 2012, 2018 Red Hat, Inc. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ++ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ */ ++ ++#ifndef JOB_H ++#define JOB_H ++ ++typedef struct JobDriver JobDriver; ++ ++/** ++ * Long-running operation. ++ */ ++typedef struct Job { ++ /** The ID of the job. May be NULL for internal jobs. */ ++ char *id; ++ ++ /** The type of this job. */ ++ const JobDriver *driver; ++} Job; ++ ++/** ++ * Callbacks and other information about a Job driver. ++ */ ++struct JobDriver { ++ /** Derived Job struct size */ ++ size_t instance_size; ++}; ++ ++ ++/** ++ * Create a new long-running job and return it. ++ * ++ * @job_id: The id of the newly-created job, or %NULL for internal jobs ++ * @driver: The class object for the newly-created job. ++ * @errp: Error object. ++ */ ++void *job_create(const char *job_id, const JobDriver *driver, Error **errp); ++ ++#endif +diff --git a/job.c b/job.c +new file mode 100644 +index 0000000..87fd484 +--- /dev/null ++++ b/job.c +@@ -0,0 +1,48 @@ ++/* ++ * Background jobs (long-running operations) ++ * ++ * Copyright (c) 2011 IBM Corp. ++ * Copyright (c) 2012, 2018 Red Hat, Inc. ++ * ++ * Permission is hereby granted, free of charge, to any person obtaining a copy ++ * of this software and associated documentation files (the "Software"), to deal ++ * in the Software without restriction, including without limitation the rights ++ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell ++ * copies of the Software, and to permit persons to whom the Software is ++ * furnished to do so, subject to the following conditions: ++ * ++ * The above copyright notice and this permission notice shall be included in ++ * all copies or substantial portions of the Software. ++ * ++ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR ++ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, ++ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL ++ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER ++ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ++ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN ++ * THE SOFTWARE. ++ */ ++ ++#include "qemu/osdep.h" ++#include "qemu-common.h" ++#include "qapi/error.h" ++#include "qemu/job.h" ++#include "qemu/id.h" ++ ++void *job_create(const char *job_id, const JobDriver *driver, Error **errp) ++{ ++ Job *job; ++ ++ if (job_id) { ++ if (!id_wellformed(job_id)) { ++ error_setg(errp, "Invalid job ID '%s'", job_id); ++ return NULL; ++ } ++ } ++ ++ job = g_malloc0(driver->instance_size); ++ job->driver = driver; ++ job->id = g_strdup(job_id); ++ ++ return job; ++} +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 7673de1..fe9f412 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -520,7 +520,9 @@ static void test_job_complete(BlockJob *job, Error **errp) + } + + BlockJobDriver test_job_driver = { +- .instance_size = sizeof(TestBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(TestBlockJob), ++ }, + .start = test_job_start, + .complete = test_job_complete, + }; +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 5789893..48b12d1 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -74,7 +74,9 @@ static void test_block_job_cb(void *opaque, int ret) + } + + static const BlockJobDriver test_block_job_driver = { +- .instance_size = sizeof(TestBlockJob), ++ .job_driver = { ++ .instance_size = sizeof(TestBlockJob), ++ }, + .start = test_block_job_run, + }; + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 8946bfd..b820261 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -17,7 +17,9 @@ + #include "sysemu/block-backend.h" + + static const BlockJobDriver test_block_job_driver = { +- .instance_size = sizeof(BlockJob), ++ .job_driver = { ++ .instance_size = sizeof(BlockJob), ++ }, + }; + + static void block_job_cb(void *opaque, int ret) +@@ -38,9 +40,9 @@ static BlockJob *mk_job(BlockBackend *blk, const char *id, + g_assert_null(errp); + g_assert_nonnull(job); + if (id) { +- g_assert_cmpstr(job->id, ==, id); ++ g_assert_cmpstr(job->job.id, ==, id); + } else { +- g_assert_cmpstr(job->id, ==, blk_name(blk)); ++ g_assert_cmpstr(job->job.id, ==, blk_name(blk)); + } + } else { + g_assert_nonnull(errp); +@@ -192,7 +194,9 @@ static void coroutine_fn cancel_job_start(void *opaque) + } + + static const BlockJobDriver test_cancel_driver = { +- .instance_size = sizeof(CancelJob), ++ .job_driver = { ++ .instance_size = sizeof(CancelJob), ++ }, + .start = cancel_job_start, + .complete = cancel_job_complete, + }; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Fix-missing-locking-due-to-mismerge.patch b/SOURCES/kvm-job-Fix-missing-locking-due-to-mismerge.patch new file mode 100644 index 0000000..992e8c0 --- /dev/null +++ b/SOURCES/kvm-job-Fix-missing-locking-due-to-mismerge.patch @@ -0,0 +1,55 @@ +From f842ac5501037ccea54e3b6c8a7a2471b30aaea7 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:55 +0100 +Subject: [PATCH 29/49] job: Fix missing locking due to mismerge + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-17-kwolf@redhat.com> +Patchwork-id: 82607 +O-Subject: [RHEL-8 qemu-kvm PATCH 26/44] job: Fix missing locking due to mismerge +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +job_completed() had a problem with double locking that was recently +fixed independently by two different commits: + +"job: Fix nested aio_poll() hanging in job_txn_apply" +"jobs: add exit shim" + +One fix removed the first aio_context_acquire(), the other fix removed +the other one. Now we have a bug again and the code is run without any +locking. + +Add it back in one of the places. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit d1756c780b7879fb64e41135feac781d84a1f995) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + job.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/job.c b/job.c +index 5d117fb..db53163 100644 +--- a/job.c ++++ b/job.c +@@ -842,7 +842,11 @@ static void job_completed(Job *job) + static void job_exit(void *opaque) + { + Job *job = (Job *)opaque; ++ AioContext *ctx = job->aio_context; ++ ++ aio_context_acquire(ctx); + job_completed(job); ++ aio_context_release(ctx); + } + + /** +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Fix-nested-aio_poll-hanging-in-job_txn_apply.patch b/SOURCES/kvm-job-Fix-nested-aio_poll-hanging-in-job_txn_apply.patch new file mode 100644 index 0000000..823b505 --- /dev/null +++ b/SOURCES/kvm-job-Fix-nested-aio_poll-hanging-in-job_txn_apply.patch @@ -0,0 +1,105 @@ +From bdc8bf786dcd258488ffd64fa37ecb0e801141ce Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:54 +0100 +Subject: [PATCH 28/49] job: Fix nested aio_poll() hanging in job_txn_apply + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-16-kwolf@redhat.com> +Patchwork-id: 82605 +O-Subject: [RHEL-8 qemu-kvm PATCH 25/44] job: Fix nested aio_poll() hanging in job_txn_apply +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +From: Fam Zheng + +All callers have acquired ctx already. Doing that again results in +aio_poll() hang. This fixes the problem that a BDRV_POLL_WHILE() in the +callback cannot make progress because ctx is recursively locked, for +example, when drive-backup finishes. + +There are two callers of job_finalize(): + + fam@lemon:~/work/qemu [master]$ git grep -w -A1 '^\s*job_finalize' + blockdev.c: job_finalize(&job->job, errp); + blockdev.c- aio_context_release(aio_context); + -- + job-qmp.c: job_finalize(job, errp); + job-qmp.c- aio_context_release(aio_context); + -- + tests/test-blockjob.c: job_finalize(&job->job, &error_abort); + tests/test-blockjob.c- assert(job->job.status == JOB_STATUS_CONCLUDED); + +Ignoring the test, it's easy to see both callers to job_finalize (and +job_do_finalize) have acquired the context. + +Cc: qemu-stable@nongnu.org +Reported-by: Gu Nini +Reviewed-by: Eric Blake +Signed-off-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit 49880165a44f26dc84651858750facdee31f2513) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + job.c | 18 +++++------------- + 1 file changed, 5 insertions(+), 13 deletions(-) + +diff --git a/job.c b/job.c +index dfba4bc..5d117fb 100644 +--- a/job.c ++++ b/job.c +@@ -136,21 +136,13 @@ static void job_txn_del_job(Job *job) + } + } + +-static int job_txn_apply(JobTxn *txn, int fn(Job *), bool lock) ++static int job_txn_apply(JobTxn *txn, int fn(Job *)) + { +- AioContext *ctx; + Job *job, *next; + int rc = 0; + + QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { +- if (lock) { +- ctx = job->aio_context; +- aio_context_acquire(ctx); +- } + rc = fn(job); +- if (lock) { +- aio_context_release(ctx); +- } + if (rc) { + break; + } +@@ -775,11 +767,11 @@ static void job_do_finalize(Job *job) + assert(job && job->txn); + + /* prepare the transaction to complete */ +- rc = job_txn_apply(job->txn, job_prepare, true); ++ rc = job_txn_apply(job->txn, job_prepare); + if (rc) { + job_completed_txn_abort(job); + } else { +- job_txn_apply(job->txn, job_finalize_single, true); ++ job_txn_apply(job->txn, job_finalize_single); + } + } + +@@ -825,10 +817,10 @@ static void job_completed_txn_success(Job *job) + assert(other_job->ret == 0); + } + +- job_txn_apply(txn, job_transition_to_pending, false); ++ job_txn_apply(txn, job_transition_to_pending); + + /* If no jobs need manual finalization, automatically do so */ +- if (job_txn_apply(txn, job_needs_finalize, false) == 0) { ++ if (job_txn_apply(txn, job_needs_finalize) == 0) { + job_do_finalize(job); + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Introduce-qapi-job.json.patch b/SOURCES/kvm-job-Introduce-qapi-job.json.patch new file mode 100644 index 0000000..c2adafd --- /dev/null +++ b/SOURCES/kvm-job-Introduce-qapi-job.json.patch @@ -0,0 +1,381 @@ +From 379a26a4794704679a8df09fdf046f60f0abea3b Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:30 +0200 +Subject: [PATCH 122/268] job: Introduce qapi/job.json + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-48-kwolf@redhat.com> +Patchwork-id: 81099 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 47/73] job: Introduce qapi/job.json +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a separate schema file for all job-related definitions that +aren't tied to the block layer. + +For a start, move the enums JobType, JobStatus and JobVerb. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit bf42508f24ee1368267b421e145fa90315b77936) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + MAINTAINERS | 1 + + Makefile | 9 +++++ + Makefile.objs | 4 +++ + qapi/block-core.json | 90 +----------------------------------------------- + qapi/job.json | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++ + qapi/qapi-schema.json | 1 + + 6 files changed, 110 insertions(+), 89 deletions(-) + create mode 100644 qapi/job.json + +diff --git a/MAINTAINERS b/MAINTAINERS +index c5e3dfb..5aaf264 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1377,6 +1377,7 @@ F: block/backup.c + F: block/commit.c + F: block/stream.c + F: block/mirror.c ++F: qapi/job.json + T: git git://github.com/codyprime/qemu-kvm-jtc.git block + + Block QAPI, monitor, command line +diff --git a/Makefile b/Makefile +index d0a848e..9803f27 100644 +--- a/Makefile ++++ b/Makefile +@@ -98,6 +98,7 @@ GENERATED_FILES += qapi/qapi-types-char.h qapi/qapi-types-char.c + GENERATED_FILES += qapi/qapi-types-common.h qapi/qapi-types-common.c + GENERATED_FILES += qapi/qapi-types-crypto.h qapi/qapi-types-crypto.c + GENERATED_FILES += qapi/qapi-types-introspect.h qapi/qapi-types-introspect.c ++GENERATED_FILES += qapi/qapi-types-job.h qapi/qapi-types-job.c + GENERATED_FILES += qapi/qapi-types-migration.h qapi/qapi-types-migration.c + GENERATED_FILES += qapi/qapi-types-misc.h qapi/qapi-types-misc.c + GENERATED_FILES += qapi/qapi-types-net.h qapi/qapi-types-net.c +@@ -116,6 +117,7 @@ GENERATED_FILES += qapi/qapi-visit-char.h qapi/qapi-visit-char.c + GENERATED_FILES += qapi/qapi-visit-common.h qapi/qapi-visit-common.c + GENERATED_FILES += qapi/qapi-visit-crypto.h qapi/qapi-visit-crypto.c + GENERATED_FILES += qapi/qapi-visit-introspect.h qapi/qapi-visit-introspect.c ++GENERATED_FILES += qapi/qapi-visit-job.h qapi/qapi-visit-job.c + GENERATED_FILES += qapi/qapi-visit-migration.h qapi/qapi-visit-migration.c + GENERATED_FILES += qapi/qapi-visit-misc.h qapi/qapi-visit-misc.c + GENERATED_FILES += qapi/qapi-visit-net.h qapi/qapi-visit-net.c +@@ -133,6 +135,7 @@ GENERATED_FILES += qapi/qapi-commands-char.h qapi/qapi-commands-char.c + GENERATED_FILES += qapi/qapi-commands-common.h qapi/qapi-commands-common.c + GENERATED_FILES += qapi/qapi-commands-crypto.h qapi/qapi-commands-crypto.c + GENERATED_FILES += qapi/qapi-commands-introspect.h qapi/qapi-commands-introspect.c ++GENERATED_FILES += qapi/qapi-commands-job.h qapi/qapi-commands-job.c + GENERATED_FILES += qapi/qapi-commands-migration.h qapi/qapi-commands-migration.c + GENERATED_FILES += qapi/qapi-commands-misc.h qapi/qapi-commands-misc.c + GENERATED_FILES += qapi/qapi-commands-net.h qapi/qapi-commands-net.c +@@ -150,6 +153,7 @@ GENERATED_FILES += qapi/qapi-events-char.h qapi/qapi-events-char.c + GENERATED_FILES += qapi/qapi-events-common.h qapi/qapi-events-common.c + GENERATED_FILES += qapi/qapi-events-crypto.h qapi/qapi-events-crypto.c + GENERATED_FILES += qapi/qapi-events-introspect.h qapi/qapi-events-introspect.c ++GENERATED_FILES += qapi/qapi-events-job.h qapi/qapi-events-job.c + GENERATED_FILES += qapi/qapi-events-migration.h qapi/qapi-events-migration.c + GENERATED_FILES += qapi/qapi-events-misc.h qapi/qapi-events-misc.c + GENERATED_FILES += qapi/qapi-events-net.h qapi/qapi-events-net.c +@@ -582,6 +586,7 @@ qapi-modules = $(SRC_PATH)/qapi/qapi-schema.json $(SRC_PATH)/qapi/common.json \ + $(SRC_PATH)/qapi/char.json \ + $(SRC_PATH)/qapi/crypto.json \ + $(SRC_PATH)/qapi/introspect.json \ ++ $(SRC_PATH)/qapi/job.json \ + $(SRC_PATH)/qapi/migration.json \ + $(SRC_PATH)/qapi/misc.json \ + $(SRC_PATH)/qapi/net.json \ +@@ -601,6 +606,7 @@ qapi/qapi-types-char.c qapi/qapi-types-char.h \ + qapi/qapi-types-common.c qapi/qapi-types-common.h \ + qapi/qapi-types-crypto.c qapi/qapi-types-crypto.h \ + qapi/qapi-types-introspect.c qapi/qapi-types-introspect.h \ ++qapi/qapi-types-job.c qapi/qapi-types-job.h \ + qapi/qapi-types-migration.c qapi/qapi-types-migration.h \ + qapi/qapi-types-misc.c qapi/qapi-types-misc.h \ + qapi/qapi-types-net.c qapi/qapi-types-net.h \ +@@ -619,6 +625,7 @@ qapi/qapi-visit-char.c qapi/qapi-visit-char.h \ + qapi/qapi-visit-common.c qapi/qapi-visit-common.h \ + qapi/qapi-visit-crypto.c qapi/qapi-visit-crypto.h \ + qapi/qapi-visit-introspect.c qapi/qapi-visit-introspect.h \ ++qapi/qapi-visit-job.c qapi/qapi-visit-job.h \ + qapi/qapi-visit-migration.c qapi/qapi-visit-migration.h \ + qapi/qapi-visit-misc.c qapi/qapi-visit-misc.h \ + qapi/qapi-visit-net.c qapi/qapi-visit-net.h \ +@@ -636,6 +643,7 @@ qapi/qapi-commands-char.c qapi/qapi-commands-char.h \ + qapi/qapi-commands-common.c qapi/qapi-commands-common.h \ + qapi/qapi-commands-crypto.c qapi/qapi-commands-crypto.h \ + qapi/qapi-commands-introspect.c qapi/qapi-commands-introspect.h \ ++qapi/qapi-commands-job.c qapi/qapi-commands-job.h \ + qapi/qapi-commands-migration.c qapi/qapi-commands-migration.h \ + qapi/qapi-commands-misc.c qapi/qapi-commands-misc.h \ + qapi/qapi-commands-net.c qapi/qapi-commands-net.h \ +@@ -653,6 +661,7 @@ qapi/qapi-events-char.c qapi/qapi-events-char.h \ + qapi/qapi-events-common.c qapi/qapi-events-common.h \ + qapi/qapi-events-crypto.c qapi/qapi-events-crypto.h \ + qapi/qapi-events-introspect.c qapi/qapi-events-introspect.h \ ++qapi/qapi-events-job.c qapi/qapi-events-job.h \ + qapi/qapi-events-migration.c qapi/qapi-events-migration.h \ + qapi/qapi-events-misc.c qapi/qapi-events-misc.h \ + qapi/qapi-events-net.c qapi/qapi-events-net.h \ +diff --git a/Makefile.objs b/Makefile.objs +index 92b73fc..3df8d58 100644 +--- a/Makefile.objs ++++ b/Makefile.objs +@@ -10,6 +10,7 @@ util-obj-y += qapi/qapi-types-char.o + util-obj-y += qapi/qapi-types-common.o + util-obj-y += qapi/qapi-types-crypto.o + util-obj-y += qapi/qapi-types-introspect.o ++util-obj-y += qapi/qapi-types-job.o + util-obj-y += qapi/qapi-types-migration.o + util-obj-y += qapi/qapi-types-misc.o + util-obj-y += qapi/qapi-types-net.o +@@ -28,6 +29,7 @@ util-obj-y += qapi/qapi-visit-char.o + util-obj-y += qapi/qapi-visit-common.o + util-obj-y += qapi/qapi-visit-crypto.o + util-obj-y += qapi/qapi-visit-introspect.o ++util-obj-y += qapi/qapi-visit-job.o + util-obj-y += qapi/qapi-visit-migration.o + util-obj-y += qapi/qapi-visit-misc.o + util-obj-y += qapi/qapi-visit-net.o +@@ -45,6 +47,7 @@ util-obj-y += qapi/qapi-events-char.o + util-obj-y += qapi/qapi-events-common.o + util-obj-y += qapi/qapi-events-crypto.o + util-obj-y += qapi/qapi-events-introspect.o ++util-obj-y += qapi/qapi-events-job.o + util-obj-y += qapi/qapi-events-migration.o + util-obj-y += qapi/qapi-events-misc.o + util-obj-y += qapi/qapi-events-net.o +@@ -140,6 +143,7 @@ common-obj-y += qapi/qapi-commands-char.o + common-obj-y += qapi/qapi-commands-common.o + common-obj-y += qapi/qapi-commands-crypto.o + common-obj-y += qapi/qapi-commands-introspect.o ++common-obj-y += qapi/qapi-commands-job.o + common-obj-y += qapi/qapi-commands-migration.o + common-obj-y += qapi/qapi-commands-misc.o + common-obj-y += qapi/qapi-commands-net.o +diff --git a/qapi/block-core.json b/qapi/block-core.json +index f2ed7a8..96ddf87 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -6,6 +6,7 @@ + + { 'include': 'common.json' } + { 'include': 'crypto.json' } ++{ 'include': 'job.json' } + { 'include': 'sockets.json' } + + ## +@@ -1050,95 +1051,6 @@ + 'data': ['top', 'full', 'none', 'incremental'] } + + ## +-# @JobType: +-# +-# Type of a background job. +-# +-# @commit: block commit job type, see "block-commit" +-# +-# @stream: block stream job type, see "block-stream" +-# +-# @mirror: drive mirror job type, see "drive-mirror" +-# +-# @backup: drive backup job type, see "drive-backup" +-# +-# Since: 1.7 +-## +-{ 'enum': 'JobType', +- 'data': ['commit', 'stream', 'mirror', 'backup'] } +- +-## +-# @JobVerb: +-# +-# Represents command verbs that can be applied to a job. +-# +-# @cancel: see @block-job-cancel +-# +-# @pause: see @block-job-pause +-# +-# @resume: see @block-job-resume +-# +-# @set-speed: see @block-job-set-speed +-# +-# @complete: see @block-job-complete +-# +-# @dismiss: see @block-job-dismiss +-# +-# @finalize: see @block-job-finalize +-# +-# Since: 2.12 +-## +-{ 'enum': 'JobVerb', +- 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', +- 'finalize' ] } +- +-## +-# @JobStatus: +-# +-# Indicates the present state of a given job in its lifetime. +-# +-# @undefined: Erroneous, default state. Should not ever be visible. +-# +-# @created: The job has been created, but not yet started. +-# +-# @running: The job is currently running. +-# +-# @paused: The job is running, but paused. The pause may be requested by +-# either the QMP user or by internal processes. +-# +-# @ready: The job is running, but is ready for the user to signal completion. +-# This is used for long-running jobs like mirror that are designed to +-# run indefinitely. +-# +-# @standby: The job is ready, but paused. This is nearly identical to @paused. +-# The job may return to @ready or otherwise be canceled. +-# +-# @waiting: The job is waiting for other jobs in the transaction to converge +-# to the waiting state. This status will likely not be visible for +-# the last job in a transaction. +-# +-# @pending: The job has finished its work, but has finalization steps that it +-# needs to make prior to completing. These changes may require +-# manual intervention by the management process if manual was set +-# to true. These changes may still fail. +-# +-# @aborting: The job is in the process of being aborted, and will finish with +-# an error. The job will afterwards report that it is @concluded. +-# This status may not be visible to the management process. +-# +-# @concluded: The job has finished all work. If manual was set to true, the job +-# will remain in the query list until it is dismissed. +-# +-# @null: The job is in the process of being dismantled. This state should not +-# ever be visible externally. +-# +-# Since: 2.12 +-## +-{ 'enum': 'JobStatus', +- 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', +- 'waiting', 'pending', 'aborting', 'concluded', 'null' ] } +- +-## + # @BlockJobInfo: + # + # Information about a long-running block device operation. +diff --git a/qapi/job.json b/qapi/job.json +new file mode 100644 +index 0000000..a472c0c +--- /dev/null ++++ b/qapi/job.json +@@ -0,0 +1,94 @@ ++# -*- Mode: Python -*- ++ ++## ++# == Background jobs ++## ++ ++## ++# @JobType: ++# ++# Type of a background job. ++# ++# @commit: block commit job type, see "block-commit" ++# ++# @stream: block stream job type, see "block-stream" ++# ++# @mirror: drive mirror job type, see "drive-mirror" ++# ++# @backup: drive backup job type, see "drive-backup" ++# ++# Since: 1.7 ++## ++{ 'enum': 'JobType', ++ 'data': ['commit', 'stream', 'mirror', 'backup'] } ++ ++## ++# @JobStatus: ++# ++# Indicates the present state of a given job in its lifetime. ++# ++# @undefined: Erroneous, default state. Should not ever be visible. ++# ++# @created: The job has been created, but not yet started. ++# ++# @running: The job is currently running. ++# ++# @paused: The job is running, but paused. The pause may be requested by ++# either the QMP user or by internal processes. ++# ++# @ready: The job is running, but is ready for the user to signal completion. ++# This is used for long-running jobs like mirror that are designed to ++# run indefinitely. ++# ++# @standby: The job is ready, but paused. This is nearly identical to @paused. ++# The job may return to @ready or otherwise be canceled. ++# ++# @waiting: The job is waiting for other jobs in the transaction to converge ++# to the waiting state. This status will likely not be visible for ++# the last job in a transaction. ++# ++# @pending: The job has finished its work, but has finalization steps that it ++# needs to make prior to completing. These changes may require ++# manual intervention by the management process if manual was set ++# to true. These changes may still fail. ++# ++# @aborting: The job is in the process of being aborted, and will finish with ++# an error. The job will afterwards report that it is @concluded. ++# This status may not be visible to the management process. ++# ++# @concluded: The job has finished all work. If manual was set to true, the job ++# will remain in the query list until it is dismissed. ++# ++# @null: The job is in the process of being dismantled. This state should not ++# ever be visible externally. ++# ++# Since: 2.12 ++## ++{ 'enum': 'JobStatus', ++ 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', ++ 'waiting', 'pending', 'aborting', 'concluded', 'null' ] } ++ ++## ++# @JobVerb: ++# ++# Represents command verbs that can be applied to a job. ++# ++# @cancel: see @block-job-cancel ++# ++# @pause: see @block-job-pause ++# ++# @resume: see @block-job-resume ++# ++# @set-speed: see @block-job-set-speed ++# ++# @complete: see @block-job-complete ++# ++# @dismiss: see @block-job-dismiss ++# ++# @finalize: see @block-job-finalize ++# ++# Since: 2.12 ++## ++{ 'enum': 'JobVerb', ++ 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', ++ 'finalize' ] } +diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json +index 25bce78..65b6dc2 100644 +--- a/qapi/qapi-schema.json ++++ b/qapi/qapi-schema.json +@@ -84,6 +84,7 @@ + { 'include': 'crypto.json' } + { 'include': 'block.json' } + { 'include': 'char.json' } ++{ 'include': 'job.json' } + { 'include': 'net.json' } + { 'include': 'rocker.json' } + { 'include': 'tpm.json' } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Maintain-a-list-of-all-jobs.patch b/SOURCES/kvm-job-Maintain-a-list-of-all-jobs.patch new file mode 100644 index 0000000..b293079 --- /dev/null +++ b/SOURCES/kvm-job-Maintain-a-list-of-all-jobs.patch @@ -0,0 +1,262 @@ +From 04abeb0ef2f62cf5eb252672afef623d9cc99545 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:03 +0200 +Subject: [PATCH 095/268] job: Maintain a list of all jobs + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-21-kwolf@redhat.com> +Patchwork-id: 81105 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 20/73] job: Maintain a list of all jobs +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves the job list from BlockJob to Job. Now we can check for +duplicate IDs in job_create(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit e7c1d78bbd5867804debeb7159b137fd9a6c44d3) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 46 ++++++++++++++++++++++++---------------------- + include/block/blockjob.h | 3 --- + include/qemu/job.h | 19 +++++++++++++++++++ + job.c | 31 +++++++++++++++++++++++++++++++ + 4 files changed, 74 insertions(+), 25 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 430a67b..c69b2e7 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -129,8 +129,6 @@ struct BlockJobTxn { + int refcnt; + }; + +-static QLIST_HEAD(, BlockJob) block_jobs = QLIST_HEAD_INITIALIZER(block_jobs); +- + /* + * The block job API is composed of two categories of functions. + * +@@ -146,25 +144,34 @@ static QLIST_HEAD(, BlockJob) block_jobs = QLIST_HEAD_INITIALIZER(block_jobs); + * blockjob_int.h. + */ + +-BlockJob *block_job_next(BlockJob *job) ++static bool is_block_job(Job *job) + { +- if (!job) { +- return QLIST_FIRST(&block_jobs); +- } +- return QLIST_NEXT(job, job_list); ++ return job_type(job) == JOB_TYPE_BACKUP || ++ job_type(job) == JOB_TYPE_COMMIT || ++ job_type(job) == JOB_TYPE_MIRROR || ++ job_type(job) == JOB_TYPE_STREAM; ++} ++ ++BlockJob *block_job_next(BlockJob *bjob) ++{ ++ Job *job = bjob ? &bjob->job : NULL; ++ ++ do { ++ job = job_next(job); ++ } while (job && !is_block_job(job)); ++ ++ return job ? container_of(job, BlockJob, job) : NULL; + } + + BlockJob *block_job_get(const char *id) + { +- BlockJob *job; ++ Job *job = job_get(id); + +- QLIST_FOREACH(job, &block_jobs, job_list) { +- if (job->job.id && !strcmp(id, job->job.id)) { +- return job; +- } ++ if (job && is_block_job(job)) { ++ return container_of(job, BlockJob, job); ++ } else { ++ return NULL; + } +- +- return NULL; + } + + BlockJobTxn *block_job_txn_new(void) +@@ -253,7 +260,6 @@ void block_job_unref(BlockJob *job) + assert(job->status == BLOCK_JOB_STATUS_NULL); + assert(!job->txn); + BlockDriverState *bs = blk_bs(job->blk); +- QLIST_REMOVE(job, job_list); + bs->job = NULL; + block_job_remove_all_bdrv(job); + blk_remove_aio_context_notifier(job->blk, +@@ -812,7 +818,7 @@ void block_job_cancel_sync_all(void) + BlockJob *job; + AioContext *aio_context; + +- while ((job = QLIST_FIRST(&block_jobs))) { ++ while ((job = block_job_next(NULL))) { + aio_context = blk_get_aio_context(job->blk); + aio_context_acquire(aio_context); + block_job_cancel_sync(job); +@@ -942,10 +948,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + error_setg(errp, "Cannot specify job ID for internal block job"); + return NULL; + } +- if (block_job_get(job_id)) { +- error_setg(errp, "Job ID '%s' already in use", job_id); +- return NULL; +- } + } + + blk = blk_new(perm, shared_perm); +@@ -961,6 +963,8 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return NULL; + } + ++ assert(is_block_job(&job->job)); ++ + job->driver = driver; + job->blk = blk; + job->cb = cb; +@@ -983,8 +987,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + + bdrv_op_unblock(bs, BLOCK_OP_TYPE_DATAPLANE, job->blocker); + +- QLIST_INSERT_HEAD(&block_jobs, job, job_list); +- + blk_add_aio_context_notifier(blk, block_job_attached_aio_context, + block_job_detach_aio_context, job); + +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 640e649..10bd9f7 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -105,9 +105,6 @@ typedef struct BlockJob { + */ + bool deferred_to_main_loop; + +- /** Element of the list of block jobs */ +- QLIST_ENTRY(BlockJob) job_list; +- + /** Status that is published by the query-block-jobs QMP API */ + BlockDeviceIoStatus iostatus; + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 43dc2e4..bae2b09 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -27,6 +27,7 @@ + #define JOB_H + + #include "qapi/qapi-types-block-core.h" ++#include "qemu/queue.h" + + typedef struct JobDriver JobDriver; + +@@ -39,6 +40,9 @@ typedef struct Job { + + /** The type of this job. */ + const JobDriver *driver; ++ ++ /** Element of the list of jobs */ ++ QLIST_ENTRY(Job) job_list; + } Job; + + /** +@@ -71,4 +75,19 @@ JobType job_type(const Job *job); + /** Returns the enum string for the JobType of a given Job. */ + const char *job_type_str(const Job *job); + ++/** ++ * Get the next element from the list of block jobs after @job, or the ++ * first one if @job is %NULL. ++ * ++ * Returns the requested job, or %NULL if there are no more jobs left. ++ */ ++Job *job_next(Job *job); ++ ++/** ++ * Get the job identified by @id (which must not be %NULL). ++ * ++ * Returns the requested job, or %NULL if it doesn't exist. ++ */ ++Job *job_get(const char *id); ++ + #endif +diff --git a/job.c b/job.c +index cfdd008..e57303c 100644 +--- a/job.c ++++ b/job.c +@@ -29,6 +29,8 @@ + #include "qemu/job.h" + #include "qemu/id.h" + ++static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); ++ + JobType job_type(const Job *job) + { + return job->driver->job_type; +@@ -39,6 +41,27 @@ const char *job_type_str(const Job *job) + return JobType_str(job_type(job)); + } + ++Job *job_next(Job *job) ++{ ++ if (!job) { ++ return QLIST_FIRST(&jobs); ++ } ++ return QLIST_NEXT(job, job_list); ++} ++ ++Job *job_get(const char *id) ++{ ++ Job *job; ++ ++ QLIST_FOREACH(job, &jobs, job_list) { ++ if (job->id && !strcmp(id, job->id)) { ++ return job; ++ } ++ } ++ ++ return NULL; ++} ++ + void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + { + Job *job; +@@ -48,17 +71,25 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + error_setg(errp, "Invalid job ID '%s'", job_id); + return NULL; + } ++ if (job_get(job_id)) { ++ error_setg(errp, "Job ID '%s' already in use", job_id); ++ return NULL; ++ } + } + + job = g_malloc0(driver->instance_size); + job->driver = driver; + job->id = g_strdup(job_id); + ++ QLIST_INSERT_HEAD(&jobs, job, job_list); ++ + return job; + } + + void job_delete(Job *job) + { ++ QLIST_REMOVE(job, job_list); ++ + g_free(job->id); + g_free(job); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-.complete-callback-to-Job.patch b/SOURCES/kvm-job-Move-.complete-callback-to-Job.patch new file mode 100644 index 0000000..2885011 --- /dev/null +++ b/SOURCES/kvm-job-Move-.complete-callback-to-Job.patch @@ -0,0 +1,298 @@ +From 02727b3014f58b63b589176a191a0ced5def4a65 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:19 +0200 +Subject: [PATCH 111/268] job: Move .complete callback to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-37-kwolf@redhat.com> +Patchwork-id: 81084 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 36/73] job: Move .complete callback to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves the .complete callback that tells a READY job to complete +from BlockJobDriver to JobDriver. The wrapper function job_complete() +doesn't require anything block job specific any more and can be moved +to Job. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 3453d97243c72988c89a0105fa9546890eae7bd4) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/mirror.c | 10 +++++----- + blockdev.c | 2 +- + blockjob.c | 23 +++++------------------ + include/block/blockjob.h | 10 ---------- + include/block/blockjob_int.h | 6 ------ + include/qemu/job.h | 8 ++++++++ + job.c | 16 ++++++++++++++++ + tests/test-bdrv-drain.c | 6 +++--- + tests/test-blockjob.c | 10 +++++----- + 9 files changed, 43 insertions(+), 48 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index a579bd8..656237a 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -905,16 +905,16 @@ immediate_exit: + job_defer_to_main_loop(&s->common.job, mirror_exit, data); + } + +-static void mirror_complete(BlockJob *job, Error **errp) ++static void mirror_complete(Job *job, Error **errp) + { +- MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); ++ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); + BlockDriverState *target; + + target = blk_bs(s->target); + + if (!s->synced) { + error_setg(errp, "The active block job '%s' cannot be completed", +- job->job.id); ++ job->id); + return; + } + +@@ -995,8 +995,8 @@ static const BlockJobDriver mirror_job_driver = { + .drain = block_job_drain, + .start = mirror_run, + .pause = mirror_pause, ++ .complete = mirror_complete, + }, +- .complete = mirror_complete, + .attached_aio_context = mirror_attached_aio_context, + .drain = mirror_drain, + }; +@@ -1010,8 +1010,8 @@ static const BlockJobDriver commit_active_job_driver = { + .drain = block_job_drain, + .start = mirror_run, + .pause = mirror_pause, ++ .complete = mirror_complete, + }, +- .complete = mirror_complete, + .attached_aio_context = mirror_attached_aio_context, + .drain = mirror_drain, + }; +diff --git a/blockdev.c b/blockdev.c +index 24deaf1..dd9a080 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3895,7 +3895,7 @@ void qmp_block_job_complete(const char *device, Error **errp) + } + + trace_qmp_block_job_complete(job); +- block_job_complete(job, errp); ++ job_complete(&job->job, errp); + aio_context_release(aio_context); + } + +diff --git a/blockjob.c b/blockjob.c +index 63e1669..0ca7672 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -481,24 +481,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) + return ratelimit_calculate_delay(&job->limit, n); + } + +-void block_job_complete(BlockJob *job, Error **errp) +-{ +- /* Should not be reachable via external interface for internal jobs */ +- assert(job->job.id); +- if (job_apply_verb(&job->job, JOB_VERB_COMPLETE, errp)) { +- return; +- } +- if (job->job.pause_count || job_is_cancelled(&job->job) || +- !job->driver->complete) +- { +- error_setg(errp, "The active block job '%s' cannot be completed", +- job->job.id); +- return; +- } +- +- job->driver->complete(job, errp); +-} +- + void block_job_finalize(BlockJob *job, Error **errp) + { + assert(job && job->job.id); +@@ -571,6 +553,11 @@ void block_job_cancel_sync_all(void) + } + } + ++static void block_job_complete(BlockJob *job, Error **errp) ++{ ++ job_complete(&job->job, errp); ++} ++ + int block_job_complete_sync(BlockJob *job, Error **errp) + { + return block_job_finish_sync(job, &block_job_complete, errp); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index d975efe..85ce18a 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -154,16 +154,6 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); + void block_job_cancel(BlockJob *job, bool force); + + /** +- * block_job_complete: +- * @job: The job to be completed. +- * @errp: Error object. +- * +- * Asynchronously complete the specified job. +- */ +-void block_job_complete(BlockJob *job, Error **errp); +- +- +-/** + * block_job_finalize: + * @job: The job to fully commit and finish. + * @errp: Error object. +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 38fe22d..b8ca7bb 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -39,12 +39,6 @@ struct BlockJobDriver { + JobDriver job_driver; + + /** +- * Optional callback for job types whose completion must be triggered +- * manually. +- */ +- void (*complete)(BlockJob *job, Error **errp); +- +- /** + * If the callback is not NULL, prepare will be invoked when all the jobs + * belonging to the same transaction complete; or upon this job's completion + * if it is not in a transaction. +diff --git a/include/qemu/job.h b/include/qemu/job.h +index aebc195..8f7f71a 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -167,6 +167,12 @@ struct JobDriver { + */ + void (*user_resume)(Job *job); + ++ /** ++ * Optional callback for job types whose completion must be triggered ++ * manually. ++ */ ++ void (*complete)(Job *job, Error **errp); ++ + /* + * If the callback is not NULL, it will be invoked when the job has to be + * synchronously cancelled or completed; it should drain any activities +@@ -363,6 +369,8 @@ int job_apply_verb(Job *job, JobVerb verb, Error **errp); + /** The @job could not be started, free it. */ + void job_early_fail(Job *job); + ++/** Asynchronously complete the specified @job. */ ++void job_complete(Job *job, Error **errp);; + + typedef void JobDeferToMainLoopFn(Job *job, void *opaque); + +diff --git a/job.c b/job.c +index 3772a35..8ceac0b 100644 +--- a/job.c ++++ b/job.c +@@ -556,6 +556,22 @@ int job_finalize_single(Job *job) + return 0; + } + ++void job_complete(Job *job, Error **errp) ++{ ++ /* Should not be reachable via external interface for internal jobs */ ++ assert(job->id); ++ if (job_apply_verb(job, JOB_VERB_COMPLETE, errp)) { ++ return; ++ } ++ if (job->pause_count || job_is_cancelled(job) || !job->driver->complete) { ++ error_setg(errp, "The active block job '%s' cannot be completed", ++ job->id); ++ return; ++ } ++ ++ job->driver->complete(job, errp); ++} ++ + + typedef struct { + Job *job; +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 58ea566..b428aac 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -514,9 +514,9 @@ static void coroutine_fn test_job_start(void *opaque) + job_defer_to_main_loop(&s->common.job, test_job_completed, NULL); + } + +-static void test_job_complete(BlockJob *job, Error **errp) ++static void test_job_complete(Job *job, Error **errp) + { +- TestBlockJob *s = container_of(job, TestBlockJob, common); ++ TestBlockJob *s = container_of(job, TestBlockJob, common.job); + s->should_complete = true; + } + +@@ -527,8 +527,8 @@ BlockJobDriver test_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .start = test_job_start, ++ .complete = test_job_complete, + }, +- .complete = test_job_complete, + }; + + static void test_blockjob_common(enum drain_type drain_type) +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 592a136..e44c608 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -171,9 +171,9 @@ static void cancel_job_completed(Job *job, void *opaque) + block_job_completed(bjob, 0); + } + +-static void cancel_job_complete(BlockJob *job, Error **errp) ++static void cancel_job_complete(Job *job, Error **errp) + { +- CancelJob *s = container_of(job, CancelJob, common); ++ CancelJob *s = container_of(job, CancelJob, common.job); + s->should_complete = true; + } + +@@ -204,8 +204,8 @@ static const BlockJobDriver test_cancel_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .start = cancel_job_start, ++ .complete = cancel_job_complete, + }, +- .complete = cancel_job_complete, + }; + + static CancelJob *create_common(BlockJob **pjob) +@@ -333,7 +333,7 @@ static void test_cancel_pending(void) + block_job_enter(job); + assert(job->job.status == JOB_STATUS_READY); + +- block_job_complete(job, &error_abort); ++ job_complete(&job->job, &error_abort); + block_job_enter(job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); +@@ -357,7 +357,7 @@ static void test_cancel_concluded(void) + block_job_enter(job); + assert(job->job.status == JOB_STATUS_READY); + +- block_job_complete(job, &error_abort); ++ job_complete(&job->job, &error_abort); + block_job_enter(job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-BlockJobCreateFlags-to-Job.patch b/SOURCES/kvm-job-Move-BlockJobCreateFlags-to-Job.patch new file mode 100644 index 0000000..108ab93 --- /dev/null +++ b/SOURCES/kvm-job-Move-BlockJobCreateFlags-to-Job.patch @@ -0,0 +1,423 @@ +From 0d87cf6bb623f8f1d0d7ffb30f25f0e1526528f7 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:13 +0200 +Subject: [PATCH 105/268] job: Move BlockJobCreateFlags to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-31-kwolf@redhat.com> +Patchwork-id: 81106 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 30/73] job: Move BlockJobCreateFlags to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This renames the BlockJobCreateFlags constants, moves a few JOB_INTERNAL +checks to job_create() and the auto_{finalize,dismiss} fields from +BlockJob to Job. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit bb02b65c7d57e4f2136f39bfba95cc68d89eb216) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/commit.c | 2 +- + block/mirror.c | 2 +- + block/replication.c | 4 ++-- + block/stream.c | 2 +- + blockdev.c | 14 +++++++------- + blockjob.c | 27 +++++++-------------------- + include/block/blockjob.h | 17 ----------------- + include/block/blockjob_int.h | 3 +-- + include/qemu/job.h | 20 +++++++++++++++++++- + job.c | 11 ++++++++++- + qemu-img.c | 2 +- + tests/test-blockjob-txn.c | 2 +- + tests/test-blockjob.c | 4 ++-- + 13 files changed, 53 insertions(+), 57 deletions(-) + +diff --git a/block/commit.c b/block/commit.c +index c4a98e5..7a6ae59 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -282,7 +282,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, + } + + s = block_job_create(job_id, &commit_job_driver, NULL, bs, 0, BLK_PERM_ALL, +- speed, BLOCK_JOB_DEFAULT, NULL, NULL, errp); ++ speed, JOB_DEFAULT, NULL, NULL, errp); + if (!s) { + return; + } +diff --git a/block/mirror.c b/block/mirror.c +index 9a7226f..5091e72 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -1284,7 +1284,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs, + } + is_none_mode = mode == MIRROR_SYNC_MODE_NONE; + base = mode == MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL; +- mirror_start_job(job_id, bs, BLOCK_JOB_DEFAULT, target, replaces, ++ mirror_start_job(job_id, bs, JOB_DEFAULT, target, replaces, + speed, granularity, buf_size, backing_mode, + on_source_error, on_target_error, unmap, NULL, NULL, + &mirror_job_driver, is_none_mode, base, false, +diff --git a/block/replication.c b/block/replication.c +index 3f7500e..8241400 100644 +--- a/block/replication.c ++++ b/block/replication.c +@@ -566,7 +566,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, + job = backup_job_create(NULL, s->secondary_disk->bs, s->hidden_disk->bs, + 0, MIRROR_SYNC_MODE_NONE, NULL, false, + BLOCKDEV_ON_ERROR_REPORT, +- BLOCKDEV_ON_ERROR_REPORT, BLOCK_JOB_INTERNAL, ++ BLOCKDEV_ON_ERROR_REPORT, JOB_INTERNAL, + backup_job_completed, bs, NULL, &local_err); + if (local_err) { + error_propagate(errp, local_err); +@@ -691,7 +691,7 @@ static void replication_stop(ReplicationState *rs, bool failover, Error **errp) + + s->stage = BLOCK_REPLICATION_FAILOVER; + commit_active_start(NULL, s->active_disk->bs, s->secondary_disk->bs, +- BLOCK_JOB_INTERNAL, 0, BLOCKDEV_ON_ERROR_REPORT, ++ JOB_INTERNAL, 0, BLOCKDEV_ON_ERROR_REPORT, + NULL, replication_done, bs, true, errp); + break; + default: +diff --git a/block/stream.c b/block/stream.c +index e81b488..eee0253 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -242,7 +242,7 @@ void stream_start(const char *job_id, BlockDriverState *bs, + BLK_PERM_GRAPH_MOD, + BLK_PERM_CONSISTENT_READ | BLK_PERM_WRITE_UNCHANGED | + BLK_PERM_WRITE, +- speed, BLOCK_JOB_DEFAULT, NULL, NULL, errp); ++ speed, JOB_DEFAULT, NULL, NULL, errp); + if (!s) { + goto fail; + } +diff --git a/blockdev.c b/blockdev.c +index 522158c..24deaf1 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3245,7 +3245,7 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + goto out; + } + commit_active_start(has_job_id ? job_id : NULL, bs, base_bs, +- BLOCK_JOB_DEFAULT, speed, on_error, ++ JOB_DEFAULT, speed, on_error, + filter_node_name, NULL, NULL, false, &local_err); + } else { + BlockDriverState *overlay_bs = bdrv_find_overlay(bs, top_bs); +@@ -3276,7 +3276,7 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, + AioContext *aio_context; + QDict *options = NULL; + Error *local_err = NULL; +- int flags, job_flags = BLOCK_JOB_DEFAULT; ++ int flags, job_flags = JOB_DEFAULT; + int64_t size; + bool set_backing_hd = false; + +@@ -3399,10 +3399,10 @@ static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, + } + } + if (!backup->auto_finalize) { +- job_flags |= BLOCK_JOB_MANUAL_FINALIZE; ++ job_flags |= JOB_MANUAL_FINALIZE; + } + if (!backup->auto_dismiss) { +- job_flags |= BLOCK_JOB_MANUAL_DISMISS; ++ job_flags |= JOB_MANUAL_DISMISS; + } + + job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, +@@ -3443,7 +3443,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, + Error *local_err = NULL; + AioContext *aio_context; + BlockJob *job = NULL; +- int job_flags = BLOCK_JOB_DEFAULT; ++ int job_flags = JOB_DEFAULT; + + if (!backup->has_speed) { + backup->speed = 0; +@@ -3492,10 +3492,10 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, + } + } + if (!backup->auto_finalize) { +- job_flags |= BLOCK_JOB_MANUAL_FINALIZE; ++ job_flags |= JOB_MANUAL_FINALIZE; + } + if (!backup->auto_dismiss) { +- job_flags |= BLOCK_JOB_MANUAL_DISMISS; ++ job_flags |= JOB_MANUAL_DISMISS; + } + job = backup_job_create(backup->job_id, bs, target_bs, backup->speed, + backup->sync, NULL, backup->compress, +diff --git a/blockjob.c b/blockjob.c +index a1d1f48..d9d8ff7 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -285,7 +285,7 @@ static void block_job_do_dismiss(BlockJob *job) + static void block_job_conclude(BlockJob *job) + { + job_state_transition(&job->job, JOB_STATUS_CONCLUDED); +- if (job->auto_dismiss || !job_started(&job->job)) { ++ if (job->job.auto_dismiss || !job_started(&job->job)) { + block_job_do_dismiss(job); + } + } +@@ -483,7 +483,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + + static int block_job_needs_finalize(BlockJob *job) + { +- return !job->auto_finalize; ++ return !job->job.auto_finalize; + } + + static void block_job_do_finalize(BlockJob *job) +@@ -688,8 +688,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info->io_status = job->iostatus; + info->ready = job->ready; + info->status = job->job.status; +- info->auto_finalize = job->auto_finalize; +- info->auto_dismiss = job->auto_dismiss; ++ info->auto_finalize = job->job.auto_finalize; ++ info->auto_dismiss = job->job.auto_dismiss; + info->has_error = job->ret != 0; + info->error = job->ret ? g_strdup(strerror(-job->ret)) : NULL; + return info; +@@ -736,7 +736,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg) + static int block_job_event_pending(BlockJob *job) + { + job_state_transition(&job->job, JOB_STATUS_PENDING); +- if (!job->auto_finalize && !block_job_is_internal(job)) { ++ if (!job->job.auto_finalize && !block_job_is_internal(job)) { + qapi_event_send_block_job_pending(job_type(&job->job), + job->job.id, + &error_abort); +@@ -763,19 +763,8 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return NULL; + } + +- if (job_id == NULL && !(flags & BLOCK_JOB_INTERNAL)) { ++ if (job_id == NULL && !(flags & JOB_INTERNAL)) { + job_id = bdrv_get_device_name(bs); +- if (!*job_id) { +- error_setg(errp, "An explicit job ID is required for this node"); +- return NULL; +- } +- } +- +- if (job_id) { +- if (flags & BLOCK_JOB_INTERNAL) { +- error_setg(errp, "Cannot specify job ID for internal block job"); +- return NULL; +- } + } + + blk = blk_new(perm, shared_perm); +@@ -786,7 +775,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + } + + job = job_create(job_id, &driver->job_driver, blk_get_aio_context(blk), +- errp); ++ flags, errp); + if (job == NULL) { + blk_unref(blk); + return NULL; +@@ -800,8 +789,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->blk = blk; + job->cb = cb; + job->opaque = opaque; +- job->auto_finalize = !(flags & BLOCK_JOB_MANUAL_FINALIZE); +- job->auto_dismiss = !(flags & BLOCK_JOB_MANUAL_DISMISS); + + error_setg(&job->blocker, "block device is in use by block job: %s", + job_type_str(&job->job)); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 3e94e18..f9aaaaa 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -91,27 +91,10 @@ typedef struct BlockJob { + /** ret code passed to block_job_completed. */ + int ret; + +- /** True if this job should automatically finalize itself */ +- bool auto_finalize; +- +- /** True if this job should automatically dismiss itself */ +- bool auto_dismiss; +- + BlockJobTxn *txn; + QLIST_ENTRY(BlockJob) txn_list; + } BlockJob; + +-typedef enum BlockJobCreateFlags { +- /* Default behavior */ +- BLOCK_JOB_DEFAULT = 0x00, +- /* BlockJob is not QMP-created and should not send QMP events */ +- BLOCK_JOB_INTERNAL = 0x01, +- /* BlockJob requires manual finalize step */ +- BLOCK_JOB_MANUAL_FINALIZE = 0x02, +- /* BlockJob requires manual dismiss step */ +- BLOCK_JOB_MANUAL_DISMISS = 0x04, +-} BlockJobCreateFlags; +- + /** + * block_job_next: + * @job: A block job, or %NULL. +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 7e705ae..88639f7 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -106,8 +106,7 @@ struct BlockJobDriver { + * @bs: The block + * @perm, @shared_perm: Permissions to request for @bs + * @speed: The maximum speed, in bytes per second, or 0 for unlimited. +- * @flags: Creation flags for the Block Job. +- * See @BlockJobCreateFlags ++ * @flags: Creation flags for the Block Job. See @JobCreateFlags. + * @cb: Completion function for the job. + * @opaque: Opaque pointer value passed to @cb. + * @errp: Error object. +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 858f3be..9783e40 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -99,6 +99,12 @@ typedef struct Job { + /** Set to true when the job has deferred work to the main loop. */ + bool deferred_to_main_loop; + ++ /** True if this job should automatically finalize itself */ ++ bool auto_finalize; ++ ++ /** True if this job should automatically dismiss itself */ ++ bool auto_dismiss; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + } Job; +@@ -140,6 +146,17 @@ struct JobDriver { + void (*free)(Job *job); + }; + ++typedef enum JobCreateFlags { ++ /* Default behavior */ ++ JOB_DEFAULT = 0x00, ++ /* Job is not QMP-created and should not send QMP events */ ++ JOB_INTERNAL = 0x01, ++ /* Job requires manual finalize step */ ++ JOB_MANUAL_FINALIZE = 0x02, ++ /* Job requires manual dismiss step */ ++ JOB_MANUAL_DISMISS = 0x04, ++} JobCreateFlags; ++ + + /** + * Create a new long-running job and return it. +@@ -147,10 +164,11 @@ struct JobDriver { + * @job_id: The id of the newly-created job, or %NULL for internal jobs + * @driver: The class object for the newly-created job. + * @ctx: The AioContext to run the job coroutine in. ++ * @flags: Creation flags for the job. See @JobCreateFlags. + * @errp: Error object. + */ + void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, +- Error **errp); ++ int flags, Error **errp); + + /** + * Add a reference to Job refcnt, it will be decreased with job_unref, and then +diff --git a/job.c b/job.c +index aaacfcc..dd46170 100644 +--- a/job.c ++++ b/job.c +@@ -182,11 +182,15 @@ static void job_sleep_timer_cb(void *opaque) + } + + void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, +- Error **errp) ++ int flags, Error **errp) + { + Job *job; + + if (job_id) { ++ if (flags & JOB_INTERNAL) { ++ error_setg(errp, "Cannot specify job ID for internal job"); ++ return NULL; ++ } + if (!id_wellformed(job_id)) { + error_setg(errp, "Invalid job ID '%s'", job_id); + return NULL; +@@ -195,6 +199,9 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + error_setg(errp, "Job ID '%s' already in use", job_id); + return NULL; + } ++ } else if (!(flags & JOB_INTERNAL)) { ++ error_setg(errp, "An explicit job ID is required"); ++ return NULL; + } + + job = g_malloc0(driver->instance_size); +@@ -205,6 +212,8 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + job->busy = false; + job->paused = true; + job->pause_count = 1; ++ job->auto_finalize = !(flags & JOB_MANUAL_FINALIZE); ++ job->auto_dismiss = !(flags & JOB_MANUAL_DISMISS); + + job_state_transition(job, JOB_STATUS_CREATED); + aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, +diff --git a/qemu-img.c b/qemu-img.c +index f5bb0ef..843dc6a 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -1026,7 +1026,7 @@ static int img_commit(int argc, char **argv) + + aio_context = bdrv_get_aio_context(bs); + aio_context_acquire(aio_context); +- commit_active_start("commit", bs, base_bs, BLOCK_JOB_DEFAULT, 0, ++ commit_active_start("commit", bs, base_bs, JOB_DEFAULT, 0, + BLOCKDEV_ON_ERROR_REPORT, NULL, common_block_job_cb, + &cbi, false, &local_err); + aio_context_release(aio_context); +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 93d1ff0..60e9fa2 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -107,7 +107,7 @@ static BlockJob *test_block_job_start(unsigned int iterations, + + snprintf(job_id, sizeof(job_id), "job%u", counter++); + s = block_job_create(job_id, &test_block_job_driver, txn, bs, +- 0, BLK_PERM_ALL, 0, BLOCK_JOB_DEFAULT, ++ 0, BLK_PERM_ALL, 0, JOB_DEFAULT, + test_block_job_cb, data, &error_abort); + s->iterations = iterations; + s->use_timer = use_timer; +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index ceb5960..8bb0aa8 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -59,7 +59,7 @@ static BlockJob *do_test_id(BlockBackend *blk, const char *id, + bool should_succeed) + { + return mk_job(blk, id, &test_block_job_driver, +- should_succeed, BLOCK_JOB_DEFAULT); ++ should_succeed, JOB_DEFAULT); + } + + /* This creates a BlockBackend (optionally with a name) with a +@@ -214,7 +214,7 @@ static CancelJob *create_common(BlockJob **pjob) + + blk = create_blk(NULL); + job = mk_job(blk, "Steve", &test_cancel_driver, true, +- BLOCK_JOB_MANUAL_FINALIZE | BLOCK_JOB_MANUAL_DISMISS); ++ JOB_MANUAL_FINALIZE | JOB_MANUAL_DISMISS); + job_ref(&job->job); + assert(job->job.status == JOB_STATUS_CREATED); + s = container_of(job, CancelJob, common); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-cancelled-to-Job.patch b/SOURCES/kvm-job-Move-cancelled-to-Job.patch new file mode 100644 index 0000000..01c493b --- /dev/null +++ b/SOURCES/kvm-job-Move-cancelled-to-Job.patch @@ -0,0 +1,439 @@ +From 1a1266008b6159179bfaf7010ccb6b4b373dd468 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:06 +0200 +Subject: [PATCH 098/268] job: Move cancelled to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-24-kwolf@redhat.com> +Patchwork-id: 81103 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 23/73] job: Move cancelled to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +We cannot yet move the whole logic around job cancelling to Job because +it depends on quite a few other things that are still only in BlockJob, +but we can move the cancelled field at least. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit daa7f2f9467bc5624f04f28d4b01b88f08c6589c) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 6 +++--- + block/commit.c | 4 ++-- + block/mirror.c | 20 ++++++++++---------- + block/stream.c | 4 ++-- + blockjob.c | 28 +++++++++++++--------------- + include/block/blockjob.h | 8 -------- + include/block/blockjob_int.h | 8 -------- + include/qemu/job.h | 11 +++++++++++ + job.c | 5 +++++ + tests/test-blockjob-txn.c | 6 +++--- + tests/test-blockjob.c | 2 +- + 11 files changed, 50 insertions(+), 52 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index cfdb89d..ef0aa0e 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -329,7 +329,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) + { + uint64_t delay_ns; + +- if (block_job_is_cancelled(&job->common)) { ++ if (job_is_cancelled(&job->common.job)) { + return true; + } + +@@ -339,7 +339,7 @@ static bool coroutine_fn yield_and_check(BackupBlockJob *job) + job->bytes_read = 0; + block_job_sleep_ns(&job->common, delay_ns); + +- if (block_job_is_cancelled(&job->common)) { ++ if (job_is_cancelled(&job->common.job)) { + return true; + } + +@@ -441,7 +441,7 @@ static void coroutine_fn backup_run(void *opaque) + if (job->sync_mode == MIRROR_SYNC_MODE_NONE) { + /* All bits are set in copy_bitmap to allow any cluster to be copied. + * This does not actually require them to be copied. */ +- while (!block_job_is_cancelled(&job->common)) { ++ while (!job_is_cancelled(&job->common.job)) { + /* Yield until the job is cancelled. We just let our before_write + * notify callback service CoW requests. */ + block_job_yield(&job->common); +diff --git a/block/commit.c b/block/commit.c +index 925c96a..85baea8 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -90,7 +90,7 @@ static void commit_complete(BlockJob *job, void *opaque) + * the normal backing chain can be restored. */ + blk_unref(s->base); + +- if (!block_job_is_cancelled(&s->common) && ret == 0) { ++ if (!job_is_cancelled(&s->common.job) && ret == 0) { + /* success */ + ret = bdrv_drop_intermediate(s->commit_top_bs, base, + s->backing_file_str); +@@ -172,7 +172,7 @@ static void coroutine_fn commit_run(void *opaque) + * with no pending I/O here so that bdrv_drain_all() returns. + */ + block_job_sleep_ns(&s->common, delay_ns); +- if (block_job_is_cancelled(&s->common)) { ++ if (job_is_cancelled(&s->common.job)) { + break; + } + /* Copy if allocated above the base */ +diff --git a/block/mirror.c b/block/mirror.c +index 0df4f70..424072e 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -622,7 +622,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s) + + mirror_throttle(s); + +- if (block_job_is_cancelled(&s->common)) { ++ if (job_is_cancelled(&s->common.job)) { + s->initial_zeroing_ongoing = false; + return 0; + } +@@ -650,7 +650,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s) + + mirror_throttle(s); + +- if (block_job_is_cancelled(&s->common)) { ++ if (job_is_cancelled(&s->common.job)) { + return 0; + } + +@@ -695,7 +695,7 @@ static void coroutine_fn mirror_run(void *opaque) + checking for a NULL string */ + int ret = 0; + +- if (block_job_is_cancelled(&s->common)) { ++ if (job_is_cancelled(&s->common.job)) { + goto immediate_exit; + } + +@@ -729,10 +729,10 @@ static void coroutine_fn mirror_run(void *opaque) + /* Report BLOCK_JOB_READY and wait for complete. */ + block_job_event_ready(&s->common); + s->synced = true; +- while (!block_job_is_cancelled(&s->common) && !s->should_complete) { ++ while (!job_is_cancelled(&s->common.job) && !s->should_complete) { + block_job_yield(&s->common); + } +- s->common.cancelled = false; ++ s->common.job.cancelled = false; + goto immediate_exit; + } + +@@ -768,7 +768,7 @@ static void coroutine_fn mirror_run(void *opaque) + s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME); + if (!s->is_none_mode) { + ret = mirror_dirty_init(s); +- if (ret < 0 || block_job_is_cancelled(&s->common)) { ++ if (ret < 0 || job_is_cancelled(&s->common.job)) { + goto immediate_exit; + } + } +@@ -828,7 +828,7 @@ static void coroutine_fn mirror_run(void *opaque) + } + + should_complete = s->should_complete || +- block_job_is_cancelled(&s->common); ++ job_is_cancelled(&s->common.job); + cnt = bdrv_get_dirty_count(s->dirty_bitmap); + } + +@@ -856,7 +856,7 @@ static void coroutine_fn mirror_run(void *opaque) + * completion. + */ + assert(QLIST_EMPTY(&bs->tracked_requests)); +- s->common.cancelled = false; ++ s->common.job.cancelled = false; + need_drain = false; + break; + } +@@ -869,7 +869,7 @@ static void coroutine_fn mirror_run(void *opaque) + } + trace_mirror_before_sleep(s, cnt, s->synced, delay_ns); + block_job_sleep_ns(&s->common, delay_ns); +- if (block_job_is_cancelled(&s->common) && ++ if (job_is_cancelled(&s->common.job) && + (!s->synced || s->common.force)) + { + break; +@@ -884,7 +884,7 @@ immediate_exit: + * the target is a copy of the source. + */ + assert(ret < 0 || ((s->common.force || !s->synced) && +- block_job_is_cancelled(&s->common))); ++ job_is_cancelled(&s->common.job))); + assert(need_drain); + mirror_wait_for_all_io(s); + } +diff --git a/block/stream.c b/block/stream.c +index 7273d22..22c71ae 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -66,7 +66,7 @@ static void stream_complete(BlockJob *job, void *opaque) + BlockDriverState *base = s->base; + Error *local_err = NULL; + +- if (!block_job_is_cancelled(&s->common) && bs->backing && ++ if (!job_is_cancelled(&s->common.job) && bs->backing && + data->ret == 0) { + const char *base_id = NULL, *base_fmt = NULL; + if (base) { +@@ -141,7 +141,7 @@ static void coroutine_fn stream_run(void *opaque) + * with no pending I/O here so that bdrv_drain_all() returns. + */ + block_job_sleep_ns(&s->common, delay_ns); +- if (block_job_is_cancelled(&s->common)) { ++ if (job_is_cancelled(&s->common.job)) { + break; + } + +diff --git a/blockjob.c b/blockjob.c +index 0bf0a26..f4f9956 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -379,7 +379,7 @@ static void block_job_conclude(BlockJob *job) + + static void block_job_update_rc(BlockJob *job) + { +- if (!job->ret && block_job_is_cancelled(job)) { ++ if (!job->ret && job_is_cancelled(&job->job)) { + job->ret = -ECANCELED; + } + if (job->ret) { +@@ -438,7 +438,7 @@ static int block_job_finalize_single(BlockJob *job) + + /* Emit events only if we actually started */ + if (block_job_started(job)) { +- if (block_job_is_cancelled(job)) { ++ if (job_is_cancelled(&job->job)) { + block_job_event_cancelled(job); + } else { + const char *msg = NULL; +@@ -464,7 +464,7 @@ static void block_job_cancel_async(BlockJob *job, bool force) + job->user_paused = false; + job->pause_count--; + } +- job->cancelled = true; ++ job->job.cancelled = true; + /* To prevent 'force == false' overriding a previous 'force == true' */ + job->force |= force; + } +@@ -519,7 +519,8 @@ static int block_job_finish_sync(BlockJob *job, + while (!job->completed) { + aio_poll(qemu_get_aio_context(), true); + } +- ret = (job->cancelled && job->ret == 0) ? -ECANCELED : job->ret; ++ ret = (job_is_cancelled(&job->job) && job->ret == 0) ++ ? -ECANCELED : job->ret; + job_unref(&job->job); + return ret; + } +@@ -557,7 +558,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + other_job = QLIST_FIRST(&txn->jobs); + ctx = blk_get_aio_context(other_job->blk); + if (!other_job->completed) { +- assert(other_job->cancelled); ++ assert(job_is_cancelled(&other_job->job)); + block_job_finish_sync(other_job, NULL, NULL); + } + block_job_finalize_single(other_job); +@@ -651,7 +652,9 @@ void block_job_complete(BlockJob *job, Error **errp) + if (job_apply_verb(&job->job, JOB_VERB_COMPLETE, errp)) { + return; + } +- if (job->pause_count || job->cancelled || !job->driver->complete) { ++ if (job->pause_count || job_is_cancelled(&job->job) || ++ !job->driver->complete) ++ { + error_setg(errp, "The active block job '%s' cannot be completed", + job->job.id); + return; +@@ -1006,7 +1009,7 @@ void coroutine_fn block_job_pause_point(BlockJob *job) + if (!block_job_should_pause(job)) { + return; + } +- if (block_job_is_cancelled(job)) { ++ if (job_is_cancelled(&job->job)) { + return; + } + +@@ -1014,7 +1017,7 @@ void coroutine_fn block_job_pause_point(BlockJob *job) + job->driver->pause(job); + } + +- if (block_job_should_pause(job) && !block_job_is_cancelled(job)) { ++ if (block_job_should_pause(job) && !job_is_cancelled(&job->job)) { + JobStatus status = job->job.status; + job_state_transition(&job->job, status == JOB_STATUS_READY + ? JOB_STATUS_STANDBY +@@ -1066,17 +1069,12 @@ void block_job_enter(BlockJob *job) + block_job_enter_cond(job, NULL); + } + +-bool block_job_is_cancelled(BlockJob *job) +-{ +- return job->cancelled; +-} +- + void block_job_sleep_ns(BlockJob *job, int64_t ns) + { + assert(job->busy); + + /* Check cancellation *before* setting busy = false, too! */ +- if (block_job_is_cancelled(job)) { ++ if (job_is_cancelled(&job->job)) { + return; + } + +@@ -1092,7 +1090,7 @@ void block_job_yield(BlockJob *job) + assert(job->busy); + + /* Check cancellation *before* setting busy = false, too! */ +- if (block_job_is_cancelled(job)) { ++ if (job_is_cancelled(&job->job)) { + return; + } + +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 087e782..1e708f4 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -57,14 +57,6 @@ typedef struct BlockJob { + Coroutine *co; + + /** +- * Set to true if the job should cancel itself. The flag must +- * always be tested just before toggling the busy flag from false +- * to true. After a job has been cancelled, it should only yield +- * if #aio_poll will ("sooner or later") reenter the coroutine. +- */ +- bool cancelled; +- +- /** + * Set to true if the job should abort immediately without waiting + * for data to be in sync. + */ +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 6f0fe3c..d64f30e 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -196,14 +196,6 @@ void block_job_early_fail(BlockJob *job); + void block_job_completed(BlockJob *job, int ret); + + /** +- * block_job_is_cancelled: +- * @job: The job being queried. +- * +- * Returns whether the job is scheduled for cancellation. +- */ +-bool block_job_is_cancelled(BlockJob *job); +- +-/** + * block_job_pause_point: + * @job: The job that is ready to pause. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 0751e2a..5dfbec5 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -47,6 +47,14 @@ typedef struct Job { + /** Current state; See @JobStatus for details. */ + JobStatus status; + ++ /** ++ * Set to true if the job should cancel itself. The flag must ++ * always be tested just before toggling the busy flag from false ++ * to true. After a job has been cancelled, it should only yield ++ * if #aio_poll will ("sooner or later") reenter the coroutine. ++ */ ++ bool cancelled; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + } Job; +@@ -93,6 +101,9 @@ JobType job_type(const Job *job); + /** Returns the enum string for the JobType of a given Job. */ + const char *job_type_str(const Job *job); + ++/** Returns whether the job is scheduled for cancellation. */ ++bool job_is_cancelled(Job *job); ++ + /** + * Get the next element from the list of block jobs after @job, or the + * first one if @job is %NULL. +diff --git a/job.c b/job.c +index 926f1de..1abca6a 100644 +--- a/job.c ++++ b/job.c +@@ -95,6 +95,11 @@ const char *job_type_str(const Job *job) + return JobType_str(job_type(job)); + } + ++bool job_is_cancelled(Job *job) ++{ ++ return job->cancelled; ++} ++ + Job *job_next(Job *job) + { + if (!job) { +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index b49b28c..26b4bbb 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -29,7 +29,7 @@ static void test_block_job_complete(BlockJob *job, void *opaque) + BlockDriverState *bs = blk_bs(job->blk); + int rc = (intptr_t)opaque; + +- if (block_job_is_cancelled(job)) { ++ if (job_is_cancelled(&job->job)) { + rc = -ECANCELED; + } + +@@ -49,7 +49,7 @@ static void coroutine_fn test_block_job_run(void *opaque) + block_job_yield(job); + } + +- if (block_job_is_cancelled(job)) { ++ if (job_is_cancelled(&job->job)) { + break; + } + } +@@ -66,7 +66,7 @@ typedef struct { + static void test_block_job_cb(void *opaque, int ret) + { + TestBlockJobCBData *data = opaque; +- if (!ret && block_job_is_cancelled(&data->job->common)) { ++ if (!ret && job_is_cancelled(&data->job->common.job)) { + ret = -ECANCELED; + } + *data->result = ret; +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index e24fc3f..fa31481 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -179,7 +179,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + CancelJob *s = opaque; + + while (!s->should_complete) { +- if (block_job_is_cancelled(&s->common)) { ++ if (job_is_cancelled(&s->common.job)) { + goto defer; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-completion-and-cancellation-to-Job.patch b/SOURCES/kvm-job-Move-completion-and-cancellation-to-Job.patch new file mode 100644 index 0000000..d15e4c5 --- /dev/null +++ b/SOURCES/kvm-job-Move-completion-and-cancellation-to-Job.patch @@ -0,0 +1,843 @@ +From 426d98dc12ef863627d90890278e6de92a81446f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:23 +0200 +Subject: [PATCH 115/268] job: Move completion and cancellation to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-41-kwolf@redhat.com> +Patchwork-id: 81116 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 40/73] job: Move completion and cancellation to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves the top-level job completion and cancellation functions from +BlockJob to Job. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 3d70ff53b6bf90d9eec6f97024ec9895f6799d9e) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block.c | 4 ++- + block/backup.c | 3 +- + block/commit.c | 6 ++-- + block/mirror.c | 6 ++-- + block/replication.c | 4 +-- + block/stream.c | 2 +- + block/trace-events | 3 -- + blockdev.c | 8 ++--- + blockjob.c | 76 --------------------------------------- + include/block/blockjob.h | 55 ----------------------------- + include/block/blockjob_int.h | 18 ---------- + include/qemu/job.h | 68 +++++++++++++++++++++++++++++------ + job.c | 84 +++++++++++++++++++++++++++++++++++++++----- + qemu-img.c | 2 +- + tests/test-bdrv-drain.c | 5 ++- + tests/test-blockjob-txn.c | 14 ++++---- + tests/test-blockjob.c | 21 ++++++----- + trace-events | 1 + + 18 files changed, 171 insertions(+), 209 deletions(-) + +diff --git a/block.c b/block.c +index d991a09..a83787f 100644 +--- a/block.c ++++ b/block.c +@@ -3374,7 +3374,9 @@ static void bdrv_close(BlockDriverState *bs) + + void bdrv_close_all(void) + { +- block_job_cancel_sync_all(); ++ /* TODO We do want to cancel all jobs instead of just block jobs on ++ * shutdown, but bdrv_close_all() isn't the right place any more. */ ++ job_cancel_sync_all(); + nbd_export_close_all(); + + /* Drop references from requests still in flight, such as canceled block +diff --git a/block/backup.c b/block/backup.c +index 6172f90..b13f91d 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -319,10 +319,9 @@ typedef struct { + + static void backup_complete(Job *job, void *opaque) + { +- BlockJob *bjob = container_of(job, BlockJob, job); + BackupCompleteData *data = opaque; + +- block_job_completed(bjob, data->ret); ++ job_completed(job, data->ret); + g_free(data); + } + +diff --git a/block/commit.c b/block/commit.c +index 40d97a3..b0a847e 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -112,12 +112,12 @@ static void commit_complete(Job *job, void *opaque) + blk_unref(s->top); + + /* If there is more than one reference to the job (e.g. if called from +- * job_finish_sync()), block_job_completed() won't free it and therefore +- * the blockers on the intermediate nodes remain. This would cause ++ * job_finish_sync()), job_completed() won't free it and therefore the ++ * blockers on the intermediate nodes remain. This would cause + * bdrv_set_backing_hd() to fail. */ + block_job_remove_all_bdrv(bjob); + +- block_job_completed(&s->common, ret); ++ job_completed(job, ret); + g_free(data); + + /* If bdrv_drop_intermediate() didn't already do that, remove the commit +diff --git a/block/mirror.c b/block/mirror.c +index 656237a..c63cf7c 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -498,7 +498,7 @@ static void mirror_exit(Job *job, void *opaque) + bdrv_release_dirty_bitmap(src, s->dirty_bitmap); + + /* Make sure that the source BDS doesn't go away before we called +- * block_job_completed(). */ ++ * job_completed(). */ + bdrv_ref(src); + bdrv_ref(mirror_top_bs); + bdrv_ref(target_bs); +@@ -581,7 +581,7 @@ static void mirror_exit(Job *job, void *opaque) + blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); + blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort); + +- block_job_completed(&s->common, data->ret); ++ job_completed(job, data->ret); + + g_free(data); + bdrv_drained_end(src); +@@ -954,7 +954,7 @@ static void mirror_complete(Job *job, Error **errp) + } + + s->should_complete = true; +- block_job_enter(&s->common); ++ job_enter(job); + } + + static void mirror_pause(Job *job) +diff --git a/block/replication.c b/block/replication.c +index 8241400..f3b2839 100644 +--- a/block/replication.c ++++ b/block/replication.c +@@ -145,7 +145,7 @@ static void replication_close(BlockDriverState *bs) + replication_stop(s->rs, false, NULL); + } + if (s->stage == BLOCK_REPLICATION_FAILOVER) { +- block_job_cancel_sync(s->active_disk->bs->job); ++ job_cancel_sync(&s->active_disk->bs->job->job); + } + + if (s->mode == REPLICATION_MODE_SECONDARY) { +@@ -679,7 +679,7 @@ static void replication_stop(ReplicationState *rs, bool failover, Error **errp) + * disk, secondary disk in backup_job_completed(). + */ + if (s->secondary_disk->bs->job) { +- block_job_cancel_sync(s->secondary_disk->bs->job); ++ job_cancel_sync(&s->secondary_disk->bs->job->job); + } + + if (!failover) { +diff --git a/block/stream.c b/block/stream.c +index b996278..8546c41 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -93,7 +93,7 @@ out: + } + + g_free(s->backing_file_str); +- block_job_completed(&s->common, data->ret); ++ job_completed(job, data->ret); + g_free(data); + } + +diff --git a/block/trace-events b/block/trace-events +index 93b9279..2d59b53 100644 +--- a/block/trace-events ++++ b/block/trace-events +@@ -4,9 +4,6 @@ + bdrv_open_common(void *bs, const char *filename, int flags, const char *format_name) "bs %p filename \"%s\" flags 0x%x format_name \"%s\"" + bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" + +-# blockjob.c +-block_job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" +- + # block/block-backend.c + blk_co_preadv(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" + blk_co_pwritev(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" +diff --git a/blockdev.c b/blockdev.c +index 9aa2e79..c768e68 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -151,7 +151,7 @@ void blockdev_mark_auto_del(BlockBackend *blk) + aio_context_acquire(aio_context); + + if (bs->job) { +- block_job_cancel(bs->job, false); ++ job_cancel(&bs->job->job, false); + } + + aio_context_release(aio_context); +@@ -1926,7 +1926,7 @@ static void drive_backup_abort(BlkActionState *common) + aio_context = bdrv_get_aio_context(state->bs); + aio_context_acquire(aio_context); + +- block_job_cancel_sync(state->job); ++ job_cancel_sync(&state->job->job); + + aio_context_release(aio_context); + } +@@ -2024,7 +2024,7 @@ static void blockdev_backup_abort(BlkActionState *common) + aio_context = bdrv_get_aio_context(state->bs); + aio_context_acquire(aio_context); + +- block_job_cancel_sync(state->job); ++ job_cancel_sync(&state->job->job); + + aio_context_release(aio_context); + } +@@ -3852,7 +3852,7 @@ void qmp_block_job_cancel(const char *device, + } + + trace_qmp_block_job_cancel(job); +- block_job_user_cancel(job, force, errp); ++ job_user_cancel(&job->job, force, errp); + out: + aio_context_release(aio_context); + } +diff --git a/blockjob.c b/blockjob.c +index 14b21c8..438baa1 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -255,63 +255,6 @@ void block_job_dismiss(BlockJob **jobptr, Error **errp) + *jobptr = NULL; + } + +-void block_job_cancel(BlockJob *job, bool force) +-{ +- if (job->job.status == JOB_STATUS_CONCLUDED) { +- job_do_dismiss(&job->job); +- return; +- } +- job_cancel_async(&job->job, force); +- if (!job_started(&job->job)) { +- block_job_completed(job, -ECANCELED); +- } else if (job->job.deferred_to_main_loop) { +- job_completed_txn_abort(&job->job); +- } else { +- block_job_enter(job); +- } +-} +- +-void block_job_user_cancel(BlockJob *job, bool force, Error **errp) +-{ +- if (job_apply_verb(&job->job, JOB_VERB_CANCEL, errp)) { +- return; +- } +- block_job_cancel(job, force); +-} +- +-/* A wrapper around block_job_cancel() taking an Error ** parameter so it may be +- * used with job_finish_sync() without the need for (rather nasty) function +- * pointer casts there. */ +-static void block_job_cancel_err(Job *job, Error **errp) +-{ +- BlockJob *bjob = container_of(job, BlockJob, job); +- assert(is_block_job(job)); +- block_job_cancel(bjob, false); +-} +- +-int block_job_cancel_sync(BlockJob *job) +-{ +- return job_finish_sync(&job->job, &block_job_cancel_err, NULL); +-} +- +-void block_job_cancel_sync_all(void) +-{ +- BlockJob *job; +- AioContext *aio_context; +- +- while ((job = block_job_next(NULL))) { +- aio_context = blk_get_aio_context(job->blk); +- aio_context_acquire(aio_context); +- block_job_cancel_sync(job); +- aio_context_release(aio_context); +- } +-} +- +-int block_job_complete_sync(BlockJob *job, Error **errp) +-{ +- return job_finish_sync(&job->job, job_complete, errp); +-} +- + void block_job_progress_update(BlockJob *job, uint64_t done) + { + job->offset += done; +@@ -488,25 +431,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return job; + } + +-void block_job_completed(BlockJob *job, int ret) +-{ +- assert(job && job->job.txn && !job_is_completed(&job->job)); +- assert(blk_bs(job->blk)->job == job); +- job->job.ret = ret; +- job_update_rc(&job->job); +- trace_block_job_completed(job, ret, job->job.ret); +- if (job->job.ret) { +- job_completed_txn_abort(&job->job); +- } else { +- job_completed_txn_success(&job->job); +- } +-} +- +-void block_job_enter(BlockJob *job) +-{ +- job_enter_cond(&job->job, NULL); +-} +- + void block_job_yield(BlockJob *job) + { + assert(job->job.busy); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 09e6bb4..e9ed7b8 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -141,15 +141,6 @@ void block_job_remove_all_bdrv(BlockJob *job); + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); + + /** +- * block_job_cancel: +- * @job: The job to be canceled. +- * @force: Quit a job without waiting for data to be in sync. +- * +- * Asynchronously cancel the specified job. +- */ +-void block_job_cancel(BlockJob *job, bool force); +- +-/** + * block_job_dismiss: + * @job: The job to be dismissed. + * @errp: Error object. +@@ -186,52 +177,6 @@ void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining); + BlockJobInfo *block_job_query(BlockJob *job, Error **errp); + + /** +- * block_job_user_cancel: +- * @job: The job to be cancelled. +- * @force: Quit a job without waiting for data to be in sync. +- * +- * Cancels the specified job, but may refuse to do so if the +- * operation isn't currently meaningful. +- */ +-void block_job_user_cancel(BlockJob *job, bool force, Error **errp); +- +-/** +- * block_job_cancel_sync: +- * @job: The job to be canceled. +- * +- * Synchronously cancel the job. The completion callback is called +- * before the function returns. The job may actually complete +- * instead of canceling itself; the circumstances under which this +- * happens depend on the kind of job that is active. +- * +- * Returns the return value from the job if the job actually completed +- * during the call, or -ECANCELED if it was canceled. +- */ +-int block_job_cancel_sync(BlockJob *job); +- +-/** +- * block_job_cancel_sync_all: +- * +- * Synchronously cancels all jobs using block_job_cancel_sync(). +- */ +-void block_job_cancel_sync_all(void); +- +-/** +- * block_job_complete_sync: +- * @job: The job to be completed. +- * @errp: Error object which may be set by block_job_complete(); this is not +- * necessarily set on every error, the job return value has to be +- * checked as well. +- * +- * Synchronously complete the job. The completion callback is called before the +- * function returns, unless it is NULL (which is permissible when using this +- * function). +- * +- * Returns the return value from the job. +- */ +-int block_job_complete_sync(BlockJob *job, Error **errp); +- +-/** + * block_job_iostatus_reset: + * @job: The job whose I/O status should be reset. + * +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 29a2802..7df07b2 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -124,24 +124,6 @@ void block_job_yield(BlockJob *job); + int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n); + + /** +- * block_job_completed: +- * @job: The job being completed. +- * @ret: The status code. +- * +- * Call the completion function that was registered at creation time, and +- * free @job. +- */ +-void block_job_completed(BlockJob *job, int ret); +- +-/** +- * block_job_enter: +- * @job: The job to enter. +- * +- * Continue the specified job by entering the coroutine. +- */ +-void block_job_enter(BlockJob *job); +- +-/** + * block_job_event_ready: + * @job: The job which is now ready to be completed. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 39d74ab..bbe1b0c 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -74,8 +74,8 @@ typedef struct Job { + + /** + * Set to false by the job while the coroutine has yielded and may be +- * re-entered by block_job_enter(). There may still be I/O or event loop +- * activity pending. Accessed under block_job_mutex (in blockjob.c). ++ * re-entered by job_enter(). There may still be I/O or event loop activity ++ * pending. Accessed under block_job_mutex (in blockjob.c). + */ + bool busy; + +@@ -114,7 +114,7 @@ typedef struct Job { + /** True if this job should automatically dismiss itself */ + bool auto_dismiss; + +- /** ret code passed to block_job_completed. */ ++ /** ret code passed to job_completed. */ + int ret; + + /** The completion function that will be called when the job completes. */ +@@ -266,8 +266,8 @@ void job_txn_unref(JobTxn *txn); + * @job: Job to add to the transaction + * + * Add @job to the transaction. The @job must not already be in a transaction. +- * The caller must call either job_txn_unref() or block_job_completed() to +- * release the reference that is automatically grabbed here. ++ * The caller must call either job_txn_unref() or job_completed() to release ++ * the reference that is automatically grabbed here. + * + * If @txn is NULL, the function does nothing. + */ +@@ -416,8 +416,59 @@ int job_apply_verb(Job *job, JobVerb verb, Error **errp); + /** The @job could not be started, free it. */ + void job_early_fail(Job *job); + ++/** ++ * @job: The job being completed. ++ * @ret: The status code. ++ * ++ * Marks @job as completed. If @ret is non-zero, the job transaction it is part ++ * of is aborted. If @ret is zero, the job moves into the WAITING state. If it ++ * is the last job to complete in its transaction, all jobs in the transaction ++ * move from WAITING to PENDING. ++ */ ++void job_completed(Job *job, int ret); ++ + /** Asynchronously complete the specified @job. */ +-void job_complete(Job *job, Error **errp);; ++void job_complete(Job *job, Error **errp); ++ ++/** ++ * Asynchronously cancel the specified @job. If @force is true, the job should ++ * be cancelled immediately without waiting for a consistent state. ++ */ ++void job_cancel(Job *job, bool force); ++ ++/** ++ * Cancels the specified job like job_cancel(), but may refuse to do so if the ++ * operation isn't meaningful in the current state of the job. ++ */ ++void job_user_cancel(Job *job, bool force, Error **errp); ++ ++/** ++ * Synchronously cancel the @job. The completion callback is called ++ * before the function returns. The job may actually complete ++ * instead of canceling itself; the circumstances under which this ++ * happens depend on the kind of job that is active. ++ * ++ * Returns the return value from the job if the job actually completed ++ * during the call, or -ECANCELED if it was canceled. ++ */ ++int job_cancel_sync(Job *job); ++ ++/** Synchronously cancels all jobs using job_cancel_sync(). */ ++void job_cancel_sync_all(void); ++ ++/** ++ * @job: The job to be completed. ++ * @errp: Error object which may be set by job_complete(); this is not ++ * necessarily set on every error, the job return value has to be ++ * checked as well. ++ * ++ * Synchronously complete the job. The completion callback is called before the ++ * function returns, unless it is NULL (which is permissible when using this ++ * function). ++ * ++ * Returns the return value from the job. ++ */ ++int job_complete_sync(Job *job, Error **errp); + + /** + * For a @job that has finished its work and is pending awaiting explicit +@@ -459,11 +510,6 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + void job_state_transition(Job *job, JobStatus s1); + void coroutine_fn job_do_yield(Job *job, uint64_t ns); + bool job_should_pause(Job *job); +-bool job_started(Job *job); + void job_do_dismiss(Job *job); +-void job_update_rc(Job *job); +-void job_cancel_async(Job *job, bool force); +-void job_completed_txn_abort(Job *job); +-void job_completed_txn_success(Job *job); + + #endif +diff --git a/job.c b/job.c +index 4f6fd73..2e453f6 100644 +--- a/job.c ++++ b/job.c +@@ -221,7 +221,7 @@ bool job_is_completed(Job *job) + return false; + } + +-bool job_started(Job *job) ++static bool job_started(Job *job) + { + return job->co; + } +@@ -391,10 +391,10 @@ void job_enter(Job *job) + } + + /* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. +- * Reentering the job coroutine with block_job_enter() before the timer has +- * expired is allowed and cancels the timer. ++ * Reentering the job coroutine with job_enter() before the timer has expired ++ * is allowed and cancels the timer. + * +- * If @ns is (uint64_t) -1, no timer is scheduled and block_job_enter() must be ++ * If @ns is (uint64_t) -1, no timer is scheduled and job_enter() must be + * called explicitly. */ + void coroutine_fn job_do_yield(Job *job, uint64_t ns) + { +@@ -579,7 +579,7 @@ static void job_conclude(Job *job) + } + } + +-void job_update_rc(Job *job) ++static void job_update_rc(Job *job) + { + if (!job->ret && job_is_cancelled(job)) { + job->ret = -ECANCELED; +@@ -644,7 +644,7 @@ static int job_finalize_single(Job *job) + return 0; + } + +-void job_cancel_async(Job *job, bool force) ++static void job_cancel_async(Job *job, bool force) + { + if (job->user_paused) { + /* Do not call job_enter here, the caller will handle it. */ +@@ -660,7 +660,7 @@ void job_cancel_async(Job *job, bool force) + job->force_cancel |= force; + } + +-void job_completed_txn_abort(Job *job) ++static void job_completed_txn_abort(Job *job) + { + AioContext *ctx; + JobTxn *txn = job->txn; +@@ -748,7 +748,7 @@ static int job_transition_to_pending(Job *job) + return 0; + } + +-void job_completed_txn_success(Job *job) ++static void job_completed_txn_success(Job *job) + { + JobTxn *txn = job->txn; + Job *other_job; +@@ -774,6 +774,74 @@ void job_completed_txn_success(Job *job) + } + } + ++void job_completed(Job *job, int ret) ++{ ++ assert(job && job->txn && !job_is_completed(job)); ++ job->ret = ret; ++ job_update_rc(job); ++ trace_job_completed(job, ret, job->ret); ++ if (job->ret) { ++ job_completed_txn_abort(job); ++ } else { ++ job_completed_txn_success(job); ++ } ++} ++ ++void job_cancel(Job *job, bool force) ++{ ++ if (job->status == JOB_STATUS_CONCLUDED) { ++ job_do_dismiss(job); ++ return; ++ } ++ job_cancel_async(job, force); ++ if (!job_started(job)) { ++ job_completed(job, -ECANCELED); ++ } else if (job->deferred_to_main_loop) { ++ job_completed_txn_abort(job); ++ } else { ++ job_enter(job); ++ } ++} ++ ++void job_user_cancel(Job *job, bool force, Error **errp) ++{ ++ if (job_apply_verb(job, JOB_VERB_CANCEL, errp)) { ++ return; ++ } ++ job_cancel(job, force); ++} ++ ++/* A wrapper around job_cancel() taking an Error ** parameter so it may be ++ * used with job_finish_sync() without the need for (rather nasty) function ++ * pointer casts there. */ ++static void job_cancel_err(Job *job, Error **errp) ++{ ++ job_cancel(job, false); ++} ++ ++int job_cancel_sync(Job *job) ++{ ++ return job_finish_sync(job, &job_cancel_err, NULL); ++} ++ ++void job_cancel_sync_all(void) ++{ ++ Job *job; ++ AioContext *aio_context; ++ ++ while ((job = job_next(NULL))) { ++ aio_context = job->aio_context; ++ aio_context_acquire(aio_context); ++ job_cancel_sync(job); ++ aio_context_release(aio_context); ++ } ++} ++ ++int job_complete_sync(Job *job, Error **errp) ++{ ++ return job_finish_sync(job, job_complete, errp); ++} ++ + void job_complete(Job *job, Error **errp) + { + /* Should not be reachable via external interface for internal jobs */ +diff --git a/qemu-img.c b/qemu-img.c +index 91b3151..734ea94 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -881,7 +881,7 @@ static void run_block_job(BlockJob *job, Error **errp) + } while (!job->ready && !job_is_completed(&job->job)); + + if (!job_is_completed(&job->job)) { +- ret = block_job_complete_sync(job, errp); ++ ret = job_complete_sync(&job->job, errp); + } else { + ret = job->job.ret; + } +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index b428aac..3600ffd 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -498,8 +498,7 @@ typedef struct TestBlockJob { + + static void test_job_completed(Job *job, void *opaque) + { +- BlockJob *bjob = container_of(job, BlockJob, job); +- block_job_completed(bjob, 0); ++ job_completed(job, 0); + } + + static void coroutine_fn test_job_start(void *opaque) +@@ -593,7 +592,7 @@ static void test_blockjob_common(enum drain_type drain_type) + g_assert_false(job->job.paused); + g_assert_false(job->job.busy); /* We're in job_sleep_ns() */ + +- ret = block_job_complete_sync(job, &error_abort); ++ ret = job_complete_sync(&job->job, &error_abort); + g_assert_cmpint(ret, ==, 0); + + blk_unref(blk_src); +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 6ee31d5..34ee179 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -34,7 +34,7 @@ static void test_block_job_complete(Job *job, void *opaque) + rc = -ECANCELED; + } + +- block_job_completed(bjob, rc); ++ job_completed(job, rc); + bdrv_unref(bs); + } + +@@ -130,7 +130,7 @@ static void test_single_job(int expected) + job_start(&job->job); + + if (expected == -ECANCELED) { +- block_job_cancel(job, false); ++ job_cancel(&job->job, false); + } + + while (result == -EINPROGRESS) { +@@ -176,10 +176,10 @@ static void test_pair_jobs(int expected1, int expected2) + job_txn_unref(txn); + + if (expected1 == -ECANCELED) { +- block_job_cancel(job1, false); ++ job_cancel(&job1->job, false); + } + if (expected2 == -ECANCELED) { +- block_job_cancel(job2, false); ++ job_cancel(&job2->job, false); + } + + while (result1 == -EINPROGRESS || result2 == -EINPROGRESS) { +@@ -232,13 +232,13 @@ static void test_pair_jobs_fail_cancel_race(void) + job_start(&job1->job); + job_start(&job2->job); + +- block_job_cancel(job1, false); ++ job_cancel(&job1->job, false); + + /* Now make job2 finish before the main loop kicks jobs. This simulates + * the race between a pending kick and another job completing. + */ +- block_job_enter(job2); +- block_job_enter(job2); ++ job_enter(&job2->job); ++ job_enter(&job2->job); + + while (result1 == -EINPROGRESS || result2 == -EINPROGRESS) { + aio_poll(qemu_get_aio_context(), true); +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 1e052c2..46a7873 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -165,10 +165,9 @@ typedef struct CancelJob { + + static void cancel_job_completed(Job *job, void *opaque) + { +- BlockJob *bjob = container_of(job, BlockJob, job); + CancelJob *s = opaque; + s->completed = true; +- block_job_completed(bjob, 0); ++ job_completed(job, 0); + } + + static void cancel_job_complete(Job *job, Error **errp) +@@ -232,7 +231,7 @@ static void cancel_common(CancelJob *s) + BlockBackend *blk = s->blk; + JobStatus sts = job->job.status; + +- block_job_cancel_sync(job); ++ job_cancel_sync(&job->job); + if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { + BlockJob *dummy = job; + block_job_dismiss(&dummy, &error_abort); +@@ -275,7 +274,7 @@ static void test_cancel_paused(void) + assert(job->job.status == JOB_STATUS_RUNNING); + + job_user_pause(&job->job, &error_abort); +- block_job_enter(job); ++ job_enter(&job->job); + assert(job->job.status == JOB_STATUS_PAUSED); + + cancel_common(s); +@@ -292,7 +291,7 @@ static void test_cancel_ready(void) + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- block_job_enter(job); ++ job_enter(&job->job); + assert(job->job.status == JOB_STATUS_READY); + + cancel_common(s); +@@ -309,11 +308,11 @@ static void test_cancel_standby(void) + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- block_job_enter(job); ++ job_enter(&job->job); + assert(job->job.status == JOB_STATUS_READY); + + job_user_pause(&job->job, &error_abort); +- block_job_enter(job); ++ job_enter(&job->job); + assert(job->job.status == JOB_STATUS_STANDBY); + + cancel_common(s); +@@ -330,11 +329,11 @@ static void test_cancel_pending(void) + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- block_job_enter(job); ++ job_enter(&job->job); + assert(job->job.status == JOB_STATUS_READY); + + job_complete(&job->job, &error_abort); +- block_job_enter(job); ++ job_enter(&job->job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); + } +@@ -354,11 +353,11 @@ static void test_cancel_concluded(void) + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- block_job_enter(job); ++ job_enter(&job->job); + assert(job->job.status == JOB_STATUS_READY); + + job_complete(&job->job, &error_abort); +- block_job_enter(job); ++ job_enter(&job->job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); + } +diff --git a/trace-events b/trace-events +index 2507e13..ef7579a 100644 +--- a/trace-events ++++ b/trace-events +@@ -107,6 +107,7 @@ gdbstub_err_checksum_incorrect(uint8_t expected, uint8_t got) "got command packe + # job.c + job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" + job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" ++job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" + + ### Guest events, keep at bottom + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-coroutine-and-related-code-to-Job.patch b/SOURCES/kvm-job-Move-coroutine-and-related-code-to-Job.patch new file mode 100644 index 0000000..3bd1916 --- /dev/null +++ b/SOURCES/kvm-job-Move-coroutine-and-related-code-to-Job.patch @@ -0,0 +1,1257 @@ +From 72dcce395974c2cc4ae59c9ba11243abf04f1bbd Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:09 +0200 +Subject: [PATCH 101/268] job: Move coroutine and related code to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-27-kwolf@redhat.com> +Patchwork-id: 81113 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 26/73] job: Move coroutine and related code to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This commit moves some core functions for dealing with the job coroutine +from BlockJob to Job. This includes primarily entering the coroutine +(both for the first and reentering) and yielding explicitly and at pause +points. + +Signed-off-by: Kevin Wolf +Reviewed-by: John Snow +(cherry picked from commit da01ff7f38f52791f93fc3ca59afcfbb220f15af) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + block/commit.c | 4 +- + block/mirror.c | 22 ++--- + block/replication.c | 2 +- + block/stream.c | 4 +- + blockdev.c | 8 +- + blockjob.c | 219 ++++++++----------------------------------- + include/block/blockjob.h | 40 -------- + include/block/blockjob_int.h | 26 ----- + include/qemu/job.h | 76 +++++++++++++++ + job.c | 137 +++++++++++++++++++++++++++ + tests/test-bdrv-drain.c | 38 ++++---- + tests/test-blockjob-txn.c | 12 +-- + tests/test-blockjob.c | 14 +-- + 14 files changed, 305 insertions(+), 299 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 22dd368..7d9aad9 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -528,8 +528,8 @@ static const BlockJobDriver backup_job_driver = { + .instance_size = sizeof(BackupBlockJob), + .job_type = JOB_TYPE_BACKUP, + .free = block_job_free, ++ .start = backup_run, + }, +- .start = backup_run, + .commit = backup_commit, + .abort = backup_abort, + .clean = backup_clean, +diff --git a/block/commit.c b/block/commit.c +index d326766..2fbc310 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -220,8 +220,8 @@ static const BlockJobDriver commit_job_driver = { + .instance_size = sizeof(CommitBlockJob), + .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, ++ .start = commit_run, + }, +- .start = commit_run, + }; + + static int coroutine_fn bdrv_commit_top_preadv(BlockDriverState *bs, +@@ -371,7 +371,7 @@ void commit_start(const char *job_id, BlockDriverState *bs, + s->on_error = on_error; + + trace_commit_start(bs, base, top, s); +- block_job_start(&s->common); ++ job_start(&s->common.job); + return; + + fail: +diff --git a/block/mirror.c b/block/mirror.c +index 90d4ac9..95fc807 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -126,7 +126,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret) + g_free(op); + + if (s->waiting_for_io) { +- qemu_coroutine_enter(s->common.co); ++ qemu_coroutine_enter(s->common.job.co); + } + } + +@@ -345,7 +345,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s) + mirror_wait_for_io(s); + } + +- block_job_pause_point(&s->common); ++ job_pause_point(&s->common.job); + + /* Find the number of consective dirty chunks following the first dirty + * one, and wait for in flight requests in them. */ +@@ -597,7 +597,7 @@ static void mirror_throttle(MirrorBlockJob *s) + s->last_pause_ns = now; + block_job_sleep_ns(&s->common, 0); + } else { +- block_job_pause_point(&s->common); ++ job_pause_point(&s->common.job); + } + } + +@@ -786,7 +786,7 @@ static void coroutine_fn mirror_run(void *opaque) + goto immediate_exit; + } + +- block_job_pause_point(&s->common); ++ job_pause_point(&s->common.job); + + cnt = bdrv_get_dirty_count(s->dirty_bitmap); + /* cnt is the number of dirty bytes remaining and s->bytes_in_flight is +@@ -957,9 +957,9 @@ static void mirror_complete(BlockJob *job, Error **errp) + block_job_enter(&s->common); + } + +-static void mirror_pause(BlockJob *job) ++static void mirror_pause(Job *job) + { +- MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); ++ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); + + mirror_wait_for_all_io(s); + } +@@ -991,10 +991,10 @@ static const BlockJobDriver mirror_job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = JOB_TYPE_MIRROR, + .free = block_job_free, ++ .start = mirror_run, ++ .pause = mirror_pause, + }, +- .start = mirror_run, + .complete = mirror_complete, +- .pause = mirror_pause, + .attached_aio_context = mirror_attached_aio_context, + .drain = mirror_drain, + }; +@@ -1004,10 +1004,10 @@ static const BlockJobDriver commit_active_job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, ++ .start = mirror_run, ++ .pause = mirror_pause, + }, +- .start = mirror_run, + .complete = mirror_complete, +- .pause = mirror_pause, + .attached_aio_context = mirror_attached_aio_context, + .drain = mirror_drain, + }; +@@ -1244,7 +1244,7 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs, + } + + trace_mirror_start(bs, s, opaque); +- block_job_start(&s->common); ++ job_start(&s->common.job); + return; + + fail: +diff --git a/block/replication.c b/block/replication.c +index 6c0c718..3f7500e 100644 +--- a/block/replication.c ++++ b/block/replication.c +@@ -574,7 +574,7 @@ static void replication_start(ReplicationState *rs, ReplicationMode mode, + aio_context_release(aio_context); + return; + } +- block_job_start(job); ++ job_start(&job->job); + break; + default: + aio_context_release(aio_context); +diff --git a/block/stream.c b/block/stream.c +index 0bba816..6d8b7b6 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -213,8 +213,8 @@ static const BlockJobDriver stream_job_driver = { + .instance_size = sizeof(StreamBlockJob), + .job_type = JOB_TYPE_STREAM, + .free = block_job_free, ++ .start = stream_run, + }, +- .start = stream_run, + }; + + void stream_start(const char *job_id, BlockDriverState *bs, +@@ -262,7 +262,7 @@ void stream_start(const char *job_id, BlockDriverState *bs, + + s->on_error = on_error; + trace_stream_start(bs, base, s); +- block_job_start(&s->common); ++ job_start(&s->common.job); + return; + + fail: +diff --git a/blockdev.c b/blockdev.c +index 19c04d9..93a4cdf 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -1911,7 +1911,7 @@ static void drive_backup_commit(BlkActionState *common) + aio_context_acquire(aio_context); + + assert(state->job); +- block_job_start(state->job); ++ job_start(&state->job->job); + + aio_context_release(aio_context); + } +@@ -2009,7 +2009,7 @@ static void blockdev_backup_commit(BlkActionState *common) + aio_context_acquire(aio_context); + + assert(state->job); +- block_job_start(state->job); ++ job_start(&state->job->job); + + aio_context_release(aio_context); + } +@@ -3426,7 +3426,7 @@ void qmp_drive_backup(DriveBackup *arg, Error **errp) + BlockJob *job; + job = do_drive_backup(arg, NULL, errp); + if (job) { +- block_job_start(job); ++ job_start(&job->job); + } + } + +@@ -3514,7 +3514,7 @@ void qmp_blockdev_backup(BlockdevBackup *arg, Error **errp) + BlockJob *job; + job = do_blockdev_backup(arg, NULL, errp); + if (job) { +- block_job_start(job); ++ job_start(&job->job); + } + } + +diff --git a/blockjob.c b/blockjob.c +index 3ede511..313b1ff 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -36,30 +36,9 @@ + #include "qemu/coroutine.h" + #include "qemu/timer.h" + +-/* Right now, this mutex is only needed to synchronize accesses to job->busy +- * and job->sleep_timer, such as concurrent calls to block_job_do_yield and +- * block_job_enter. */ +-static QemuMutex block_job_mutex; +- +-static void block_job_lock(void) +-{ +- qemu_mutex_lock(&block_job_mutex); +-} +- +-static void block_job_unlock(void) +-{ +- qemu_mutex_unlock(&block_job_mutex); +-} +- +-static void __attribute__((__constructor__)) block_job_init(void) +-{ +- qemu_mutex_init(&block_job_mutex); +-} +- + static void block_job_event_cancelled(BlockJob *job); + static void block_job_event_completed(BlockJob *job, const char *msg); + static int block_job_event_pending(BlockJob *job); +-static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job)); + + /* Transactional group of block jobs */ + struct BlockJobTxn { +@@ -161,33 +140,27 @@ static void block_job_txn_del_job(BlockJob *job) + } + } + +-/* Assumes the block_job_mutex is held */ +-static bool block_job_timer_pending(BlockJob *job) +-{ +- return timer_pending(&job->sleep_timer); +-} +- +-/* Assumes the block_job_mutex is held */ +-static bool block_job_timer_not_pending(BlockJob *job) ++/* Assumes the job_mutex is held */ ++static bool job_timer_not_pending(Job *job) + { +- return !block_job_timer_pending(job); ++ return !timer_pending(&job->sleep_timer); + } + + static void block_job_pause(BlockJob *job) + { +- job->pause_count++; ++ job->job.pause_count++; + } + + static void block_job_resume(BlockJob *job) + { +- assert(job->pause_count > 0); +- job->pause_count--; +- if (job->pause_count) { ++ assert(job->job.pause_count > 0); ++ job->job.pause_count--; ++ if (job->job.pause_count) { + return; + } + + /* kick only if no timer is pending */ +- block_job_enter_cond(job, block_job_timer_not_pending); ++ job_enter_cond(&job->job, job_timer_not_pending); + } + + static void block_job_attached_aio_context(AioContext *new_context, +@@ -208,7 +181,7 @@ void block_job_free(Job *job) + block_job_detach_aio_context, bjob); + blk_unref(bjob->blk); + error_free(bjob->blocker); +- assert(!timer_pending(&bjob->sleep_timer)); ++ assert(!timer_pending(&bjob->job.sleep_timer)); + } + + static void block_job_attached_aio_context(AioContext *new_context, +@@ -226,7 +199,7 @@ static void block_job_attached_aio_context(AioContext *new_context, + + static void block_job_drain(BlockJob *job) + { +- /* If job is !job->busy this kicks it into the next pause point. */ ++ /* If job is !job->job.busy this kicks it into the next pause point. */ + block_job_enter(job); + + blk_drain(job->blk); +@@ -244,7 +217,7 @@ static void block_job_detach_aio_context(void *opaque) + + block_job_pause(job); + +- while (!job->paused && !job->completed) { ++ while (!job->job.paused && !job->completed) { + block_job_drain(job); + } + +@@ -312,29 +285,11 @@ bool block_job_is_internal(BlockJob *job) + return (job->job.id == NULL); + } + +-static bool block_job_started(BlockJob *job) +-{ +- return job->co; +-} +- + const BlockJobDriver *block_job_driver(BlockJob *job) + { + return job->driver; + } + +-/** +- * All jobs must allow a pause point before entering their job proper. This +- * ensures that jobs can be paused prior to being started, then resumed later. +- */ +-static void coroutine_fn block_job_co_entry(void *opaque) +-{ +- BlockJob *job = opaque; +- +- assert(job && job->driver && job->driver->start); +- block_job_pause_point(job); +- job->driver->start(job); +-} +- + static void block_job_sleep_timer_cb(void *opaque) + { + BlockJob *job = opaque; +@@ -342,24 +297,12 @@ static void block_job_sleep_timer_cb(void *opaque) + block_job_enter(job); + } + +-void block_job_start(BlockJob *job) +-{ +- assert(job && !block_job_started(job) && job->paused && +- job->driver && job->driver->start); +- job->co = qemu_coroutine_create(block_job_co_entry, job); +- job->pause_count--; +- job->busy = true; +- job->paused = false; +- job_state_transition(&job->job, JOB_STATUS_RUNNING); +- bdrv_coroutine_enter(blk_bs(job->blk), job->co); +-} +- + static void block_job_decommission(BlockJob *job) + { + assert(job); + job->completed = true; +- job->busy = false; +- job->paused = false; ++ job->job.busy = false; ++ job->job.paused = false; + job->job.deferred_to_main_loop = true; + block_job_txn_del_job(job); + job_state_transition(&job->job, JOB_STATUS_NULL); +@@ -374,7 +317,7 @@ static void block_job_do_dismiss(BlockJob *job) + static void block_job_conclude(BlockJob *job) + { + job_state_transition(&job->job, JOB_STATUS_CONCLUDED); +- if (job->auto_dismiss || !block_job_started(job)) { ++ if (job->auto_dismiss || !job_started(&job->job)) { + block_job_do_dismiss(job); + } + } +@@ -439,7 +382,7 @@ static int block_job_finalize_single(BlockJob *job) + } + + /* Emit events only if we actually started */ +- if (block_job_started(job)) { ++ if (job_started(&job->job)) { + if (job_is_cancelled(&job->job)) { + block_job_event_cancelled(job); + } else { +@@ -464,7 +407,7 @@ static void block_job_cancel_async(BlockJob *job, bool force) + if (job->user_paused) { + /* Do not call block_job_enter here, the caller will handle it. */ + job->user_paused = false; +- job->pause_count--; ++ job->job.pause_count--; + } + job->job.cancelled = true; + /* To prevent 'force == false' overriding a previous 'force == true' */ +@@ -615,6 +558,12 @@ static void block_job_completed_txn_success(BlockJob *job) + } + } + ++/* Assumes the job_mutex is held */ ++static bool job_timer_pending(Job *job) ++{ ++ return timer_pending(&job->sleep_timer); ++} ++ + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) + { + int64_t old_speed = job->speed; +@@ -635,7 +584,7 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) + } + + /* kick only if a timer is pending */ +- block_job_enter_cond(job, block_job_timer_pending); ++ job_enter_cond(&job->job, job_timer_pending); + } + + int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) +@@ -654,7 +603,7 @@ void block_job_complete(BlockJob *job, Error **errp) + if (job_apply_verb(&job->job, JOB_VERB_COMPLETE, errp)) { + return; + } +- if (job->pause_count || job_is_cancelled(&job->job) || ++ if (job->job.pause_count || job_is_cancelled(&job->job) || + !job->driver->complete) + { + error_setg(errp, "The active block job '%s' cannot be completed", +@@ -708,7 +657,7 @@ bool block_job_user_paused(BlockJob *job) + void block_job_user_resume(BlockJob *job, Error **errp) + { + assert(job); +- if (!job->user_paused || job->pause_count <= 0) { ++ if (!job->user_paused || job->job.pause_count <= 0) { + error_setg(errp, "Can't resume a job that was not paused"); + return; + } +@@ -727,7 +676,7 @@ void block_job_cancel(BlockJob *job, bool force) + return; + } + block_job_cancel_async(job, force); +- if (!block_job_started(job)) { ++ if (!job_started(&job->job)) { + block_job_completed(job, -ECANCELED); + } else if (job->job.deferred_to_main_loop) { + block_job_completed_txn_abort(job); +@@ -797,8 +746,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info->type = g_strdup(job_type_str(&job->job)); + info->device = g_strdup(job->job.id); + info->len = job->len; +- info->busy = atomic_read(&job->busy); +- info->paused = job->pause_count > 0; ++ info->busy = atomic_read(&job->job.busy); ++ info->paused = job->job.pause_count > 0; + info->offset = job->offset; + info->speed = job->speed; + info->io_status = job->iostatus; +@@ -915,12 +864,9 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->blk = blk; + job->cb = cb; + job->opaque = opaque; +- job->busy = false; +- job->paused = true; +- job->pause_count = 1; + job->auto_finalize = !(flags & BLOCK_JOB_MANUAL_FINALIZE); + job->auto_dismiss = !(flags & BLOCK_JOB_MANUAL_DISMISS); +- aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, ++ aio_timer_init(qemu_get_aio_context(), &job->job.sleep_timer, + QEMU_CLOCK_REALTIME, SCALE_NS, + block_job_sleep_timer_cb, job); + +@@ -980,128 +926,41 @@ void block_job_completed(BlockJob *job, int ret) + } + } + +-static bool block_job_should_pause(BlockJob *job) +-{ +- return job->pause_count > 0; +-} +- +-/* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. +- * Reentering the job coroutine with block_job_enter() before the timer has +- * expired is allowed and cancels the timer. +- * +- * If @ns is (uint64_t) -1, no timer is scheduled and block_job_enter() must be +- * called explicitly. */ +-static void block_job_do_yield(BlockJob *job, uint64_t ns) +-{ +- block_job_lock(); +- if (ns != -1) { +- timer_mod(&job->sleep_timer, ns); +- } +- job->busy = false; +- block_job_unlock(); +- qemu_coroutine_yield(); +- +- /* Set by block_job_enter before re-entering the coroutine. */ +- assert(job->busy); +-} +- +-void coroutine_fn block_job_pause_point(BlockJob *job) +-{ +- assert(job && block_job_started(job)); +- +- if (!block_job_should_pause(job)) { +- return; +- } +- if (job_is_cancelled(&job->job)) { +- return; +- } +- +- if (job->driver->pause) { +- job->driver->pause(job); +- } +- +- if (block_job_should_pause(job) && !job_is_cancelled(&job->job)) { +- JobStatus status = job->job.status; +- job_state_transition(&job->job, status == JOB_STATUS_READY +- ? JOB_STATUS_STANDBY +- : JOB_STATUS_PAUSED); +- job->paused = true; +- block_job_do_yield(job, -1); +- job->paused = false; +- job_state_transition(&job->job, status); +- } +- +- if (job->driver->resume) { +- job->driver->resume(job); +- } +-} +- +-/* +- * Conditionally enter a block_job pending a call to fn() while +- * under the block_job_lock critical section. +- */ +-static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job)) +-{ +- if (!block_job_started(job)) { +- return; +- } +- if (job->job.deferred_to_main_loop) { +- return; +- } +- +- block_job_lock(); +- if (job->busy) { +- block_job_unlock(); +- return; +- } +- +- if (fn && !fn(job)) { +- block_job_unlock(); +- return; +- } +- +- assert(!job->job.deferred_to_main_loop); +- timer_del(&job->sleep_timer); +- job->busy = true; +- block_job_unlock(); +- aio_co_wake(job->co); +-} +- + void block_job_enter(BlockJob *job) + { +- block_job_enter_cond(job, NULL); ++ job_enter_cond(&job->job, NULL); + } + + void block_job_sleep_ns(BlockJob *job, int64_t ns) + { +- assert(job->busy); ++ assert(job->job.busy); + + /* Check cancellation *before* setting busy = false, too! */ + if (job_is_cancelled(&job->job)) { + return; + } + +- if (!block_job_should_pause(job)) { +- block_job_do_yield(job, qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + ns); ++ if (!job_should_pause(&job->job)) { ++ job_do_yield(&job->job, qemu_clock_get_ns(QEMU_CLOCK_REALTIME) + ns); + } + +- block_job_pause_point(job); ++ job_pause_point(&job->job); + } + + void block_job_yield(BlockJob *job) + { +- assert(job->busy); ++ assert(job->job.busy); + + /* Check cancellation *before* setting busy = false, too! */ + if (job_is_cancelled(&job->job)) { + return; + } + +- if (!block_job_should_pause(job)) { +- block_job_do_yield(job, -1); ++ if (!job_should_pause(&job->job)) { ++ job_do_yield(&job->job, -1); + } + +- block_job_pause_point(job); ++ job_pause_point(&job->job); + } + + void block_job_iostatus_reset(BlockJob *job) +@@ -1109,7 +968,7 @@ void block_job_iostatus_reset(BlockJob *job) + if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) { + return; + } +- assert(job->user_paused && job->pause_count > 0); ++ assert(job->user_paused && job->job.pause_count > 0); + job->iostatus = BLOCK_DEVICE_IO_STATUS_OK; + } + +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 2a9e865..b60d919 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -51,43 +51,18 @@ typedef struct BlockJob { + BlockBackend *blk; + + /** +- * The coroutine that executes the job. If not NULL, it is +- * reentered when busy is false and the job is cancelled. +- */ +- Coroutine *co; +- +- /** + * Set to true if the job should abort immediately without waiting + * for data to be in sync. + */ + bool force; + + /** +- * Counter for pause request. If non-zero, the block job is either paused, +- * or if busy == true will pause itself as soon as possible. +- */ +- int pause_count; +- +- /** + * Set to true if the job is paused by user. Can be unpaused with the + * block-job-resume QMP command. + */ + bool user_paused; + + /** +- * Set to false by the job while the coroutine has yielded and may be +- * re-entered by block_job_enter(). There may still be I/O or event loop +- * activity pending. Accessed under block_job_mutex (in blockjob.c). +- */ +- bool busy; +- +- /** +- * Set to true by the job while it is in a quiescent state, where +- * no I/O or event loop activity is pending. +- */ +- bool paused; +- +- /** + * Set to true when the job is ready to be completed. + */ + bool ready; +@@ -125,12 +100,6 @@ typedef struct BlockJob { + /** ret code passed to block_job_completed. */ + int ret; + +- /** +- * Timer that is used by @block_job_sleep_ns. Accessed under +- * block_job_mutex (in blockjob.c). +- */ +- QEMUTimer sleep_timer; +- + /** True if this job should automatically finalize itself */ + bool auto_finalize; + +@@ -208,15 +177,6 @@ void block_job_remove_all_bdrv(BlockJob *job); + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); + + /** +- * block_job_start: +- * @job: A job that has not yet been started. +- * +- * Begins execution of a block job. +- * Takes ownership of one reference to the job object. +- */ +-void block_job_start(BlockJob *job); +- +-/** + * block_job_cancel: + * @job: The job to be canceled. + * @force: Quit a job without waiting for data to be in sync. +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 0c2f8de..0a614a8 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -38,9 +38,6 @@ struct BlockJobDriver { + /** Generic JobDriver callbacks and settings */ + JobDriver job_driver; + +- /** Mandatory: Entrypoint for the Coroutine. */ +- CoroutineEntry *start; +- + /** + * Optional callback for job types whose completion must be triggered + * manually. +@@ -85,20 +82,6 @@ struct BlockJobDriver { + */ + void (*clean)(BlockJob *job); + +- /** +- * If the callback is not NULL, it will be invoked when the job transitions +- * into the paused state. Paused jobs must not perform any asynchronous +- * I/O or event loop activity. This callback is used to quiesce jobs. +- */ +- void coroutine_fn (*pause)(BlockJob *job); +- +- /** +- * If the callback is not NULL, it will be invoked when the job transitions +- * out of the paused state. Any asynchronous I/O or event loop activity +- * should be restarted from this callback. +- */ +- void coroutine_fn (*resume)(BlockJob *job); +- + /* + * If the callback is not NULL, it will be invoked before the job is + * resumed in a new AioContext. This is the place to move any resources +@@ -196,15 +179,6 @@ void block_job_early_fail(BlockJob *job); + void block_job_completed(BlockJob *job, int ret); + + /** +- * block_job_pause_point: +- * @job: The job that is ready to pause. +- * +- * Pause now if block_job_pause() has been called. Block jobs that perform +- * lots of I/O must call this between requests so that the job can be paused. +- */ +-void coroutine_fn block_job_pause_point(BlockJob *job); +- +-/** + * block_job_enter: + * @job: The job to enter. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 933e0ab..9dcff12 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -28,6 +28,7 @@ + + #include "qapi/qapi-types-block-core.h" + #include "qemu/queue.h" ++#include "qemu/coroutine.h" + + typedef struct JobDriver JobDriver; + +@@ -51,6 +52,37 @@ typedef struct Job { + AioContext *aio_context; + + /** ++ * The coroutine that executes the job. If not NULL, it is reentered when ++ * busy is false and the job is cancelled. ++ */ ++ Coroutine *co; ++ ++ /** ++ * Timer that is used by @block_job_sleep_ns. Accessed under job_mutex (in ++ * job.c). ++ */ ++ QEMUTimer sleep_timer; ++ ++ /** ++ * Counter for pause request. If non-zero, the block job is either paused, ++ * or if busy == true will pause itself as soon as possible. ++ */ ++ int pause_count; ++ ++ /** ++ * Set to false by the job while the coroutine has yielded and may be ++ * re-entered by block_job_enter(). There may still be I/O or event loop ++ * activity pending. Accessed under block_job_mutex (in blockjob.c). ++ */ ++ bool busy; ++ ++ /** ++ * Set to true by the job while it is in a quiescent state, where ++ * no I/O or event loop activity is pending. ++ */ ++ bool paused; ++ ++ /** + * Set to true if the job should cancel itself. The flag must + * always be tested just before toggling the busy flag from false + * to true. After a job has been cancelled, it should only yield +@@ -75,6 +107,23 @@ struct JobDriver { + /** Enum describing the operation */ + JobType job_type; + ++ /** Mandatory: Entrypoint for the Coroutine. */ ++ CoroutineEntry *start; ++ ++ /** ++ * If the callback is not NULL, it will be invoked when the job transitions ++ * into the paused state. Paused jobs must not perform any asynchronous ++ * I/O or event loop activity. This callback is used to quiesce jobs. ++ */ ++ void coroutine_fn (*pause)(Job *job); ++ ++ /** ++ * If the callback is not NULL, it will be invoked when the job transitions ++ * out of the paused state. Any asynchronous I/O or event loop activity ++ * should be restarted from this callback. ++ */ ++ void coroutine_fn (*resume)(Job *job); ++ + /** Called when the job is freed */ + void (*free)(Job *job); + }; +@@ -103,6 +152,30 @@ void job_ref(Job *job); + */ + void job_unref(Job *job); + ++/** ++ * Conditionally enter the job coroutine if the job is ready to run, not ++ * already busy and fn() returns true. fn() is called while under the job_lock ++ * critical section. ++ */ ++void job_enter_cond(Job *job, bool(*fn)(Job *job)); ++ ++/** ++ * @job: A job that has not yet been started. ++ * ++ * Begins execution of a job. ++ * Takes ownership of one reference to the job object. ++ */ ++void job_start(Job *job); ++ ++/** ++ * @job: The job that is ready to pause. ++ * ++ * Pause now if job_pause() has been called. Jobs that perform lots of I/O ++ * must call this between requests so that the job can be paused. ++ */ ++void coroutine_fn job_pause_point(Job *job); ++ ++ + /** Returns the JobType of a given Job. */ + JobType job_type(const Job *job); + +@@ -153,5 +226,8 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque); + + /* TODO To be removed from the public interface */ + void job_state_transition(Job *job, JobStatus s1); ++void coroutine_fn job_do_yield(Job *job, uint64_t ns); ++bool job_should_pause(Job *job); ++bool job_started(Job *job); + + #endif +diff --git a/job.c b/job.c +index c5a37fb..78497fd 100644 +--- a/job.c ++++ b/job.c +@@ -60,6 +60,26 @@ bool JobVerbTable[JOB_VERB__MAX][JOB_STATUS__MAX] = { + [JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, + }; + ++/* Right now, this mutex is only needed to synchronize accesses to job->busy ++ * and job->sleep_timer, such as concurrent calls to job_do_yield and ++ * job_enter. */ ++static QemuMutex job_mutex; ++ ++static void job_lock(void) ++{ ++ qemu_mutex_lock(&job_mutex); ++} ++ ++static void job_unlock(void) ++{ ++ qemu_mutex_unlock(&job_mutex); ++} ++ ++static void __attribute__((__constructor__)) job_init(void) ++{ ++ qemu_mutex_init(&job_mutex); ++} ++ + /* TODO Make static once the whole state machine is in job.c */ + void job_state_transition(Job *job, JobStatus s1) + { +@@ -101,6 +121,16 @@ bool job_is_cancelled(Job *job) + return job->cancelled; + } + ++bool job_started(Job *job) ++{ ++ return job->co; ++} ++ ++bool job_should_pause(Job *job) ++{ ++ return job->pause_count > 0; ++} ++ + Job *job_next(Job *job) + { + if (!job) { +@@ -143,6 +173,9 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + job->id = g_strdup(job_id); + job->refcnt = 1; + job->aio_context = ctx; ++ job->busy = false; ++ job->paused = true; ++ job->pause_count = 1; + + job_state_transition(job, JOB_STATUS_CREATED); + +@@ -172,6 +205,110 @@ void job_unref(Job *job) + } + } + ++void job_enter_cond(Job *job, bool(*fn)(Job *job)) ++{ ++ if (!job_started(job)) { ++ return; ++ } ++ if (job->deferred_to_main_loop) { ++ return; ++ } ++ ++ job_lock(); ++ if (job->busy) { ++ job_unlock(); ++ return; ++ } ++ ++ if (fn && !fn(job)) { ++ job_unlock(); ++ return; ++ } ++ ++ assert(!job->deferred_to_main_loop); ++ timer_del(&job->sleep_timer); ++ job->busy = true; ++ job_unlock(); ++ aio_co_wake(job->co); ++} ++ ++/* Yield, and schedule a timer to reenter the coroutine after @ns nanoseconds. ++ * Reentering the job coroutine with block_job_enter() before the timer has ++ * expired is allowed and cancels the timer. ++ * ++ * If @ns is (uint64_t) -1, no timer is scheduled and block_job_enter() must be ++ * called explicitly. */ ++void coroutine_fn job_do_yield(Job *job, uint64_t ns) ++{ ++ job_lock(); ++ if (ns != -1) { ++ timer_mod(&job->sleep_timer, ns); ++ } ++ job->busy = false; ++ job_unlock(); ++ qemu_coroutine_yield(); ++ ++ /* Set by job_enter_cond() before re-entering the coroutine. */ ++ assert(job->busy); ++} ++ ++void coroutine_fn job_pause_point(Job *job) ++{ ++ assert(job && job_started(job)); ++ ++ if (!job_should_pause(job)) { ++ return; ++ } ++ if (job_is_cancelled(job)) { ++ return; ++ } ++ ++ if (job->driver->pause) { ++ job->driver->pause(job); ++ } ++ ++ if (job_should_pause(job) && !job_is_cancelled(job)) { ++ JobStatus status = job->status; ++ job_state_transition(job, status == JOB_STATUS_READY ++ ? JOB_STATUS_STANDBY ++ : JOB_STATUS_PAUSED); ++ job->paused = true; ++ job_do_yield(job, -1); ++ job->paused = false; ++ job_state_transition(job, status); ++ } ++ ++ if (job->driver->resume) { ++ job->driver->resume(job); ++ } ++} ++ ++/** ++ * All jobs must allow a pause point before entering their job proper. This ++ * ensures that jobs can be paused prior to being started, then resumed later. ++ */ ++static void coroutine_fn job_co_entry(void *opaque) ++{ ++ Job *job = opaque; ++ ++ assert(job && job->driver && job->driver->start); ++ job_pause_point(job); ++ job->driver->start(job); ++} ++ ++ ++void job_start(Job *job) ++{ ++ assert(job && !job_started(job) && job->paused && ++ job->driver && job->driver->start); ++ job->co = qemu_coroutine_create(job_co_entry, job); ++ job->pause_count--; ++ job->busy = true; ++ job->paused = false; ++ job_state_transition(job, JOB_STATUS_RUNNING); ++ aio_co_enter(job->aio_context, job->co); ++} ++ + typedef struct { + Job *job; + JobDeferToMainLoopFn *fn; +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 4f8cba8..c9f2f9b 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -524,8 +524,8 @@ BlockJobDriver test_job_driver = { + .job_driver = { + .instance_size = sizeof(TestBlockJob), + .free = block_job_free, ++ .start = test_job_start, + }, +- .start = test_job_start, + .complete = test_job_complete, + }; + +@@ -549,47 +549,47 @@ static void test_blockjob_common(enum drain_type drain_type) + job = block_job_create("job0", &test_job_driver, NULL, src, 0, BLK_PERM_ALL, + 0, 0, NULL, NULL, &error_abort); + block_job_add_bdrv(job, "target", target, 0, BLK_PERM_ALL, &error_abort); +- block_job_start(job); ++ job_start(&job->job); + +- g_assert_cmpint(job->pause_count, ==, 0); +- g_assert_false(job->paused); +- g_assert_false(job->busy); /* We're in block_job_sleep_ns() */ ++ g_assert_cmpint(job->job.pause_count, ==, 0); ++ g_assert_false(job->job.paused); ++ g_assert_false(job->job.busy); /* We're in block_job_sleep_ns() */ + + do_drain_begin(drain_type, src); + + if (drain_type == BDRV_DRAIN_ALL) { + /* bdrv_drain_all() drains both src and target */ +- g_assert_cmpint(job->pause_count, ==, 2); ++ g_assert_cmpint(job->job.pause_count, ==, 2); + } else { +- g_assert_cmpint(job->pause_count, ==, 1); ++ g_assert_cmpint(job->job.pause_count, ==, 1); + } + /* XXX We don't wait until the job is actually paused. Is this okay? */ +- /* g_assert_true(job->paused); */ +- g_assert_false(job->busy); /* The job is paused */ ++ /* g_assert_true(job->job.paused); */ ++ g_assert_false(job->job.busy); /* The job is paused */ + + do_drain_end(drain_type, src); + +- g_assert_cmpint(job->pause_count, ==, 0); +- g_assert_false(job->paused); +- g_assert_false(job->busy); /* We're in block_job_sleep_ns() */ ++ g_assert_cmpint(job->job.pause_count, ==, 0); ++ g_assert_false(job->job.paused); ++ g_assert_false(job->job.busy); /* We're in block_job_sleep_ns() */ + + do_drain_begin(drain_type, target); + + if (drain_type == BDRV_DRAIN_ALL) { + /* bdrv_drain_all() drains both src and target */ +- g_assert_cmpint(job->pause_count, ==, 2); ++ g_assert_cmpint(job->job.pause_count, ==, 2); + } else { +- g_assert_cmpint(job->pause_count, ==, 1); ++ g_assert_cmpint(job->job.pause_count, ==, 1); + } + /* XXX We don't wait until the job is actually paused. Is this okay? */ +- /* g_assert_true(job->paused); */ +- g_assert_false(job->busy); /* The job is paused */ ++ /* g_assert_true(job->job.paused); */ ++ g_assert_false(job->job.busy); /* The job is paused */ + + do_drain_end(drain_type, target); + +- g_assert_cmpint(job->pause_count, ==, 0); +- g_assert_false(job->paused); +- g_assert_false(job->busy); /* We're in block_job_sleep_ns() */ ++ g_assert_cmpint(job->job.pause_count, ==, 0); ++ g_assert_false(job->job.paused); ++ g_assert_false(job->job.busy); /* We're in block_job_sleep_ns() */ + + ret = block_job_complete_sync(job, &error_abort); + g_assert_cmpint(ret, ==, 0); +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index c03f966..323e154 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -78,8 +78,8 @@ static const BlockJobDriver test_block_job_driver = { + .job_driver = { + .instance_size = sizeof(TestBlockJob), + .free = block_job_free, ++ .start = test_block_job_run, + }, +- .start = test_block_job_run, + }; + + /* Create a block job that completes with a given return code after a given +@@ -125,7 +125,7 @@ static void test_single_job(int expected) + + txn = block_job_txn_new(); + job = test_block_job_start(1, true, expected, &result, txn); +- block_job_start(job); ++ job_start(&job->job); + + if (expected == -ECANCELED) { + block_job_cancel(job, false); +@@ -165,8 +165,8 @@ static void test_pair_jobs(int expected1, int expected2) + txn = block_job_txn_new(); + job1 = test_block_job_start(1, true, expected1, &result1, txn); + job2 = test_block_job_start(2, true, expected2, &result2, txn); +- block_job_start(job1); +- block_job_start(job2); ++ job_start(&job1->job); ++ job_start(&job2->job); + + /* Release our reference now to trigger as many nice + * use-after-free bugs as possible. +@@ -227,8 +227,8 @@ static void test_pair_jobs_fail_cancel_race(void) + txn = block_job_txn_new(); + job1 = test_block_job_start(1, true, -ECANCELED, &result1, txn); + job2 = test_block_job_start(2, false, 0, &result2, txn); +- block_job_start(job1); +- block_job_start(job2); ++ job_start(&job1->job); ++ job_start(&job2->job); + + block_job_cancel(job1, false); + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 5f43bd7..1d18325 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -199,8 +199,8 @@ static const BlockJobDriver test_cancel_driver = { + .job_driver = { + .instance_size = sizeof(CancelJob), + .free = block_job_free, ++ .start = cancel_job_start, + }, +- .start = cancel_job_start, + .complete = cancel_job_complete, + }; + +@@ -254,7 +254,7 @@ static void test_cancel_running(void) + + s = create_common(&job); + +- block_job_start(job); ++ job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + + cancel_common(s); +@@ -267,7 +267,7 @@ static void test_cancel_paused(void) + + s = create_common(&job); + +- block_job_start(job); ++ job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + + block_job_user_pause(job, &error_abort); +@@ -284,7 +284,7 @@ static void test_cancel_ready(void) + + s = create_common(&job); + +- block_job_start(job); ++ job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +@@ -301,7 +301,7 @@ static void test_cancel_standby(void) + + s = create_common(&job); + +- block_job_start(job); ++ job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +@@ -322,7 +322,7 @@ static void test_cancel_pending(void) + + s = create_common(&job); + +- block_job_start(job); ++ job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +@@ -346,7 +346,7 @@ static void test_cancel_concluded(void) + + s = create_common(&job); + +- block_job_start(job); ++ job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-defer_to_main_loop-to-Job.patch b/SOURCES/kvm-job-Move-defer_to_main_loop-to-Job.patch new file mode 100644 index 0000000..c198039 --- /dev/null +++ b/SOURCES/kvm-job-Move-defer_to_main_loop-to-Job.patch @@ -0,0 +1,535 @@ +From 8096c2b37856e96d3eea5dc96537c02f35c534f8 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:08 +0200 +Subject: [PATCH 100/268] job: Move defer_to_main_loop to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-26-kwolf@redhat.com> +Patchwork-id: 81091 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 25/73] job: Move defer_to_main_loop to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Move the defer_to_main_loop functionality from BlockJob to Job. + +The code can be simplified because we can use job->aio_context in +job_defer_to_main_loop_bh() now, instead of having to access the +BlockDriverState. + +Probably taking the data->aio_context lock in addition was already +unnecessary in the old code because we didn't actually make use of +anything protected by the old AioContext except getting the new +AioContext, in case it changed between scheduling the BH and running it. +But it's certainly unnecessary now that the BDS isn't accessed at all +any more. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 1908a5590c7d214b1b6886bc19b81076fb65cec9) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 7 +++--- + block/commit.c | 11 +++++---- + block/mirror.c | 15 ++++++------ + block/stream.c | 14 +++++------ + blockjob.c | 57 ++++---------------------------------------- + include/block/blockjob.h | 5 ---- + include/block/blockjob_int.h | 19 --------------- + include/qemu/job.h | 20 ++++++++++++++++ + job.c | 32 +++++++++++++++++++++++++ + tests/test-bdrv-drain.c | 7 +++--- + tests/test-blockjob-txn.c | 13 +++++----- + tests/test-blockjob.c | 7 +++--- + 12 files changed, 97 insertions(+), 110 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index ef0aa0e..22dd368 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -317,11 +317,12 @@ typedef struct { + int ret; + } BackupCompleteData; + +-static void backup_complete(BlockJob *job, void *opaque) ++static void backup_complete(Job *job, void *opaque) + { ++ BlockJob *bjob = container_of(job, BlockJob, job); + BackupCompleteData *data = opaque; + +- block_job_completed(job, data->ret); ++ block_job_completed(bjob, data->ret); + g_free(data); + } + +@@ -519,7 +520,7 @@ static void coroutine_fn backup_run(void *opaque) + + data = g_malloc(sizeof(*data)); + data->ret = ret; +- block_job_defer_to_main_loop(&job->common, backup_complete, data); ++ job_defer_to_main_loop(&job->common.job, backup_complete, data); + } + + static const BlockJobDriver backup_job_driver = { +diff --git a/block/commit.c b/block/commit.c +index 85baea8..d326766 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -72,9 +72,10 @@ typedef struct { + int ret; + } CommitCompleteData; + +-static void commit_complete(BlockJob *job, void *opaque) ++static void commit_complete(Job *job, void *opaque) + { +- CommitBlockJob *s = container_of(job, CommitBlockJob, common); ++ CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); ++ BlockJob *bjob = &s->common; + CommitCompleteData *data = opaque; + BlockDriverState *top = blk_bs(s->top); + BlockDriverState *base = blk_bs(s->base); +@@ -90,7 +91,7 @@ static void commit_complete(BlockJob *job, void *opaque) + * the normal backing chain can be restored. */ + blk_unref(s->base); + +- if (!job_is_cancelled(&s->common.job) && ret == 0) { ++ if (!job_is_cancelled(job) && ret == 0) { + /* success */ + ret = bdrv_drop_intermediate(s->commit_top_bs, base, + s->backing_file_str); +@@ -114,7 +115,7 @@ static void commit_complete(BlockJob *job, void *opaque) + * block_job_finish_sync()), block_job_completed() won't free it and + * therefore the blockers on the intermediate nodes remain. This would + * cause bdrv_set_backing_hd() to fail. */ +- block_job_remove_all_bdrv(job); ++ block_job_remove_all_bdrv(bjob); + + block_job_completed(&s->common, ret); + g_free(data); +@@ -211,7 +212,7 @@ out: + + data = g_malloc(sizeof(*data)); + data->ret = ret; +- block_job_defer_to_main_loop(&s->common, commit_complete, data); ++ job_defer_to_main_loop(&s->common.job, commit_complete, data); + } + + static const BlockJobDriver commit_job_driver = { +diff --git a/block/mirror.c b/block/mirror.c +index 424072e..90d4ac9 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -484,9 +484,10 @@ typedef struct { + int ret; + } MirrorExitData; + +-static void mirror_exit(BlockJob *job, void *opaque) ++static void mirror_exit(Job *job, void *opaque) + { +- MirrorBlockJob *s = container_of(job, MirrorBlockJob, common); ++ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); ++ BlockJob *bjob = &s->common; + MirrorExitData *data = opaque; + AioContext *replace_aio_context = NULL; + BlockDriverState *src = s->source; +@@ -568,7 +569,7 @@ static void mirror_exit(BlockJob *job, void *opaque) + * the blockers on the intermediate nodes so that the resulting state is + * valid. Also give up permissions on mirror_top_bs->backing, which might + * block the removal. */ +- block_job_remove_all_bdrv(job); ++ block_job_remove_all_bdrv(bjob); + bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, + &error_abort); + bdrv_replace_node(mirror_top_bs, backing_bs(mirror_top_bs), &error_abort); +@@ -576,9 +577,9 @@ static void mirror_exit(BlockJob *job, void *opaque) + /* We just changed the BDS the job BB refers to (with either or both of the + * bdrv_replace_node() calls), so switch the BB back so the cleanup does + * the right thing. We don't need any permissions any more now. */ +- blk_remove_bs(job->blk); +- blk_set_perm(job->blk, 0, BLK_PERM_ALL, &error_abort); +- blk_insert_bs(job->blk, mirror_top_bs, &error_abort); ++ blk_remove_bs(bjob->blk); ++ blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); ++ blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort); + + block_job_completed(&s->common, data->ret); + +@@ -901,7 +902,7 @@ immediate_exit: + if (need_drain) { + bdrv_drained_begin(bs); + } +- block_job_defer_to_main_loop(&s->common, mirror_exit, data); ++ job_defer_to_main_loop(&s->common.job, mirror_exit, data); + } + + static void mirror_complete(BlockJob *job, Error **errp) +diff --git a/block/stream.c b/block/stream.c +index 22c71ae..0bba816 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -58,16 +58,16 @@ typedef struct { + int ret; + } StreamCompleteData; + +-static void stream_complete(BlockJob *job, void *opaque) ++static void stream_complete(Job *job, void *opaque) + { +- StreamBlockJob *s = container_of(job, StreamBlockJob, common); ++ StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); ++ BlockJob *bjob = &s->common; + StreamCompleteData *data = opaque; +- BlockDriverState *bs = blk_bs(job->blk); ++ BlockDriverState *bs = blk_bs(bjob->blk); + BlockDriverState *base = s->base; + Error *local_err = NULL; + +- if (!job_is_cancelled(&s->common.job) && bs->backing && +- data->ret == 0) { ++ if (!job_is_cancelled(job) && bs->backing && data->ret == 0) { + const char *base_id = NULL, *base_fmt = NULL; + if (base) { + base_id = s->backing_file_str; +@@ -88,7 +88,7 @@ out: + /* Reopen the image back in read-only mode if necessary */ + if (s->bs_flags != bdrv_get_flags(bs)) { + /* Give up write permissions before making it read-only */ +- blk_set_perm(job->blk, 0, BLK_PERM_ALL, &error_abort); ++ blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); + bdrv_reopen(bs, s->bs_flags, NULL); + } + +@@ -205,7 +205,7 @@ out: + /* Modify backing chain and close BDSes in main loop */ + data = g_malloc(sizeof(*data)); + data->ret = ret; +- block_job_defer_to_main_loop(&s->common, stream_complete, data); ++ job_defer_to_main_loop(&s->common.job, stream_complete, data); + } + + static const BlockJobDriver stream_job_driver = { +diff --git a/blockjob.c b/blockjob.c +index 0a0b1c4..3ede511 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -360,7 +360,7 @@ static void block_job_decommission(BlockJob *job) + job->completed = true; + job->busy = false; + job->paused = false; +- job->deferred_to_main_loop = true; ++ job->job.deferred_to_main_loop = true; + block_job_txn_del_job(job); + job_state_transition(&job->job, JOB_STATUS_NULL); + job_unref(&job->job); +@@ -515,7 +515,7 @@ static int block_job_finish_sync(BlockJob *job, + /* block_job_drain calls block_job_enter, and it should be enough to + * induce progress until the job completes or moves to the main thread. + */ +- while (!job->deferred_to_main_loop && !job->completed) { ++ while (!job->job.deferred_to_main_loop && !job->completed) { + block_job_drain(job); + } + while (!job->completed) { +@@ -729,7 +729,7 @@ void block_job_cancel(BlockJob *job, bool force) + block_job_cancel_async(job, force); + if (!block_job_started(job)) { + block_job_completed(job, -ECANCELED); +- } else if (job->deferred_to_main_loop) { ++ } else if (job->job.deferred_to_main_loop) { + block_job_completed_txn_abort(job); + } else { + block_job_enter(job); +@@ -1045,7 +1045,7 @@ static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job)) + if (!block_job_started(job)) { + return; + } +- if (job->deferred_to_main_loop) { ++ if (job->job.deferred_to_main_loop) { + return; + } + +@@ -1060,7 +1060,7 @@ static void block_job_enter_cond(BlockJob *job, bool(*fn)(BlockJob *job)) + return; + } + +- assert(!job->deferred_to_main_loop); ++ assert(!job->job.deferred_to_main_loop); + timer_del(&job->sleep_timer); + job->busy = true; + block_job_unlock(); +@@ -1166,50 +1166,3 @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, + } + return action; + } +- +-typedef struct { +- BlockJob *job; +- AioContext *aio_context; +- BlockJobDeferToMainLoopFn *fn; +- void *opaque; +-} BlockJobDeferToMainLoopData; +- +-static void block_job_defer_to_main_loop_bh(void *opaque) +-{ +- BlockJobDeferToMainLoopData *data = opaque; +- AioContext *aio_context; +- +- /* Prevent race with block_job_defer_to_main_loop() */ +- aio_context_acquire(data->aio_context); +- +- /* Fetch BDS AioContext again, in case it has changed */ +- aio_context = blk_get_aio_context(data->job->blk); +- if (aio_context != data->aio_context) { +- aio_context_acquire(aio_context); +- } +- +- data->fn(data->job, data->opaque); +- +- if (aio_context != data->aio_context) { +- aio_context_release(aio_context); +- } +- +- aio_context_release(data->aio_context); +- +- g_free(data); +-} +- +-void block_job_defer_to_main_loop(BlockJob *job, +- BlockJobDeferToMainLoopFn *fn, +- void *opaque) +-{ +- BlockJobDeferToMainLoopData *data = g_malloc(sizeof(*data)); +- data->job = job; +- data->aio_context = blk_get_aio_context(job->blk); +- data->fn = fn; +- data->opaque = opaque; +- job->deferred_to_main_loop = true; +- +- aio_bh_schedule_oneshot(qemu_get_aio_context(), +- block_job_defer_to_main_loop_bh, data); +-} +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 1e708f4..2a9e865 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -92,11 +92,6 @@ typedef struct BlockJob { + */ + bool ready; + +- /** +- * Set to true when the job has deferred work to the main loop. +- */ +- bool deferred_to_main_loop; +- + /** Status that is published by the query-block-jobs QMP API */ + BlockDeviceIoStatus iostatus; + +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index d64f30e..0c2f8de 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -233,23 +233,4 @@ void block_job_event_ready(BlockJob *job); + BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, + int is_read, int error); + +-typedef void BlockJobDeferToMainLoopFn(BlockJob *job, void *opaque); +- +-/** +- * block_job_defer_to_main_loop: +- * @job: The job +- * @fn: The function to run in the main loop +- * @opaque: The opaque value that is passed to @fn +- * +- * This function must be called by the main job coroutine just before it +- * returns. @fn is executed in the main loop with the BlockDriverState +- * AioContext acquired. Block jobs must call bdrv_unref(), bdrv_close(), and +- * anything that uses bdrv_drain_all() in the main loop. +- * +- * The @job AioContext is held while @fn executes. +- */ +-void block_job_defer_to_main_loop(BlockJob *job, +- BlockJobDeferToMainLoopFn *fn, +- void *opaque); +- + #endif +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 01e083f..933e0ab 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -58,6 +58,9 @@ typedef struct Job { + */ + bool cancelled; + ++ /** Set to true when the job has deferred work to the main loop. */ ++ bool deferred_to_main_loop; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + } Job; +@@ -131,6 +134,23 @@ Job *job_get(const char *id); + */ + int job_apply_verb(Job *job, JobVerb verb, Error **errp); + ++typedef void JobDeferToMainLoopFn(Job *job, void *opaque); ++ ++/** ++ * @job: The job ++ * @fn: The function to run in the main loop ++ * @opaque: The opaque value that is passed to @fn ++ * ++ * This function must be called by the main job coroutine just before it ++ * returns. @fn is executed in the main loop with the job AioContext acquired. ++ * ++ * Block jobs must call bdrv_unref(), bdrv_close(), and anything that uses ++ * bdrv_drain_all() in the main loop. ++ * ++ * The @job AioContext is held while @fn executes. ++ */ ++void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque); ++ + /* TODO To be removed from the public interface */ + void job_state_transition(Job *job, JobStatus s1); + +diff --git a/job.c b/job.c +index 01074d0..c5a37fb 100644 +--- a/job.c ++++ b/job.c +@@ -28,6 +28,7 @@ + #include "qapi/error.h" + #include "qemu/job.h" + #include "qemu/id.h" ++#include "qemu/main-loop.h" + #include "trace-root.h" + + static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); +@@ -170,3 +171,34 @@ void job_unref(Job *job) + g_free(job); + } + } ++ ++typedef struct { ++ Job *job; ++ JobDeferToMainLoopFn *fn; ++ void *opaque; ++} JobDeferToMainLoopData; ++ ++static void job_defer_to_main_loop_bh(void *opaque) ++{ ++ JobDeferToMainLoopData *data = opaque; ++ Job *job = data->job; ++ AioContext *aio_context = job->aio_context; ++ ++ aio_context_acquire(aio_context); ++ data->fn(data->job, data->opaque); ++ aio_context_release(aio_context); ++ ++ g_free(data); ++} ++ ++void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque) ++{ ++ JobDeferToMainLoopData *data = g_malloc(sizeof(*data)); ++ data->job = job; ++ data->fn = fn; ++ data->opaque = opaque; ++ job->deferred_to_main_loop = true; ++ ++ aio_bh_schedule_oneshot(qemu_get_aio_context(), ++ job_defer_to_main_loop_bh, data); ++} +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index f9e37d4..4f8cba8 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -496,9 +496,10 @@ typedef struct TestBlockJob { + bool should_complete; + } TestBlockJob; + +-static void test_job_completed(BlockJob *job, void *opaque) ++static void test_job_completed(Job *job, void *opaque) + { +- block_job_completed(job, 0); ++ BlockJob *bjob = container_of(job, BlockJob, job); ++ block_job_completed(bjob, 0); + } + + static void coroutine_fn test_job_start(void *opaque) +@@ -510,7 +511,7 @@ static void coroutine_fn test_job_start(void *opaque) + block_job_sleep_ns(&s->common, 100000); + } + +- block_job_defer_to_main_loop(&s->common, test_job_completed, NULL); ++ job_defer_to_main_loop(&s->common.job, test_job_completed, NULL); + } + + static void test_job_complete(BlockJob *job, Error **errp) +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 26b4bbb..c03f966 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -24,16 +24,17 @@ typedef struct { + int *result; + } TestBlockJob; + +-static void test_block_job_complete(BlockJob *job, void *opaque) ++static void test_block_job_complete(Job *job, void *opaque) + { +- BlockDriverState *bs = blk_bs(job->blk); ++ BlockJob *bjob = container_of(job, BlockJob, job); ++ BlockDriverState *bs = blk_bs(bjob->blk); + int rc = (intptr_t)opaque; + +- if (job_is_cancelled(&job->job)) { ++ if (job_is_cancelled(job)) { + rc = -ECANCELED; + } + +- block_job_completed(job, rc); ++ block_job_completed(bjob, rc); + bdrv_unref(bs); + } + +@@ -54,8 +55,8 @@ static void coroutine_fn test_block_job_run(void *opaque) + } + } + +- block_job_defer_to_main_loop(job, test_block_job_complete, +- (void *)(intptr_t)s->rc); ++ job_defer_to_main_loop(&job->job, test_block_job_complete, ++ (void *)(intptr_t)s->rc); + } + + typedef struct { +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index fa31481..5f43bd7 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -161,11 +161,12 @@ typedef struct CancelJob { + bool completed; + } CancelJob; + +-static void cancel_job_completed(BlockJob *job, void *opaque) ++static void cancel_job_completed(Job *job, void *opaque) + { ++ BlockJob *bjob = container_of(job, BlockJob, job); + CancelJob *s = opaque; + s->completed = true; +- block_job_completed(job, 0); ++ block_job_completed(bjob, 0); + } + + static void cancel_job_complete(BlockJob *job, Error **errp) +@@ -191,7 +192,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + } + + defer: +- block_job_defer_to_main_loop(&s->common, cancel_job_completed, s); ++ job_defer_to_main_loop(&s->common.job, cancel_job_completed, s); + } + + static const BlockJobDriver test_cancel_driver = { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-job_finish_sync-to-Job.patch b/SOURCES/kvm-job-Move-job_finish_sync-to-Job.patch new file mode 100644 index 0000000..b35446f --- /dev/null +++ b/SOURCES/kvm-job-Move-job_finish_sync-to-Job.patch @@ -0,0 +1,199 @@ +From 1680cb0e35f1a465feb93ade536c630f62ef23be Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:20 +0200 +Subject: [PATCH 112/268] job: Move job_finish_sync() to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-38-kwolf@redhat.com> +Patchwork-id: 81076 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 37/73] job: Move job_finish_sync() to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +block_job_finish_sync() doesn't contain anything block job specific any +more, so it can be moved to Job. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 6a74c075aca731e7e945201a4ae2336b8e328433) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/commit.c | 6 +++--- + blockjob.c | 55 +++++++++--------------------------------------------- + include/qemu/job.h | 9 +++++++++ + job.c | 28 +++++++++++++++++++++++++++ + 4 files changed, 49 insertions(+), 49 deletions(-) + +diff --git a/block/commit.c b/block/commit.c +index 02a8af9..40d97a3 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -112,9 +112,9 @@ static void commit_complete(Job *job, void *opaque) + blk_unref(s->top); + + /* If there is more than one reference to the job (e.g. if called from +- * block_job_finish_sync()), block_job_completed() won't free it and +- * therefore the blockers on the intermediate nodes remain. This would +- * cause bdrv_set_backing_hd() to fail. */ ++ * job_finish_sync()), block_job_completed() won't free it and therefore ++ * the blockers on the intermediate nodes remain. This would cause ++ * bdrv_set_backing_hd() to fail. */ + block_job_remove_all_bdrv(bjob); + + block_job_completed(&s->common, ret); +diff --git a/blockjob.c b/blockjob.c +index 0ca7672..1ed3e9c 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -307,40 +307,6 @@ static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *), bool lock) + return rc; + } + +-static int block_job_finish_sync(BlockJob *job, +- void (*finish)(BlockJob *, Error **errp), +- Error **errp) +-{ +- Error *local_err = NULL; +- int ret; +- +- assert(blk_bs(job->blk)->job == job); +- +- job_ref(&job->job); +- +- if (finish) { +- finish(job, &local_err); +- } +- if (local_err) { +- error_propagate(errp, local_err); +- job_unref(&job->job); +- return -EBUSY; +- } +- /* job_drain calls job_enter, and it should be enough to induce progress +- * until the job completes or moves to the main thread. +- */ +- while (!job->job.deferred_to_main_loop && !job_is_completed(&job->job)) { +- job_drain(&job->job); +- } +- while (!job_is_completed(&job->job)) { +- aio_poll(qemu_get_aio_context(), true); +- } +- ret = (job_is_cancelled(&job->job) && job->job.ret == 0) +- ? -ECANCELED : job->job.ret; +- job_unref(&job->job); +- return ret; +-} +- + static void block_job_completed_txn_abort(BlockJob *job) + { + AioContext *ctx; +@@ -375,7 +341,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + ctx = blk_get_aio_context(other_job->blk); + if (!job_is_completed(&other_job->job)) { + assert(job_is_cancelled(&other_job->job)); +- block_job_finish_sync(other_job, NULL, NULL); ++ job_finish_sync(&other_job->job, NULL, NULL); + } + job_finalize_single(&other_job->job); + aio_context_release(ctx); +@@ -528,16 +494,18 @@ void block_job_user_cancel(BlockJob *job, bool force, Error **errp) + } + + /* A wrapper around block_job_cancel() taking an Error ** parameter so it may be +- * used with block_job_finish_sync() without the need for (rather nasty) +- * function pointer casts there. */ +-static void block_job_cancel_err(BlockJob *job, Error **errp) ++ * used with job_finish_sync() without the need for (rather nasty) function ++ * pointer casts there. */ ++static void block_job_cancel_err(Job *job, Error **errp) + { +- block_job_cancel(job, false); ++ BlockJob *bjob = container_of(job, BlockJob, job); ++ assert(is_block_job(job)); ++ block_job_cancel(bjob, false); + } + + int block_job_cancel_sync(BlockJob *job) + { +- return block_job_finish_sync(job, &block_job_cancel_err, NULL); ++ return job_finish_sync(&job->job, &block_job_cancel_err, NULL); + } + + void block_job_cancel_sync_all(void) +@@ -553,14 +521,9 @@ void block_job_cancel_sync_all(void) + } + } + +-static void block_job_complete(BlockJob *job, Error **errp) +-{ +- job_complete(&job->job, errp); +-} +- + int block_job_complete_sync(BlockJob *job, Error **errp) + { +- return block_job_finish_sync(job, &block_job_complete, errp); ++ return job_finish_sync(&job->job, job_complete, errp); + } + + void block_job_progress_update(BlockJob *job, uint64_t done) +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 8f7f71a..17e2cec 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -389,6 +389,15 @@ typedef void JobDeferToMainLoopFn(Job *job, void *opaque); + */ + void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque); + ++/** ++ * Synchronously finishes the given @job. If @finish is given, it is called to ++ * trigger completion or cancellation of the job. ++ * ++ * Returns 0 if the job is successfully completed, -ECANCELED if the job was ++ * cancelled before completing, and -errno in other error cases. ++ */ ++int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp); ++ + /* TODO To be removed from the public interface */ + void job_state_transition(Job *job, JobStatus s1); + void coroutine_fn job_do_yield(Job *job, uint64_t ns); +diff --git a/job.c b/job.c +index 8ceac0b..aa74b4c 100644 +--- a/job.c ++++ b/job.c +@@ -603,3 +603,31 @@ void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque) + aio_bh_schedule_oneshot(qemu_get_aio_context(), + job_defer_to_main_loop_bh, data); + } ++ ++int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) ++{ ++ Error *local_err = NULL; ++ int ret; ++ ++ job_ref(job); ++ ++ if (finish) { ++ finish(job, &local_err); ++ } ++ if (local_err) { ++ error_propagate(errp, local_err); ++ job_unref(job); ++ return -EBUSY; ++ } ++ /* job_drain calls job_enter, and it should be enough to induce progress ++ * until the job completes or moves to the main thread. */ ++ while (!job->deferred_to_main_loop && !job_is_completed(job)) { ++ job_drain(job); ++ } ++ while (!job_is_completed(job)) { ++ aio_poll(qemu_get_aio_context(), true); ++ } ++ ret = (job_is_cancelled(job) && job->ret == 0) ? -ECANCELED : job->ret; ++ job_unref(job); ++ return ret; ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-pause-resume-functions-to-Job.patch b/SOURCES/kvm-job-Move-pause-resume-functions-to-Job.patch new file mode 100644 index 0000000..2fab63e --- /dev/null +++ b/SOURCES/kvm-job-Move-pause-resume-functions-to-Job.patch @@ -0,0 +1,550 @@ +From b8d8e8478603258bc5505a9b606321a81b14d896 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:11 +0200 +Subject: [PATCH 103/268] job: Move pause/resume functions to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-29-kwolf@redhat.com> +Patchwork-id: 81062 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 28/73] job: Move pause/resume functions to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +While we already moved the state related to job pausing to Job, the +functions to do were still BlockJob only. This commit moves them over to +Job. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit b15de82867975e0b4acf644b5ee36d84904b6612) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 1 + + block/commit.c | 1 + + block/mirror.c | 2 ++ + block/stream.c | 1 + + blockdev.c | 6 ++-- + blockjob.c | 81 +++++++++----------------------------------- + include/block/blockjob.h | 32 ----------------- + include/block/blockjob_int.h | 7 ++++ + include/qemu/job.h | 37 ++++++++++++++++++++ + job.c | 59 ++++++++++++++++++++++++++++++++ + tests/test-bdrv-drain.c | 1 + + tests/test-blockjob-txn.c | 1 + + tests/test-blockjob.c | 6 ++-- + 13 files changed, 133 insertions(+), 102 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index f3a4f7c..4d011d5 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -528,6 +528,7 @@ static const BlockJobDriver backup_job_driver = { + .instance_size = sizeof(BackupBlockJob), + .job_type = JOB_TYPE_BACKUP, + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = backup_run, + }, + .commit = backup_commit, +diff --git a/block/commit.c b/block/commit.c +index 1c6cb6c..c4a98e5 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -220,6 +220,7 @@ static const BlockJobDriver commit_job_driver = { + .instance_size = sizeof(CommitBlockJob), + .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = commit_run, + }, + }; +diff --git a/block/mirror.c b/block/mirror.c +index 5d8f75c..9a7226f 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -991,6 +991,7 @@ static const BlockJobDriver mirror_job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = JOB_TYPE_MIRROR, + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = mirror_run, + .pause = mirror_pause, + }, +@@ -1004,6 +1005,7 @@ static const BlockJobDriver commit_active_job_driver = { + .instance_size = sizeof(MirrorBlockJob), + .job_type = JOB_TYPE_COMMIT, + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = mirror_run, + .pause = mirror_pause, + }, +diff --git a/block/stream.c b/block/stream.c +index 1faab02..e81b488 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -214,6 +214,7 @@ static const BlockJobDriver stream_job_driver = { + .job_type = JOB_TYPE_STREAM, + .free = block_job_free, + .start = stream_run, ++ .user_resume = block_job_user_resume, + }, + }; + +diff --git a/blockdev.c b/blockdev.c +index 93a4cdf..522158c 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3845,7 +3845,7 @@ void qmp_block_job_cancel(const char *device, + force = false; + } + +- if (block_job_user_paused(job) && !force) { ++ if (job_user_paused(&job->job) && !force) { + error_setg(errp, "The block job for device '%s' is currently paused", + device); + goto out; +@@ -3867,7 +3867,7 @@ void qmp_block_job_pause(const char *device, Error **errp) + } + + trace_qmp_block_job_pause(job); +- block_job_user_pause(job, errp); ++ job_user_pause(&job->job, errp); + aio_context_release(aio_context); + } + +@@ -3881,7 +3881,7 @@ void qmp_block_job_resume(const char *device, Error **errp) + } + + trace_qmp_block_job_resume(job); +- block_job_user_resume(job, errp); ++ job_user_resume(&job->job, errp); + aio_context_release(aio_context); + } + +diff --git a/blockjob.c b/blockjob.c +index 4dc360c..6334a54 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -140,29 +140,6 @@ static void block_job_txn_del_job(BlockJob *job) + } + } + +-/* Assumes the job_mutex is held */ +-static bool job_timer_not_pending(Job *job) +-{ +- return !timer_pending(&job->sleep_timer); +-} +- +-static void block_job_pause(BlockJob *job) +-{ +- job->job.pause_count++; +-} +- +-static void block_job_resume(BlockJob *job) +-{ +- assert(job->job.pause_count > 0); +- job->job.pause_count--; +- if (job->job.pause_count) { +- return; +- } +- +- /* kick only if no timer is pending */ +- job_enter_cond(&job->job, job_timer_not_pending); +-} +- + static void block_job_attached_aio_context(AioContext *new_context, + void *opaque); + static void block_job_detach_aio_context(void *opaque); +@@ -193,7 +170,7 @@ static void block_job_attached_aio_context(AioContext *new_context, + job->driver->attached_aio_context(job, new_context); + } + +- block_job_resume(job); ++ job_resume(&job->job); + } + + static void block_job_drain(BlockJob *job) +@@ -214,7 +191,7 @@ static void block_job_detach_aio_context(void *opaque) + /* In case the job terminates during aio_poll()... */ + job_ref(&job->job); + +- block_job_pause(job); ++ job_pause(&job->job); + + while (!job->job.paused && !job->completed) { + block_job_drain(job); +@@ -233,13 +210,13 @@ static char *child_job_get_parent_desc(BdrvChild *c) + static void child_job_drained_begin(BdrvChild *c) + { + BlockJob *job = c->opaque; +- block_job_pause(job); ++ job_pause(&job->job); + } + + static void child_job_drained_end(BdrvChild *c) + { + BlockJob *job = c->opaque; +- block_job_resume(job); ++ job_resume(&job->job); + } + + static const BdrvChildRole child_job = { +@@ -396,9 +373,9 @@ static void block_job_cancel_async(BlockJob *job, bool force) + if (job->iostatus != BLOCK_DEVICE_IO_STATUS_OK) { + block_job_iostatus_reset(job); + } +- if (job->user_paused) { ++ if (job->job.user_paused) { + /* Do not call block_job_enter here, the caller will handle it. */ +- job->user_paused = false; ++ job->job.user_paused = false; + job->job.pause_count--; + } + job->job.cancelled = true; +@@ -628,39 +605,6 @@ void block_job_dismiss(BlockJob **jobptr, Error **errp) + *jobptr = NULL; + } + +-void block_job_user_pause(BlockJob *job, Error **errp) +-{ +- if (job_apply_verb(&job->job, JOB_VERB_PAUSE, errp)) { +- return; +- } +- if (job->user_paused) { +- error_setg(errp, "Job is already paused"); +- return; +- } +- job->user_paused = true; +- block_job_pause(job); +-} +- +-bool block_job_user_paused(BlockJob *job) +-{ +- return job->user_paused; +-} +- +-void block_job_user_resume(BlockJob *job, Error **errp) +-{ +- assert(job); +- if (!job->user_paused || job->job.pause_count <= 0) { +- error_setg(errp, "Can't resume a job that was not paused"); +- return; +- } +- if (job_apply_verb(&job->job, JOB_VERB_RESUME, errp)) { +- return; +- } +- block_job_iostatus_reset(job); +- job->user_paused = false; +- block_job_resume(job); +-} +- + void block_job_cancel(BlockJob *job, bool force) + { + if (job->job.status == JOB_STATUS_CONCLUDED) { +@@ -851,6 +795,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + + assert(is_block_job(&job->job)); + assert(job->job.driver->free == &block_job_free); ++ assert(job->job.driver->user_resume == &block_job_user_resume); + + job->driver = driver; + job->blk = blk; +@@ -941,10 +886,16 @@ void block_job_iostatus_reset(BlockJob *job) + if (job->iostatus == BLOCK_DEVICE_IO_STATUS_OK) { + return; + } +- assert(job->user_paused && job->job.pause_count > 0); ++ assert(job->job.user_paused && job->job.pause_count > 0); + job->iostatus = BLOCK_DEVICE_IO_STATUS_OK; + } + ++void block_job_user_resume(Job *job) ++{ ++ BlockJob *bjob = container_of(job, BlockJob, job); ++ block_job_iostatus_reset(bjob); ++} ++ + void block_job_event_ready(BlockJob *job) + { + job_state_transition(&job->job, JOB_STATUS_READY); +@@ -991,9 +942,9 @@ BlockErrorAction block_job_error_action(BlockJob *job, BlockdevOnError on_err, + action, &error_abort); + } + if (action == BLOCK_ERROR_ACTION_STOP) { +- block_job_pause(job); ++ job_pause(&job->job); + /* make the pause user visible, which will be resumed from QMP. */ +- job->user_paused = true; ++ job->job.user_paused = true; + block_job_iostatus_set_err(job, error); + } + return action; +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index b60d919..556a8f6 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -57,12 +57,6 @@ typedef struct BlockJob { + bool force; + + /** +- * Set to true if the job is paused by user. Can be unpaused with the +- * block-job-resume QMP command. +- */ +- bool user_paused; +- +- /** + * Set to true when the job is ready to be completed. + */ + bool ready; +@@ -248,32 +242,6 @@ void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining); + BlockJobInfo *block_job_query(BlockJob *job, Error **errp); + + /** +- * block_job_user_pause: +- * @job: The job to be paused. +- * +- * Asynchronously pause the specified job. +- * Do not allow a resume until a matching call to block_job_user_resume. +- */ +-void block_job_user_pause(BlockJob *job, Error **errp); +- +-/** +- * block_job_paused: +- * @job: The job to query. +- * +- * Returns true if the job is user-paused. +- */ +-bool block_job_user_paused(BlockJob *job); +- +-/** +- * block_job_user_resume: +- * @job: The job to be resumed. +- * +- * Resume the specified job. +- * Must be paired with a preceding block_job_user_pause. +- */ +-void block_job_user_resume(BlockJob *job, Error **errp); +- +-/** + * block_job_user_cancel: + * @job: The job to be cancelled. + * @force: Quit a job without waiting for data to be in sync. +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 8937f5b..7e705ae 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -134,6 +134,13 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + void block_job_free(Job *job); + + /** ++ * block_job_user_resume: ++ * Callback to be used for JobDriver.user_resume in all block jobs. Resets the ++ * iostatus when the user resumes @job. ++ */ ++void block_job_user_resume(Job *job); ++ ++/** + * block_job_yield: + * @job: The job that calls the function. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 509408f..bc63985 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -83,6 +83,12 @@ typedef struct Job { + bool paused; + + /** ++ * Set to true if the job is paused by user. Can be unpaused with the ++ * block-job-resume QMP command. ++ */ ++ bool user_paused; ++ ++ /** + * Set to true if the job should cancel itself. The flag must + * always be tested just before toggling the busy flag from false + * to true. After a job has been cancelled, it should only yield +@@ -124,6 +130,12 @@ struct JobDriver { + */ + void coroutine_fn (*resume)(Job *job); + ++ /** ++ * Called when the job is resumed by the user (i.e. user_paused becomes ++ * false). .user_resume is called before .resume. ++ */ ++ void (*user_resume)(Job *job); ++ + /** Called when the job is freed */ + void (*free)(Job *job); + }; +@@ -203,6 +215,31 @@ const char *job_type_str(const Job *job); + bool job_is_cancelled(Job *job); + + /** ++ * Request @job to pause at the next pause point. Must be paired with ++ * job_resume(). If the job is supposed to be resumed by user action, call ++ * job_user_pause() instead. ++ */ ++void job_pause(Job *job); ++ ++/** Resumes a @job paused with job_pause. */ ++void job_resume(Job *job); ++ ++/** ++ * Asynchronously pause the specified @job. ++ * Do not allow a resume until a matching call to job_user_resume. ++ */ ++void job_user_pause(Job *job, Error **errp); ++ ++/** Returns true if the job is user-paused. */ ++bool job_user_paused(Job *job); ++ ++/** ++ * Resume the specified @job. ++ * Must be paired with a preceding job_user_pause. ++ */ ++void job_user_resume(Job *job, Error **errp); ++ ++/** + * Get the next element from the list of block jobs after @job, or the + * first one if @job is %NULL. + * +diff --git a/job.c b/job.c +index 1b8cba1..fd10b1d 100644 +--- a/job.c ++++ b/job.c +@@ -341,6 +341,65 @@ void job_start(Job *job) + aio_co_enter(job->aio_context, job->co); + } + ++/* Assumes the block_job_mutex is held */ ++static bool job_timer_not_pending(Job *job) ++{ ++ return !timer_pending(&job->sleep_timer); ++} ++ ++void job_pause(Job *job) ++{ ++ job->pause_count++; ++} ++ ++void job_resume(Job *job) ++{ ++ assert(job->pause_count > 0); ++ job->pause_count--; ++ if (job->pause_count) { ++ return; ++ } ++ ++ /* kick only if no timer is pending */ ++ job_enter_cond(job, job_timer_not_pending); ++} ++ ++void job_user_pause(Job *job, Error **errp) ++{ ++ if (job_apply_verb(job, JOB_VERB_PAUSE, errp)) { ++ return; ++ } ++ if (job->user_paused) { ++ error_setg(errp, "Job is already paused"); ++ return; ++ } ++ job->user_paused = true; ++ job_pause(job); ++} ++ ++bool job_user_paused(Job *job) ++{ ++ return job->user_paused; ++} ++ ++void job_user_resume(Job *job, Error **errp) ++{ ++ assert(job); ++ if (!job->user_paused || job->pause_count <= 0) { ++ error_setg(errp, "Can't resume a job that was not paused"); ++ return; ++ } ++ if (job_apply_verb(job, JOB_VERB_RESUME, errp)) { ++ return; ++ } ++ if (job->driver->user_resume) { ++ job->driver->user_resume(job); ++ } ++ job->user_paused = false; ++ job_resume(job); ++} ++ ++ + typedef struct { + Job *job; + JobDeferToMainLoopFn *fn; +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 50232f5..c993512 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -524,6 +524,7 @@ BlockJobDriver test_job_driver = { + .job_driver = { + .instance_size = sizeof(TestBlockJob), + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = test_job_start, + }, + .complete = test_job_complete, +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 0e6162b..93d1ff0 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -78,6 +78,7 @@ static const BlockJobDriver test_block_job_driver = { + .job_driver = { + .instance_size = sizeof(TestBlockJob), + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = test_block_job_run, + }, + }; +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index b329bd5..ceb5960 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -20,6 +20,7 @@ static const BlockJobDriver test_block_job_driver = { + .job_driver = { + .instance_size = sizeof(BlockJob), + .free = block_job_free, ++ .user_resume = block_job_user_resume, + }, + }; + +@@ -199,6 +200,7 @@ static const BlockJobDriver test_cancel_driver = { + .job_driver = { + .instance_size = sizeof(CancelJob), + .free = block_job_free, ++ .user_resume = block_job_user_resume, + .start = cancel_job_start, + }, + .complete = cancel_job_complete, +@@ -270,7 +272,7 @@ static void test_cancel_paused(void) + job_start(&job->job); + assert(job->job.status == JOB_STATUS_RUNNING); + +- block_job_user_pause(job, &error_abort); ++ job_user_pause(&job->job, &error_abort); + block_job_enter(job); + assert(job->job.status == JOB_STATUS_PAUSED); + +@@ -308,7 +310,7 @@ static void test_cancel_standby(void) + block_job_enter(job); + assert(job->job.status == JOB_STATUS_READY); + +- block_job_user_pause(job, &error_abort); ++ job_user_pause(&job->job, &error_abort); + block_job_enter(job); + assert(job->job.status == JOB_STATUS_STANDBY); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-progress-fields-to-Job.patch b/SOURCES/kvm-job-Move-progress-fields-to-Job.patch new file mode 100644 index 0000000..7680a37 --- /dev/null +++ b/SOURCES/kvm-job-Move-progress-fields-to-Job.patch @@ -0,0 +1,337 @@ +From 63d4efaba799ee14d474e545f5634ee1c4369ef6 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:29 +0200 +Subject: [PATCH 121/268] job: Move progress fields to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-47-kwolf@redhat.com> +Patchwork-id: 81115 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 46/73] job: Move progress fields to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +BlockJob has fields .offset and .len, which are actually misnomers today +because they are no longer tied to block device sizes, but just progress +counters. As such they make a lot of sense in generic Jobs. + +This patch moves the fields to Job and renames them to .progress_current +and .progress_total to describe their function better. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 30a5c887bf4a7e00d0e0ecbb08509e8ba2902620) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 8 ++++---- + block/commit.c | 4 ++-- + block/mirror.c | 4 ++-- + block/stream.c | 4 ++-- + blockjob.c | 26 ++++++++------------------ + include/block/blockjob.h | 25 ------------------------- + include/qemu/job.h | 28 ++++++++++++++++++++++++++++ + job.c | 10 ++++++++++ + qemu-img.c | 8 ++++++-- + 9 files changed, 62 insertions(+), 55 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 6f4f3df..4e228e9 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -160,7 +160,7 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job, + * offset field is an opaque progress value, it is not a disk offset. + */ + job->bytes_read += n; +- block_job_progress_update(&job->common, n); ++ job_progress_update(&job->common.job, n); + } + + out: +@@ -406,8 +406,8 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) + bdrv_set_dirty_iter(dbi, next_cluster * job->cluster_size); + } + +- /* TODO block_job_progress_set_remaining() would make more sense */ +- block_job_progress_update(&job->common, ++ /* TODO job_progress_set_remaining() would make more sense */ ++ job_progress_update(&job->common.job, + job->len - hbitmap_count(job->copy_bitmap) * job->cluster_size); + + bdrv_dirty_iter_free(dbi); +@@ -425,7 +425,7 @@ static void coroutine_fn backup_run(void *opaque) + qemu_co_rwlock_init(&job->flush_rwlock); + + nb_clusters = DIV_ROUND_UP(job->len, job->cluster_size); +- block_job_progress_set_remaining(&job->common, job->len); ++ job_progress_set_remaining(&job->common.job, job->len); + + job->copy_bitmap = hbitmap_alloc(nb_clusters, 0); + if (job->sync_mode == MIRROR_SYNC_MODE_INCREMENTAL) { +diff --git a/block/commit.c b/block/commit.c +index b0a847e..6206661 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -150,7 +150,7 @@ static void coroutine_fn commit_run(void *opaque) + if (len < 0) { + goto out; + } +- block_job_progress_set_remaining(&s->common, len); ++ job_progress_set_remaining(&s->common.job, len); + + ret = base_len = blk_getlength(s->base); + if (base_len < 0) { +@@ -196,7 +196,7 @@ static void coroutine_fn commit_run(void *opaque) + } + } + /* Publish progress */ +- block_job_progress_update(&s->common, n); ++ job_progress_update(&s->common.job, n); + + if (copy) { + delay_ns = block_job_ratelimit_get_delay(&s->common, n); +diff --git a/block/mirror.c b/block/mirror.c +index bdc1b5b..dcb66ec 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -119,7 +119,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret) + bitmap_set(s->cow_bitmap, chunk_num, nb_chunks); + } + if (!s->initial_zeroing_ongoing) { +- block_job_progress_update(&s->common, op->bytes); ++ job_progress_update(&s->common.job, op->bytes); + } + } + qemu_iovec_destroy(&op->qiov); +@@ -792,7 +792,7 @@ static void coroutine_fn mirror_run(void *opaque) + /* cnt is the number of dirty bytes remaining and s->bytes_in_flight is + * the number of bytes currently being processed; together those are + * the current remaining operation length */ +- block_job_progress_set_remaining(&s->common, s->bytes_in_flight + cnt); ++ job_progress_set_remaining(&s->common.job, s->bytes_in_flight + cnt); + + /* Note that even when no rate limit is applied we need to yield + * periodically with no pending I/O so that bdrv_drain_all() returns. +diff --git a/block/stream.c b/block/stream.c +index 8546c41..a5d6e0c 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -121,7 +121,7 @@ static void coroutine_fn stream_run(void *opaque) + ret = len; + goto out; + } +- block_job_progress_set_remaining(&s->common, len); ++ job_progress_set_remaining(&s->common.job, len); + + buf = qemu_blockalign(bs, STREAM_BUFFER_SIZE); + +@@ -184,7 +184,7 @@ static void coroutine_fn stream_run(void *opaque) + ret = 0; + + /* Publish progress */ +- block_job_progress_update(&s->common, n); ++ job_progress_update(&s->common.job, n); + if (copy) { + delay_ns = block_job_ratelimit_get_delay(&s->common, n); + } else { +diff --git a/blockjob.c b/blockjob.c +index da11b3b..5c8ff6f 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -242,16 +242,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) + return ratelimit_calculate_delay(&job->limit, n); + } + +-void block_job_progress_update(BlockJob *job, uint64_t done) +-{ +- job->offset += done; +-} +- +-void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining) +-{ +- job->len = job->offset + remaining; +-} +- + BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + { + BlockJobInfo *info; +@@ -263,10 +253,10 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info = g_new0(BlockJobInfo, 1); + info->type = g_strdup(job_type_str(&job->job)); + info->device = g_strdup(job->job.id); +- info->len = job->len; + info->busy = atomic_read(&job->job.busy); + info->paused = job->job.pause_count > 0; +- info->offset = job->offset; ++ info->offset = job->job.progress_current; ++ info->len = job->job.progress_total; + info->speed = job->speed; + info->io_status = job->iostatus; + info->ready = job_is_ready(&job->job), +@@ -296,8 +286,8 @@ static void block_job_event_cancelled(Notifier *n, void *opaque) + + qapi_event_send_block_job_cancelled(job_type(&job->job), + job->job.id, +- job->len, +- job->offset, ++ job->job.progress_total, ++ job->job.progress_current, + job->speed, + &error_abort); + } +@@ -317,8 +307,8 @@ static void block_job_event_completed(Notifier *n, void *opaque) + + qapi_event_send_block_job_completed(job_type(&job->job), + job->job.id, +- job->len, +- job->offset, ++ job->job.progress_total, ++ job->job.progress_current, + job->speed, + !!msg, + msg, +@@ -348,8 +338,8 @@ static void block_job_event_ready(Notifier *n, void *opaque) + + qapi_event_send_block_job_ready(job_type(&job->job), + job->job.id, +- job->len, +- job->offset, ++ job->job.progress_total, ++ job->job.progress_current, + job->speed, &error_abort); + } + +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 4fca45f..3021d11 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -52,12 +52,6 @@ typedef struct BlockJob { + /** Status that is published by the query-block-jobs QMP API */ + BlockDeviceIoStatus iostatus; + +- /** Offset that is published by the query-block-jobs QMP API */ +- int64_t offset; +- +- /** Length that is published by the query-block-jobs QMP API */ +- int64_t len; +- + /** Speed that was set with @block_job_set_speed. */ + int64_t speed; + +@@ -139,25 +133,6 @@ void block_job_remove_all_bdrv(BlockJob *job); + void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); + + /** +- * block_job_progress_update: +- * @job: The job that has made progress +- * @done: How much progress the job made +- * +- * Updates the progress counter of the job. +- */ +-void block_job_progress_update(BlockJob *job, uint64_t done); +- +-/** +- * block_job_progress_set_remaining: +- * @job: The job whose expected progress end value is set +- * @remaining: Expected end value of the progress counter of the job +- * +- * Sets the expected end value of the progress counter of a job so that a +- * completion percentage can be calculated when the progress is updated. +- */ +-void block_job_progress_set_remaining(BlockJob *job, uint64_t remaining); +- +-/** + * block_job_query: + * @job: The job to get information about. + * +diff --git a/include/qemu/job.h b/include/qemu/job.h +index bfc2bc5..92d1d24 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -114,6 +114,16 @@ typedef struct Job { + /** True if this job should automatically dismiss itself */ + bool auto_dismiss; + ++ /** ++ * Current progress. The unit is arbitrary as long as the ratio between ++ * progress_current and progress_total represents the estimated percentage ++ * of work already done. ++ */ ++ int64_t progress_current; ++ ++ /** Estimated progress_current value at the completion of the job */ ++ int64_t progress_total; ++ + /** ret code passed to job_completed. */ + int ret; + +@@ -304,6 +314,24 @@ void job_ref(Job *job); + */ + void job_unref(Job *job); + ++/** ++ * @job: The job that has made progress ++ * @done: How much progress the job made since the last call ++ * ++ * Updates the progress counter of the job. ++ */ ++void job_progress_update(Job *job, uint64_t done); ++ ++/** ++ * @job: The job whose expected progress end value is set ++ * @remaining: Missing progress (on top of the current progress counter value) ++ * until the new expected end value is reached ++ * ++ * Sets the expected end value of the progress counter of a job so that a ++ * completion percentage can be calculated when the progress is updated. ++ */ ++void job_progress_set_remaining(Job *job, uint64_t remaining); ++ + /** To be called when a cancelled job is finalised. */ + void job_event_cancelled(Job *job); + +diff --git a/job.c b/job.c +index b5bd51b..2046d2f 100644 +--- a/job.c ++++ b/job.c +@@ -364,6 +364,16 @@ void job_unref(Job *job) + } + } + ++void job_progress_update(Job *job, uint64_t done) ++{ ++ job->progress_current += done; ++} ++ ++void job_progress_set_remaining(Job *job, uint64_t remaining) ++{ ++ job->progress_total = job->progress_current + remaining; ++} ++ + void job_event_cancelled(Job *job) + { + notifier_list_notify(&job->on_finalize_cancelled, job); +diff --git a/qemu-img.c b/qemu-img.c +index 3c449a2..9fc8e66 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -875,9 +875,13 @@ static void run_block_job(BlockJob *job, Error **errp) + aio_context_acquire(aio_context); + job_ref(&job->job); + do { ++ float progress = 0.0f; + aio_poll(aio_context, true); +- qemu_progress_print(job->len ? +- ((float)job->offset / job->len * 100.f) : 0.0f, 0); ++ if (job->job.progress_total) { ++ progress = (float)job->job.progress_current / ++ job->job.progress_total * 100.f; ++ } ++ qemu_progress_print(progress, 0); + } while (!job_is_ready(&job->job) && !job_is_completed(&job->job)); + + if (!job_is_completed(&job->job)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-single-job-finalisation-to-Job.patch b/SOURCES/kvm-job-Move-single-job-finalisation-to-Job.patch new file mode 100644 index 0000000..6a1d59b --- /dev/null +++ b/SOURCES/kvm-job-Move-single-job-finalisation-to-Job.patch @@ -0,0 +1,736 @@ +From 0636b876de2af6bb06bbc7ec6dd55a234c591a95 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:16 +0200 +Subject: [PATCH 108/268] job: Move single job finalisation to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-34-kwolf@redhat.com> +Patchwork-id: 81070 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 33/73] job: Move single job finalisation to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves the finalisation of a single job from BlockJob to Job. + +Some part of this code depends on job transactions, and job transactions +call this code, we introduce some temporary calls from Job functions to +BlockJob ones. This will be fixed once transactions move to Job, too. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 4ad351819b974d724e926fd23cdd66bec3c9768e) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 22 +++---- + block/commit.c | 2 +- + block/mirror.c | 2 +- + blockjob.c | 142 ++++++++----------------------------------- + include/block/blockjob.h | 9 --- + include/block/blockjob_int.h | 36 ----------- + include/qemu/job.h | 53 +++++++++++++++- + job.c | 100 +++++++++++++++++++++++++++++- + qemu-img.c | 2 +- + tests/test-blockjob.c | 10 +-- + 10 files changed, 194 insertions(+), 184 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 4d011d5..bd31282 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -207,25 +207,25 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret) + } + } + +-static void backup_commit(BlockJob *job) ++static void backup_commit(Job *job) + { +- BackupBlockJob *s = container_of(job, BackupBlockJob, common); ++ BackupBlockJob *s = container_of(job, BackupBlockJob, common.job); + if (s->sync_bitmap) { + backup_cleanup_sync_bitmap(s, 0); + } + } + +-static void backup_abort(BlockJob *job) ++static void backup_abort(Job *job) + { +- BackupBlockJob *s = container_of(job, BackupBlockJob, common); ++ BackupBlockJob *s = container_of(job, BackupBlockJob, common.job); + if (s->sync_bitmap) { + backup_cleanup_sync_bitmap(s, -1); + } + } + +-static void backup_clean(BlockJob *job) ++static void backup_clean(Job *job) + { +- BackupBlockJob *s = container_of(job, BackupBlockJob, common); ++ BackupBlockJob *s = container_of(job, BackupBlockJob, common.job); + assert(s->target); + blk_unref(s->target); + s->target = NULL; +@@ -530,10 +530,10 @@ static const BlockJobDriver backup_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .start = backup_run, ++ .commit = backup_commit, ++ .abort = backup_abort, ++ .clean = backup_clean, + }, +- .commit = backup_commit, +- .abort = backup_abort, +- .clean = backup_clean, + .attached_aio_context = backup_attached_aio_context, + .drain = backup_drain, + }; +@@ -678,8 +678,8 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + bdrv_reclaim_dirty_bitmap(bs, sync_bitmap, NULL); + } + if (job) { +- backup_clean(&job->common); +- block_job_early_fail(&job->common); ++ backup_clean(&job->common.job); ++ job_early_fail(&job->common.job); + } + + return NULL; +diff --git a/block/commit.c b/block/commit.c +index 7a6ae59..e53b2d7 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -385,7 +385,7 @@ fail: + if (commit_top_bs) { + bdrv_replace_node(commit_top_bs, top, &error_abort); + } +- block_job_early_fail(&s->common); ++ job_early_fail(&s->common.job); + } + + +diff --git a/block/mirror.c b/block/mirror.c +index 5091e72..e9a90ea 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -1257,7 +1257,7 @@ fail: + + g_free(s->replaces); + blk_unref(s->target); +- block_job_early_fail(&s->common); ++ job_early_fail(&s->common.job); + } + + bdrv_child_try_set_perm(mirror_top_bs->backing, 0, BLK_PERM_ALL, +diff --git a/blockjob.c b/blockjob.c +index 05d7921..34c57da 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -127,7 +127,7 @@ void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job) + block_job_txn_ref(txn); + } + +-static void block_job_txn_del_job(BlockJob *job) ++void block_job_txn_del_job(BlockJob *job) + { + if (job->txn) { + QLIST_REMOVE(job, txn_list); +@@ -262,101 +262,12 @@ const BlockJobDriver *block_job_driver(BlockJob *job) + return job->driver; + } + +-static void block_job_decommission(BlockJob *job) +-{ +- assert(job); +- job->job.busy = false; +- job->job.paused = false; +- job->job.deferred_to_main_loop = true; +- block_job_txn_del_job(job); +- job_state_transition(&job->job, JOB_STATUS_NULL); +- job_unref(&job->job); +-} +- +-static void block_job_do_dismiss(BlockJob *job) +-{ +- block_job_decommission(job); +-} +- +-static void block_job_conclude(BlockJob *job) +-{ +- job_state_transition(&job->job, JOB_STATUS_CONCLUDED); +- if (job->job.auto_dismiss || !job_started(&job->job)) { +- block_job_do_dismiss(job); +- } +-} +- +-static void block_job_update_rc(BlockJob *job) +-{ +- if (!job->ret && job_is_cancelled(&job->job)) { +- job->ret = -ECANCELED; +- } +- if (job->ret) { +- job_state_transition(&job->job, JOB_STATUS_ABORTING); +- } +-} +- + static int block_job_prepare(BlockJob *job) + { +- if (job->ret == 0 && job->driver->prepare) { +- job->ret = job->driver->prepare(job); +- } +- return job->ret; +-} +- +-static void block_job_commit(BlockJob *job) +-{ +- assert(!job->ret); +- if (job->driver->commit) { +- job->driver->commit(job); +- } +-} +- +-static void block_job_abort(BlockJob *job) +-{ +- assert(job->ret); +- if (job->driver->abort) { +- job->driver->abort(job); +- } +-} +- +-static void block_job_clean(BlockJob *job) +-{ +- if (job->driver->clean) { +- job->driver->clean(job); ++ if (job->job.ret == 0 && job->driver->prepare) { ++ job->job.ret = job->driver->prepare(job); + } +-} +- +-static int block_job_finalize_single(BlockJob *job) +-{ +- assert(job_is_completed(&job->job)); +- +- /* Ensure abort is called for late-transactional failures */ +- block_job_update_rc(job); +- +- if (!job->ret) { +- block_job_commit(job); +- } else { +- block_job_abort(job); +- } +- block_job_clean(job); +- +- if (job->cb) { +- job->cb(job->opaque, job->ret); +- } +- +- /* Emit events only if we actually started */ +- if (job_started(&job->job)) { +- if (job_is_cancelled(&job->job)) { +- job_event_cancelled(&job->job); +- } else { +- job_event_completed(&job->job); +- } +- } +- +- block_job_txn_del_job(job); +- block_job_conclude(job); +- return 0; ++ return job->job.ret; + } + + static void block_job_cancel_async(BlockJob *job, bool force) +@@ -424,8 +335,8 @@ static int block_job_finish_sync(BlockJob *job, + while (!job_is_completed(&job->job)) { + aio_poll(qemu_get_aio_context(), true); + } +- ret = (job_is_cancelled(&job->job) && job->ret == 0) +- ? -ECANCELED : job->ret; ++ ret = (job_is_cancelled(&job->job) && job->job.ret == 0) ++ ? -ECANCELED : job->job.ret; + job_unref(&job->job); + return ret; + } +@@ -466,7 +377,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + assert(job_is_cancelled(&other_job->job)); + block_job_finish_sync(other_job, NULL, NULL); + } +- block_job_finalize_single(other_job); ++ job_finalize_single(&other_job->job); + aio_context_release(ctx); + } + +@@ -478,6 +389,11 @@ static int block_job_needs_finalize(BlockJob *job) + return !job->job.auto_finalize; + } + ++static int block_job_finalize_single(BlockJob *job) ++{ ++ return job_finalize_single(&job->job); ++} ++ + static void block_job_do_finalize(BlockJob *job) + { + int rc; +@@ -516,7 +432,7 @@ static void block_job_completed_txn_success(BlockJob *job) + if (!job_is_completed(&other_job->job)) { + return; + } +- assert(other_job->ret == 0); ++ assert(other_job->job.ret == 0); + } + + block_job_txn_apply(txn, block_job_transition_to_pending, false); +@@ -601,14 +517,14 @@ void block_job_dismiss(BlockJob **jobptr, Error **errp) + return; + } + +- block_job_do_dismiss(job); ++ job_do_dismiss(&job->job); + *jobptr = NULL; + } + + void block_job_cancel(BlockJob *job, bool force) + { + if (job->job.status == JOB_STATUS_CONCLUDED) { +- block_job_do_dismiss(job); ++ job_do_dismiss(&job->job); + return; + } + block_job_cancel_async(job, force); +@@ -691,8 +607,8 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info->status = job->job.status; + info->auto_finalize = job->job.auto_finalize; + info->auto_dismiss = job->job.auto_dismiss; +- info->has_error = job->ret != 0; +- info->error = job->ret ? g_strdup(strerror(-job->ret)) : NULL; ++ info->has_error = job->job.ret != 0; ++ info->error = job->job.ret ? g_strdup(strerror(-job->job.ret)) : NULL; + return info; + } + +@@ -729,8 +645,8 @@ static void block_job_event_completed(Notifier *n, void *opaque) + return; + } + +- if (job->ret < 0) { +- msg = strerror(-job->ret); ++ if (job->job.ret < 0) { ++ msg = strerror(-job->job.ret); + } + + qapi_event_send_block_job_completed(job_type(&job->job), +@@ -787,7 +703,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + } + + job = job_create(job_id, &driver->job_driver, blk_get_aio_context(blk), +- flags, errp); ++ flags, cb, opaque, errp); + if (job == NULL) { + blk_unref(blk); + return NULL; +@@ -799,8 +715,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + + job->driver = driver; + job->blk = blk; +- job->cb = cb; +- job->opaque = opaque; + + job->finalize_cancelled_notifier.notify = block_job_event_cancelled; + job->finalize_completed_notifier.notify = block_job_event_completed; +@@ -828,7 +742,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + + block_job_set_speed(job, speed, &local_err); + if (local_err) { +- block_job_early_fail(job); ++ job_early_fail(&job->job); + error_propagate(errp, local_err); + return NULL; + } +@@ -847,20 +761,14 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return job; + } + +-void block_job_early_fail(BlockJob *job) +-{ +- assert(job->job.status == JOB_STATUS_CREATED); +- block_job_decommission(job); +-} +- + void block_job_completed(BlockJob *job, int ret) + { + assert(job && job->txn && !job_is_completed(&job->job)); + assert(blk_bs(job->blk)->job == job); +- job->ret = ret; +- block_job_update_rc(job); +- trace_block_job_completed(job, ret, job->ret); +- if (job->ret) { ++ job->job.ret = ret; ++ job_update_rc(&job->job); ++ trace_block_job_completed(job, ret, job->job.ret); ++ if (job->job.ret) { + block_job_completed_txn_abort(job); + } else { + block_job_completed_txn_success(job); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index aef0629..3f405d1 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -76,9 +76,6 @@ typedef struct BlockJob { + /** Rate limiting data structure for implementing @speed. */ + RateLimit limit; + +- /** The completion function that will be called when the job completes. */ +- BlockCompletionFunc *cb; +- + /** Block other operations when block job is running */ + Error *blocker; + +@@ -94,12 +91,6 @@ typedef struct BlockJob { + /** BlockDriverStates that are involved in this block job */ + GSList *nodes; + +- /** The opaque value that is passed to the completion function. */ +- void *opaque; +- +- /** ret code passed to block_job_completed. */ +- int ret; +- + BlockJobTxn *txn; + QLIST_ENTRY(BlockJob) txn_list; + } BlockJob; +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index 88639f7..bf2b762 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -54,34 +54,6 @@ struct BlockJobDriver { + */ + int (*prepare)(BlockJob *job); + +- /** +- * If the callback is not NULL, it will be invoked when all the jobs +- * belonging to the same transaction complete; or upon this job's +- * completion if it is not in a transaction. Skipped if NULL. +- * +- * All jobs will complete with a call to either .commit() or .abort() but +- * never both. +- */ +- void (*commit)(BlockJob *job); +- +- /** +- * If the callback is not NULL, it will be invoked when any job in the +- * same transaction fails; or upon this job's failure (due to error or +- * cancellation) if it is not in a transaction. Skipped if NULL. +- * +- * All jobs will complete with a call to either .commit() or .abort() but +- * never both. +- */ +- void (*abort)(BlockJob *job); +- +- /** +- * If the callback is not NULL, it will be invoked after a call to either +- * .commit() or .abort(). Regardless of which callback is invoked after +- * completion, .clean() will always be called, even if the job does not +- * belong to a transaction group. +- */ +- void (*clean)(BlockJob *job); +- + /* + * If the callback is not NULL, it will be invoked before the job is + * resumed in a new AioContext. This is the place to move any resources +@@ -156,14 +128,6 @@ void block_job_yield(BlockJob *job); + int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n); + + /** +- * block_job_early_fail: +- * @bs: The block device. +- * +- * The block job could not be started, free it. +- */ +-void block_job_early_fail(BlockJob *job); +- +-/** + * block_job_completed: + * @job: The job being completed. + * @ret: The status code. +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 14d9377..3e817be 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -29,6 +29,7 @@ + #include "qapi/qapi-types-block-core.h" + #include "qemu/queue.h" + #include "qemu/coroutine.h" ++#include "block/aio.h" + + typedef struct JobDriver JobDriver; + +@@ -105,6 +106,15 @@ typedef struct Job { + /** True if this job should automatically dismiss itself */ + bool auto_dismiss; + ++ /** ret code passed to block_job_completed. */ ++ int ret; ++ ++ /** The completion function that will be called when the job completes. */ ++ BlockCompletionFunc *cb; ++ ++ /** The opaque value that is passed to the completion function. */ ++ void *opaque; ++ + /** Notifiers called when a cancelled job is finalised */ + NotifierList on_finalize_cancelled; + +@@ -151,6 +161,35 @@ struct JobDriver { + */ + void (*user_resume)(Job *job); + ++ /** ++ * If the callback is not NULL, it will be invoked when all the jobs ++ * belonging to the same transaction complete; or upon this job's ++ * completion if it is not in a transaction. Skipped if NULL. ++ * ++ * All jobs will complete with a call to either .commit() or .abort() but ++ * never both. ++ */ ++ void (*commit)(Job *job); ++ ++ /** ++ * If the callback is not NULL, it will be invoked when any job in the ++ * same transaction fails; or upon this job's failure (due to error or ++ * cancellation) if it is not in a transaction. Skipped if NULL. ++ * ++ * All jobs will complete with a call to either .commit() or .abort() but ++ * never both. ++ */ ++ void (*abort)(Job *job); ++ ++ /** ++ * If the callback is not NULL, it will be invoked after a call to either ++ * .commit() or .abort(). Regardless of which callback is invoked after ++ * completion, .clean() will always be called, even if the job does not ++ * belong to a transaction group. ++ */ ++ void (*clean)(Job *job); ++ ++ + /** Called when the job is freed */ + void (*free)(Job *job); + }; +@@ -174,10 +213,12 @@ typedef enum JobCreateFlags { + * @driver: The class object for the newly-created job. + * @ctx: The AioContext to run the job coroutine in. + * @flags: Creation flags for the job. See @JobCreateFlags. ++ * @cb: Completion function for the job. ++ * @opaque: Opaque pointer value passed to @cb. + * @errp: Error object. + */ + void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, +- int flags, Error **errp); ++ int flags, BlockCompletionFunc *cb, void *opaque, Error **errp); + + /** + * Add a reference to Job refcnt, it will be decreased with job_unref, and then +@@ -300,6 +341,10 @@ Job *job_get(const char *id); + */ + int job_apply_verb(Job *job, JobVerb verb, Error **errp); + ++/** The @job could not be started, free it. */ ++void job_early_fail(Job *job); ++ ++ + typedef void JobDeferToMainLoopFn(Job *job, void *opaque); + + /** +@@ -322,5 +367,11 @@ void job_state_transition(Job *job, JobStatus s1); + void coroutine_fn job_do_yield(Job *job, uint64_t ns); + bool job_should_pause(Job *job); + bool job_started(Job *job); ++void job_do_dismiss(Job *job); ++int job_finalize_single(Job *job); ++void job_update_rc(Job *job); ++ ++typedef struct BlockJob BlockJob; ++void block_job_txn_del_job(BlockJob *job); + + #endif +diff --git a/job.c b/job.c +index 817c3b4..64b64da 100644 +--- a/job.c ++++ b/job.c +@@ -85,7 +85,7 @@ void job_state_transition(Job *job, JobStatus s1) + { + JobStatus s0 = job->status; + assert(s1 >= 0 && s1 <= JOB_STATUS__MAX); +- trace_job_state_transition(job, /* TODO re-enable: job->ret */ 0, ++ trace_job_state_transition(job, job->ret, + JobSTT[s0][s1] ? "allowed" : "disallowed", + JobStatus_str(s0), JobStatus_str(s1)); + assert(JobSTT[s0][s1]); +@@ -182,7 +182,7 @@ static void job_sleep_timer_cb(void *opaque) + } + + void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, +- int flags, Error **errp) ++ int flags, BlockCompletionFunc *cb, void *opaque, Error **errp) + { + Job *job; + +@@ -214,6 +214,8 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + job->pause_count = 1; + job->auto_finalize = !(flags & JOB_MANUAL_FINALIZE); + job->auto_dismiss = !(flags & JOB_MANUAL_DISMISS); ++ job->cb = cb; ++ job->opaque = opaque; + + notifier_list_init(&job->on_finalize_cancelled); + notifier_list_init(&job->on_finalize_completed); +@@ -449,6 +451,100 @@ void job_user_resume(Job *job, Error **errp) + job_resume(job); + } + ++void job_do_dismiss(Job *job) ++{ ++ assert(job); ++ job->busy = false; ++ job->paused = false; ++ job->deferred_to_main_loop = true; ++ ++ /* TODO Don't assume it's a BlockJob */ ++ block_job_txn_del_job((BlockJob*) job); ++ ++ job_state_transition(job, JOB_STATUS_NULL); ++ job_unref(job); ++} ++ ++void job_early_fail(Job *job) ++{ ++ assert(job->status == JOB_STATUS_CREATED); ++ job_do_dismiss(job); ++} ++ ++static void job_conclude(Job *job) ++{ ++ job_state_transition(job, JOB_STATUS_CONCLUDED); ++ if (job->auto_dismiss || !job_started(job)) { ++ job_do_dismiss(job); ++ } ++} ++ ++void job_update_rc(Job *job) ++{ ++ if (!job->ret && job_is_cancelled(job)) { ++ job->ret = -ECANCELED; ++ } ++ if (job->ret) { ++ job_state_transition(job, JOB_STATUS_ABORTING); ++ } ++} ++ ++static void job_commit(Job *job) ++{ ++ assert(!job->ret); ++ if (job->driver->commit) { ++ job->driver->commit(job); ++ } ++} ++ ++static void job_abort(Job *job) ++{ ++ assert(job->ret); ++ if (job->driver->abort) { ++ job->driver->abort(job); ++ } ++} ++ ++static void job_clean(Job *job) ++{ ++ if (job->driver->clean) { ++ job->driver->clean(job); ++ } ++} ++ ++int job_finalize_single(Job *job) ++{ ++ assert(job_is_completed(job)); ++ ++ /* Ensure abort is called for late-transactional failures */ ++ job_update_rc(job); ++ ++ if (!job->ret) { ++ job_commit(job); ++ } else { ++ job_abort(job); ++ } ++ job_clean(job); ++ ++ if (job->cb) { ++ job->cb(job->opaque, job->ret); ++ } ++ ++ /* Emit events only if we actually started */ ++ if (job_started(job)) { ++ if (job_is_cancelled(job)) { ++ job_event_cancelled(job); ++ } else { ++ job_event_completed(job); ++ } ++ } ++ ++ /* TODO Don't assume it's a BlockJob */ ++ block_job_txn_del_job((BlockJob*) job); ++ job_conclude(job); ++ return 0; ++} ++ + + typedef struct { + Job *job; +diff --git a/qemu-img.c b/qemu-img.c +index 843dc6a..91b3151 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -883,7 +883,7 @@ static void run_block_job(BlockJob *job, Error **errp) + if (!job_is_completed(&job->job)) { + ret = block_job_complete_sync(job, errp); + } else { +- ret = job->ret; ++ ret = job->job.ret; + } + job_unref(&job->job); + aio_context_release(aio_context); +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 8bb0aa8..1fe6803 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -128,11 +128,11 @@ static void test_job_ids(void) + job[1] = do_test_id(blk[1], "id0", false); + + /* But once job[0] finishes we can reuse its ID */ +- block_job_early_fail(job[0]); ++ job_early_fail(&job[0]->job); + job[1] = do_test_id(blk[1], "id0", true); + + /* No job ID specified, defaults to the backend name ('drive1') */ +- block_job_early_fail(job[1]); ++ job_early_fail(&job[1]->job); + job[1] = do_test_id(blk[1], NULL, true); + + /* Duplicate job ID */ +@@ -145,9 +145,9 @@ static void test_job_ids(void) + /* This one is valid */ + job[2] = do_test_id(blk[2], "id_2", true); + +- block_job_early_fail(job[0]); +- block_job_early_fail(job[1]); +- block_job_early_fail(job[2]); ++ job_early_fail(&job[0]->job); ++ job_early_fail(&job[1]->job); ++ job_early_fail(&job[2]->job); + + destroy_blk(blk[0]); + destroy_blk(blk[1]); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-state-transitions-to-Job.patch b/SOURCES/kvm-job-Move-state-transitions-to-Job.patch new file mode 100644 index 0000000..a7272dc --- /dev/null +++ b/SOURCES/kvm-job-Move-state-transitions-to-Job.patch @@ -0,0 +1,640 @@ +From f0ec16e0035297f89177beaeb55995fae4131926 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:04 +0200 +Subject: [PATCH 096/268] job: Move state transitions to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-22-kwolf@redhat.com> +Patchwork-id: 81093 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 21/73] job: Move state transitions to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves BlockJob.status and the closely related functions +(block_)job_state_transition() and (block_)job_apply_verb to Job. The +two QAPI enums are renamed to JobStatus and JobVerb. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +Reviewed-by: Eric Blake +(cherry picked from commit a50c2ab858fe613fb805e53b4f6b970ab936706d) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/trace-events | 2 - + blockjob.c | 102 +++++++++++------------------------------------ + include/block/blockjob.h | 3 -- + include/qemu/job.h | 13 ++++++ + job.c | 56 ++++++++++++++++++++++++++ + qapi/block-core.json | 16 ++++---- + tests/test-blockjob.c | 39 +++++++++--------- + trace-events | 4 ++ + 8 files changed, 123 insertions(+), 112 deletions(-) + +diff --git a/block/trace-events b/block/trace-events +index f8c50b4..93b9279 100644 +--- a/block/trace-events ++++ b/block/trace-events +@@ -6,8 +6,6 @@ bdrv_lock_medium(void *bs, bool locked) "bs %p locked %d" + + # blockjob.c + block_job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" +-block_job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" +-block_job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" + + # block/block-backend.c + blk_co_preadv(void *blk, void *bs, int64_t offset, unsigned int bytes, int flags) "blk %p bs %p offset %"PRId64" bytes %u flags 0x%x" +diff --git a/blockjob.c b/blockjob.c +index c69b2e7..0fba01e 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -41,61 +41,6 @@ + * block_job_enter. */ + static QemuMutex block_job_mutex; + +-/* BlockJob State Transition Table */ +-bool BlockJobSTT[BLOCK_JOB_STATUS__MAX][BLOCK_JOB_STATUS__MAX] = { +- /* U, C, R, P, Y, S, W, D, X, E, N */ +- /* U: */ [BLOCK_JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0}, +- /* C: */ [BLOCK_JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1}, +- /* R: */ [BLOCK_JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0}, +- /* P: */ [BLOCK_JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0}, +- /* Y: */ [BLOCK_JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0}, +- /* S: */ [BLOCK_JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, +- /* W: */ [BLOCK_JOB_STATUS_WAITING] = {0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0}, +- /* D: */ [BLOCK_JOB_STATUS_PENDING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, +- /* X: */ [BLOCK_JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, +- /* E: */ [BLOCK_JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, +- /* N: */ [BLOCK_JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, +-}; +- +-bool BlockJobVerbTable[BLOCK_JOB_VERB__MAX][BLOCK_JOB_STATUS__MAX] = { +- /* U, C, R, P, Y, S, W, D, X, E, N */ +- [BLOCK_JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0}, +- [BLOCK_JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, +- [BLOCK_JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, +- [BLOCK_JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, +- [BLOCK_JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, +- [BLOCK_JOB_VERB_FINALIZE] = {0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0}, +- [BLOCK_JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, +-}; +- +-static void block_job_state_transition(BlockJob *job, BlockJobStatus s1) +-{ +- BlockJobStatus s0 = job->status; +- assert(s1 >= 0 && s1 <= BLOCK_JOB_STATUS__MAX); +- trace_block_job_state_transition(job, job->ret, BlockJobSTT[s0][s1] ? +- "allowed" : "disallowed", +- BlockJobStatus_str(s0), +- BlockJobStatus_str(s1)); +- assert(BlockJobSTT[s0][s1]); +- job->status = s1; +-} +- +-static int block_job_apply_verb(BlockJob *job, BlockJobVerb bv, Error **errp) +-{ +- assert(bv >= 0 && bv <= BLOCK_JOB_VERB__MAX); +- trace_block_job_apply_verb(job, BlockJobStatus_str(job->status), +- BlockJobVerb_str(bv), +- BlockJobVerbTable[bv][job->status] ? +- "allowed" : "prohibited"); +- if (BlockJobVerbTable[bv][job->status]) { +- return 0; +- } +- error_setg(errp, "Job '%s' in state '%s' cannot accept command verb '%s'", +- job->job.id, BlockJobStatus_str(job->status), +- BlockJobVerb_str(bv)); +- return -EPERM; +-} +- + static void block_job_lock(void) + { + qemu_mutex_lock(&block_job_mutex); +@@ -257,7 +202,7 @@ static void block_job_detach_aio_context(void *opaque); + void block_job_unref(BlockJob *job) + { + if (--job->refcnt == 0) { +- assert(job->status == BLOCK_JOB_STATUS_NULL); ++ assert(job->job.status == JOB_STATUS_NULL); + assert(!job->txn); + BlockDriverState *bs = blk_bs(job->blk); + bs->job = NULL; +@@ -409,7 +354,7 @@ void block_job_start(BlockJob *job) + job->pause_count--; + job->busy = true; + job->paused = false; +- block_job_state_transition(job, BLOCK_JOB_STATUS_RUNNING); ++ job_state_transition(&job->job, JOB_STATUS_RUNNING); + bdrv_coroutine_enter(blk_bs(job->blk), job->co); + } + +@@ -421,7 +366,7 @@ static void block_job_decommission(BlockJob *job) + job->paused = false; + job->deferred_to_main_loop = true; + block_job_txn_del_job(job); +- block_job_state_transition(job, BLOCK_JOB_STATUS_NULL); ++ job_state_transition(&job->job, JOB_STATUS_NULL); + block_job_unref(job); + } + +@@ -432,7 +377,7 @@ static void block_job_do_dismiss(BlockJob *job) + + static void block_job_conclude(BlockJob *job) + { +- block_job_state_transition(job, BLOCK_JOB_STATUS_CONCLUDED); ++ job_state_transition(&job->job, JOB_STATUS_CONCLUDED); + if (job->auto_dismiss || !block_job_started(job)) { + block_job_do_dismiss(job); + } +@@ -444,7 +389,7 @@ static void block_job_update_rc(BlockJob *job) + job->ret = -ECANCELED; + } + if (job->ret) { +- block_job_state_transition(job, BLOCK_JOB_STATUS_ABORTING); ++ job_state_transition(&job->job, JOB_STATUS_ABORTING); + } + } + +@@ -652,7 +597,7 @@ static void block_job_completed_txn_success(BlockJob *job) + BlockJobTxn *txn = job->txn; + BlockJob *other_job; + +- block_job_state_transition(job, BLOCK_JOB_STATUS_WAITING); ++ job_state_transition(&job->job, JOB_STATUS_WAITING); + + /* + * Successful completion, see if there are other running jobs in this +@@ -677,7 +622,7 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp) + { + int64_t old_speed = job->speed; + +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_SET_SPEED, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_SET_SPEED, errp)) { + return; + } + if (speed < 0) { +@@ -709,7 +654,7 @@ void block_job_complete(BlockJob *job, Error **errp) + { + /* Should not be reachable via external interface for internal jobs */ + assert(job->job.id); +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_COMPLETE, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_COMPLETE, errp)) { + return; + } + if (job->pause_count || job->cancelled || !job->driver->complete) { +@@ -724,7 +669,7 @@ void block_job_complete(BlockJob *job, Error **errp) + void block_job_finalize(BlockJob *job, Error **errp) + { + assert(job && job->job.id); +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_FINALIZE, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_FINALIZE, errp)) { + return; + } + block_job_do_finalize(job); +@@ -735,7 +680,7 @@ void block_job_dismiss(BlockJob **jobptr, Error **errp) + BlockJob *job = *jobptr; + /* similarly to _complete, this is QMP-interface only. */ + assert(job->job.id); +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_DISMISS, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_DISMISS, errp)) { + return; + } + +@@ -745,7 +690,7 @@ void block_job_dismiss(BlockJob **jobptr, Error **errp) + + void block_job_user_pause(BlockJob *job, Error **errp) + { +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_PAUSE, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_PAUSE, errp)) { + return; + } + if (job->user_paused) { +@@ -768,7 +713,7 @@ void block_job_user_resume(BlockJob *job, Error **errp) + error_setg(errp, "Can't resume a job that was not paused"); + return; + } +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_RESUME, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_RESUME, errp)) { + return; + } + block_job_iostatus_reset(job); +@@ -778,7 +723,7 @@ void block_job_user_resume(BlockJob *job, Error **errp) + + void block_job_cancel(BlockJob *job, bool force) + { +- if (job->status == BLOCK_JOB_STATUS_CONCLUDED) { ++ if (job->job.status == JOB_STATUS_CONCLUDED) { + block_job_do_dismiss(job); + return; + } +@@ -794,7 +739,7 @@ void block_job_cancel(BlockJob *job, bool force) + + void block_job_user_cancel(BlockJob *job, bool force, Error **errp) + { +- if (block_job_apply_verb(job, BLOCK_JOB_VERB_CANCEL, errp)) { ++ if (job_apply_verb(&job->job, JOB_VERB_CANCEL, errp)) { + return; + } + block_job_cancel(job, force); +@@ -859,7 +804,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + info->speed = job->speed; + info->io_status = job->iostatus; + info->ready = job->ready; +- info->status = job->status; ++ info->status = job->job.status; + info->auto_finalize = job->auto_finalize; + info->auto_dismiss = job->auto_dismiss; + info->has_error = job->ret != 0; +@@ -907,7 +852,7 @@ static void block_job_event_completed(BlockJob *job, const char *msg) + + static int block_job_event_pending(BlockJob *job) + { +- block_job_state_transition(job, BLOCK_JOB_STATUS_PENDING); ++ job_state_transition(&job->job, JOB_STATUS_PENDING); + if (!job->auto_finalize && !block_job_is_internal(job)) { + qapi_event_send_block_job_pending(job_type(&job->job), + job->job.id, +@@ -975,7 +920,6 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + job->refcnt = 1; + job->auto_finalize = !(flags & BLOCK_JOB_MANUAL_FINALIZE); + job->auto_dismiss = !(flags & BLOCK_JOB_MANUAL_DISMISS); +- block_job_state_transition(job, BLOCK_JOB_STATUS_CREATED); + aio_timer_init(qemu_get_aio_context(), &job->sleep_timer, + QEMU_CLOCK_REALTIME, SCALE_NS, + block_job_sleep_timer_cb, job); +@@ -1017,7 +961,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + + void block_job_early_fail(BlockJob *job) + { +- assert(job->status == BLOCK_JOB_STATUS_CREATED); ++ assert(job->job.status == JOB_STATUS_CREATED); + block_job_decommission(job); + } + +@@ -1077,14 +1021,14 @@ void coroutine_fn block_job_pause_point(BlockJob *job) + } + + if (block_job_should_pause(job) && !block_job_is_cancelled(job)) { +- BlockJobStatus status = job->status; +- block_job_state_transition(job, status == BLOCK_JOB_STATUS_READY ? \ +- BLOCK_JOB_STATUS_STANDBY : \ +- BLOCK_JOB_STATUS_PAUSED); ++ JobStatus status = job->job.status; ++ job_state_transition(&job->job, status == JOB_STATUS_READY ++ ? JOB_STATUS_STANDBY ++ : JOB_STATUS_PAUSED); + job->paused = true; + block_job_do_yield(job, -1); + job->paused = false; +- block_job_state_transition(job, status); ++ job_state_transition(&job->job, status); + } + + if (job->driver->resume) { +@@ -1176,7 +1120,7 @@ void block_job_iostatus_reset(BlockJob *job) + + void block_job_event_ready(BlockJob *job) + { +- block_job_state_transition(job, BLOCK_JOB_STATUS_READY); ++ job_state_transition(&job->job, JOB_STATUS_READY); + job->ready = true; + + if (block_job_is_internal(job)) { +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 10bd9f7..01cdee6 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -147,9 +147,6 @@ typedef struct BlockJob { + */ + QEMUTimer sleep_timer; + +- /** Current state; See @BlockJobStatus for details. */ +- BlockJobStatus status; +- + /** True if this job should automatically finalize itself */ + bool auto_finalize; + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index bae2b09..0b78778 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -41,6 +41,9 @@ typedef struct Job { + /** The type of this job. */ + const JobDriver *driver; + ++ /** Current state; See @JobStatus for details. */ ++ JobStatus status; ++ + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + } Job; +@@ -90,4 +93,14 @@ Job *job_next(Job *job); + */ + Job *job_get(const char *id); + ++/** ++ * Check whether the verb @verb can be applied to @job in its current state. ++ * Returns 0 if the verb can be applied; otherwise errp is set and -EPERM ++ * returned. ++ */ ++int job_apply_verb(Job *job, JobVerb verb, Error **errp); ++ ++/* TODO To be removed from the public interface */ ++void job_state_transition(Job *job, JobStatus s1); ++ + #endif +diff --git a/job.c b/job.c +index e57303c..b049a32 100644 +--- a/job.c ++++ b/job.c +@@ -28,9 +28,63 @@ + #include "qapi/error.h" + #include "qemu/job.h" + #include "qemu/id.h" ++#include "trace-root.h" + + static QLIST_HEAD(, Job) jobs = QLIST_HEAD_INITIALIZER(jobs); + ++/* Job State Transition Table */ ++bool JobSTT[JOB_STATUS__MAX][JOB_STATUS__MAX] = { ++ /* U, C, R, P, Y, S, W, D, X, E, N */ ++ /* U: */ [JOB_STATUS_UNDEFINED] = {0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0}, ++ /* C: */ [JOB_STATUS_CREATED] = {0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1}, ++ /* R: */ [JOB_STATUS_RUNNING] = {0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0}, ++ /* P: */ [JOB_STATUS_PAUSED] = {0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0}, ++ /* Y: */ [JOB_STATUS_READY] = {0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0}, ++ /* S: */ [JOB_STATUS_STANDBY] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, ++ /* W: */ [JOB_STATUS_WAITING] = {0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0}, ++ /* D: */ [JOB_STATUS_PENDING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, ++ /* X: */ [JOB_STATUS_ABORTING] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0}, ++ /* E: */ [JOB_STATUS_CONCLUDED] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1}, ++ /* N: */ [JOB_STATUS_NULL] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, ++}; ++ ++bool JobVerbTable[JOB_VERB__MAX][JOB_STATUS__MAX] = { ++ /* U, C, R, P, Y, S, W, D, X, E, N */ ++ [JOB_VERB_CANCEL] = {0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0}, ++ [JOB_VERB_PAUSE] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, ++ [JOB_VERB_RESUME] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, ++ [JOB_VERB_SET_SPEED] = {0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0}, ++ [JOB_VERB_COMPLETE] = {0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0}, ++ [JOB_VERB_FINALIZE] = {0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0}, ++ [JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, ++}; ++ ++/* TODO Make static once the whole state machine is in job.c */ ++void job_state_transition(Job *job, JobStatus s1) ++{ ++ JobStatus s0 = job->status; ++ assert(s1 >= 0 && s1 <= JOB_STATUS__MAX); ++ trace_job_state_transition(job, /* TODO re-enable: job->ret */ 0, ++ JobSTT[s0][s1] ? "allowed" : "disallowed", ++ JobStatus_str(s0), JobStatus_str(s1)); ++ assert(JobSTT[s0][s1]); ++ job->status = s1; ++} ++ ++int job_apply_verb(Job *job, JobVerb verb, Error **errp) ++{ ++ JobStatus s0 = job->status; ++ assert(verb >= 0 && verb <= JOB_VERB__MAX); ++ trace_job_apply_verb(job, JobStatus_str(s0), JobVerb_str(verb), ++ JobVerbTable[verb][s0] ? "allowed" : "prohibited"); ++ if (JobVerbTable[verb][s0]) { ++ return 0; ++ } ++ error_setg(errp, "Job '%s' in state '%s' cannot accept command verb '%s'", ++ job->id, JobStatus_str(s0), JobVerb_str(verb)); ++ return -EPERM; ++} ++ + JobType job_type(const Job *job) + { + return job->driver->job_type; +@@ -81,6 +135,8 @@ void *job_create(const char *job_id, const JobDriver *driver, Error **errp) + job->driver = driver; + job->id = g_strdup(job_id); + ++ job_state_transition(job, JOB_STATUS_CREATED); ++ + QLIST_INSERT_HEAD(&jobs, job, job_list); + + return job; +diff --git a/qapi/block-core.json b/qapi/block-core.json +index cab0697..f2ed7a8 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1068,9 +1068,9 @@ + 'data': ['commit', 'stream', 'mirror', 'backup'] } + + ## +-# @BlockJobVerb: ++# @JobVerb: + # +-# Represents command verbs that can be applied to a blockjob. ++# Represents command verbs that can be applied to a job. + # + # @cancel: see @block-job-cancel + # +@@ -1088,14 +1088,14 @@ + # + # Since: 2.12 + ## +-{ 'enum': 'BlockJobVerb', ++{ 'enum': 'JobVerb', + 'data': ['cancel', 'pause', 'resume', 'set-speed', 'complete', 'dismiss', + 'finalize' ] } + + ## +-# @BlockJobStatus: ++# @JobStatus: + # +-# Indicates the present state of a given blockjob in its lifetime. ++# Indicates the present state of a given job in its lifetime. + # + # @undefined: Erroneous, default state. Should not ever be visible. + # +@@ -1134,7 +1134,7 @@ + # + # Since: 2.12 + ## +-{ 'enum': 'BlockJobStatus', ++{ 'enum': 'JobStatus', + 'data': ['undefined', 'created', 'running', 'paused', 'ready', 'standby', + 'waiting', 'pending', 'aborting', 'concluded', 'null' ] } + +@@ -1184,7 +1184,7 @@ + 'data': {'type': 'str', 'device': 'str', 'len': 'int', + 'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int', + 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool', +- 'status': 'BlockJobStatus', ++ 'status': 'JobStatus', + 'auto-finalize': 'bool', 'auto-dismiss': 'bool', + '*error': 'str' } } + +@@ -2416,7 +2416,7 @@ + # QEMU 2.12+ job lifetime management semantics. + # + # This command will refuse to operate on any job that has not yet reached +-# its terminal state, BLOCK_JOB_STATUS_CONCLUDED. For jobs that make use of ++# its terminal state, JOB_STATUS_CONCLUDED. For jobs that make use of the + # BLOCK_JOB_READY event, block-job-cancel or block-job-complete will still need + # to be used as appropriate. + # +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index b820261..6ccd585 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -211,7 +211,7 @@ static CancelJob *create_common(BlockJob **pjob) + job = mk_job(blk, "Steve", &test_cancel_driver, true, + BLOCK_JOB_MANUAL_FINALIZE | BLOCK_JOB_MANUAL_DISMISS); + block_job_ref(job); +- assert(job->status == BLOCK_JOB_STATUS_CREATED); ++ assert(job->job.status == JOB_STATUS_CREATED); + s = container_of(job, CancelJob, common); + s->blk = blk; + +@@ -223,15 +223,14 @@ static void cancel_common(CancelJob *s) + { + BlockJob *job = &s->common; + BlockBackend *blk = s->blk; +- BlockJobStatus sts = job->status; ++ JobStatus sts = job->job.status; + + block_job_cancel_sync(job); +- if ((sts != BLOCK_JOB_STATUS_CREATED) && +- (sts != BLOCK_JOB_STATUS_CONCLUDED)) { ++ if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { + BlockJob *dummy = job; + block_job_dismiss(&dummy, &error_abort); + } +- assert(job->status == BLOCK_JOB_STATUS_NULL); ++ assert(job->job.status == JOB_STATUS_NULL); + block_job_unref(job); + destroy_blk(blk); + } +@@ -253,7 +252,7 @@ static void test_cancel_running(void) + s = create_common(&job); + + block_job_start(job); +- assert(job->status == BLOCK_JOB_STATUS_RUNNING); ++ assert(job->job.status == JOB_STATUS_RUNNING); + + cancel_common(s); + } +@@ -266,11 +265,11 @@ static void test_cancel_paused(void) + s = create_common(&job); + + block_job_start(job); +- assert(job->status == BLOCK_JOB_STATUS_RUNNING); ++ assert(job->job.status == JOB_STATUS_RUNNING); + + block_job_user_pause(job, &error_abort); + block_job_enter(job); +- assert(job->status == BLOCK_JOB_STATUS_PAUSED); ++ assert(job->job.status == JOB_STATUS_PAUSED); + + cancel_common(s); + } +@@ -283,11 +282,11 @@ static void test_cancel_ready(void) + s = create_common(&job); + + block_job_start(job); +- assert(job->status == BLOCK_JOB_STATUS_RUNNING); ++ assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; + block_job_enter(job); +- assert(job->status == BLOCK_JOB_STATUS_READY); ++ assert(job->job.status == JOB_STATUS_READY); + + cancel_common(s); + } +@@ -300,15 +299,15 @@ static void test_cancel_standby(void) + s = create_common(&job); + + block_job_start(job); +- assert(job->status == BLOCK_JOB_STATUS_RUNNING); ++ assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; + block_job_enter(job); +- assert(job->status == BLOCK_JOB_STATUS_READY); ++ assert(job->job.status == JOB_STATUS_READY); + + block_job_user_pause(job, &error_abort); + block_job_enter(job); +- assert(job->status == BLOCK_JOB_STATUS_STANDBY); ++ assert(job->job.status == JOB_STATUS_STANDBY); + + cancel_common(s); + } +@@ -321,18 +320,18 @@ static void test_cancel_pending(void) + s = create_common(&job); + + block_job_start(job); +- assert(job->status == BLOCK_JOB_STATUS_RUNNING); ++ assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; + block_job_enter(job); +- assert(job->status == BLOCK_JOB_STATUS_READY); ++ assert(job->job.status == JOB_STATUS_READY); + + block_job_complete(job, &error_abort); + block_job_enter(job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); + } +- assert(job->status == BLOCK_JOB_STATUS_PENDING); ++ assert(job->job.status == JOB_STATUS_PENDING); + + cancel_common(s); + } +@@ -345,21 +344,21 @@ static void test_cancel_concluded(void) + s = create_common(&job); + + block_job_start(job); +- assert(job->status == BLOCK_JOB_STATUS_RUNNING); ++ assert(job->job.status == JOB_STATUS_RUNNING); + + s->should_converge = true; + block_job_enter(job); +- assert(job->status == BLOCK_JOB_STATUS_READY); ++ assert(job->job.status == JOB_STATUS_READY); + + block_job_complete(job, &error_abort); + block_job_enter(job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); + } +- assert(job->status == BLOCK_JOB_STATUS_PENDING); ++ assert(job->job.status == JOB_STATUS_PENDING); + + block_job_finalize(job, &error_abort); +- assert(job->status == BLOCK_JOB_STATUS_CONCLUDED); ++ assert(job->job.status == JOB_STATUS_CONCLUDED); + + cancel_common(s); + } +diff --git a/trace-events b/trace-events +index ed71f44..2507e13 100644 +--- a/trace-events ++++ b/trace-events +@@ -104,6 +104,10 @@ gdbstub_err_invalid_rle(void) "got invalid RLE sequence" + gdbstub_err_checksum_invalid(uint8_t ch) "got invalid command checksum digit: 0x%02x" + gdbstub_err_checksum_incorrect(uint8_t expected, uint8_t got) "got command packet with incorrect checksum, expected=0x%02x, received=0x%02x" + ++# job.c ++job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" ++job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" ++ + ### Guest events, keep at bottom + + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Move-transactions-to-Job.patch b/SOURCES/kvm-job-Move-transactions-to-Job.patch new file mode 100644 index 0000000..d86742c --- /dev/null +++ b/SOURCES/kvm-job-Move-transactions-to-Job.patch @@ -0,0 +1,995 @@ +From 7ccb50f6be019258bfe5bb501142e39c5a4f52e4 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:22 +0200 +Subject: [PATCH 114/268] job: Move transactions to Job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-40-kwolf@redhat.com> +Patchwork-id: 81097 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 39/73] job: Move transactions to Job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This moves the logic that implements job transactions from BlockJob to +Job. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 7eaa8fb57da96301f4a8ce176ad503f80efc7cc0) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockdev.c | 6 +- + blockjob.c | 238 +------------------------------------------ + include/block/blockjob.h | 54 ---------- + include/block/blockjob_int.h | 10 -- + include/qemu/job.h | 71 +++++++++++-- + job.c | 234 ++++++++++++++++++++++++++++++++++++++++-- + tests/test-blockjob-txn.c | 12 +-- + tests/test-blockjob.c | 2 +- + 8 files changed, 303 insertions(+), 324 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index 6efdb30..9aa2e79 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2256,7 +2256,7 @@ void qmp_transaction(TransactionActionList *dev_list, + */ + props = get_transaction_properties(props); + if (props->completion_mode != ACTION_COMPLETION_MODE_INDIVIDUAL) { +- block_job_txn = block_job_txn_new(); ++ block_job_txn = job_txn_new(); + } + + /* drain all i/o before any operations */ +@@ -2315,7 +2315,7 @@ exit: + if (!has_props) { + qapi_free_TransactionProperties(props); + } +- block_job_txn_unref(block_job_txn); ++ job_txn_unref(block_job_txn); + } + + void qmp_eject(bool has_device, const char *device, +@@ -3909,7 +3909,7 @@ void qmp_block_job_finalize(const char *id, Error **errp) + } + + trace_qmp_block_job_finalize(job); +- block_job_finalize(job, errp); ++ job_finalize(&job->job, errp); + aio_context_release(aio_context); + } + +diff --git a/blockjob.c b/blockjob.c +index bd35c4f..14b21c8 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -36,19 +36,6 @@ + #include "qemu/coroutine.h" + #include "qemu/timer.h" + +-/* Transactional group of block jobs */ +-struct JobTxn { +- +- /* Is this txn being cancelled? */ +- bool aborting; +- +- /* List of jobs */ +- QLIST_HEAD(, Job) jobs; +- +- /* Reference count */ +- int refcnt; +-}; +- + /* + * The block job API is composed of two categories of functions. + * +@@ -94,48 +81,6 @@ BlockJob *block_job_get(const char *id) + } + } + +-JobTxn *block_job_txn_new(void) +-{ +- JobTxn *txn = g_new0(JobTxn, 1); +- QLIST_INIT(&txn->jobs); +- txn->refcnt = 1; +- return txn; +-} +- +-static void block_job_txn_ref(JobTxn *txn) +-{ +- txn->refcnt++; +-} +- +-void block_job_txn_unref(JobTxn *txn) +-{ +- if (txn && --txn->refcnt == 0) { +- g_free(txn); +- } +-} +- +-void block_job_txn_add_job(JobTxn *txn, BlockJob *job) +-{ +- if (!txn) { +- return; +- } +- +- assert(!job->txn); +- job->txn = txn; +- +- QLIST_INSERT_HEAD(&txn->jobs, &job->job, txn_list); +- block_job_txn_ref(txn); +-} +- +-void block_job_txn_del_job(BlockJob *job) +-{ +- if (job->txn) { +- QLIST_REMOVE(&job->job, txn_list); +- block_job_txn_unref(job->txn); +- job->txn = NULL; +- } +-} +- + static void block_job_attached_aio_context(AioContext *new_context, + void *opaque); + static void block_job_detach_aio_context(void *opaque); +@@ -145,8 +90,6 @@ void block_job_free(Job *job) + BlockJob *bjob = container_of(job, BlockJob, job); + BlockDriverState *bs = blk_bs(bjob->blk); + +- assert(!bjob->txn); +- + bs->job = NULL; + block_job_remove_all_bdrv(bjob); + blk_remove_aio_context_notifier(bjob->blk, +@@ -261,158 +204,6 @@ const BlockJobDriver *block_job_driver(BlockJob *job) + return job->driver; + } + +-static int block_job_prepare(BlockJob *job) +-{ +- if (job->job.ret == 0 && job->driver->prepare) { +- job->job.ret = job->driver->prepare(job); +- } +- return job->job.ret; +-} +- +-static void job_cancel_async(Job *job, bool force) +-{ +- if (job->user_paused) { +- /* Do not call job_enter here, the caller will handle it. */ +- job->user_paused = false; +- if (job->driver->user_resume) { +- job->driver->user_resume(job); +- } +- assert(job->pause_count > 0); +- job->pause_count--; +- } +- job->cancelled = true; +- /* To prevent 'force == false' overriding a previous 'force == true' */ +- job->force_cancel |= force; +-} +- +-static int block_job_txn_apply(JobTxn *txn, int fn(BlockJob *), bool lock) +-{ +- AioContext *ctx; +- Job *job, *next; +- BlockJob *bjob; +- int rc = 0; +- +- QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { +- assert(is_block_job(job)); +- bjob = container_of(job, BlockJob, job); +- +- if (lock) { +- ctx = job->aio_context; +- aio_context_acquire(ctx); +- } +- rc = fn(bjob); +- if (lock) { +- aio_context_release(ctx); +- } +- if (rc) { +- break; +- } +- } +- return rc; +-} +- +-static void block_job_completed_txn_abort(BlockJob *job) +-{ +- AioContext *ctx; +- JobTxn *txn = job->txn; +- Job *other_job; +- +- if (txn->aborting) { +- /* +- * We are cancelled by another job, which will handle everything. +- */ +- return; +- } +- txn->aborting = true; +- block_job_txn_ref(txn); +- +- /* We are the first failed job. Cancel other jobs. */ +- QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- ctx = other_job->aio_context; +- aio_context_acquire(ctx); +- } +- +- /* Other jobs are effectively cancelled by us, set the status for +- * them; this job, however, may or may not be cancelled, depending +- * on the caller, so leave it. */ +- QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- if (other_job != &job->job) { +- job_cancel_async(other_job, false); +- } +- } +- while (!QLIST_EMPTY(&txn->jobs)) { +- other_job = QLIST_FIRST(&txn->jobs); +- ctx = other_job->aio_context; +- if (!job_is_completed(other_job)) { +- assert(job_is_cancelled(other_job)); +- job_finish_sync(other_job, NULL, NULL); +- } +- job_finalize_single(other_job); +- aio_context_release(ctx); +- } +- +- block_job_txn_unref(txn); +-} +- +-static int block_job_needs_finalize(BlockJob *job) +-{ +- return !job->job.auto_finalize; +-} +- +-static int block_job_finalize_single(BlockJob *job) +-{ +- return job_finalize_single(&job->job); +-} +- +-static void block_job_do_finalize(BlockJob *job) +-{ +- int rc; +- assert(job && job->txn); +- +- /* prepare the transaction to complete */ +- rc = block_job_txn_apply(job->txn, block_job_prepare, true); +- if (rc) { +- block_job_completed_txn_abort(job); +- } else { +- block_job_txn_apply(job->txn, block_job_finalize_single, true); +- } +-} +- +-static int block_job_transition_to_pending(BlockJob *job) +-{ +- job_state_transition(&job->job, JOB_STATUS_PENDING); +- if (!job->job.auto_finalize) { +- job_event_pending(&job->job); +- } +- return 0; +-} +- +-static void block_job_completed_txn_success(BlockJob *job) +-{ +- JobTxn *txn = job->txn; +- Job *other_job; +- +- job_state_transition(&job->job, JOB_STATUS_WAITING); +- +- /* +- * Successful completion, see if there are other running jobs in this +- * txn. +- */ +- QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- if (!job_is_completed(other_job)) { +- return; +- } +- assert(other_job->ret == 0); +- } +- +- block_job_txn_apply(txn, block_job_transition_to_pending, false); +- +- /* If no jobs need manual finalization, automatically do so */ +- if (block_job_txn_apply(txn, block_job_needs_finalize, false) == 0) { +- block_job_do_finalize(job); +- } +-} +- + /* Assumes the job_mutex is held */ + static bool job_timer_pending(Job *job) + { +@@ -451,15 +242,6 @@ int64_t block_job_ratelimit_get_delay(BlockJob *job, uint64_t n) + return ratelimit_calculate_delay(&job->limit, n); + } + +-void block_job_finalize(BlockJob *job, Error **errp) +-{ +- assert(job && job->job.id); +- if (job_apply_verb(&job->job, JOB_VERB_FINALIZE, errp)) { +- return; +- } +- block_job_do_finalize(job); +-} +- + void block_job_dismiss(BlockJob **jobptr, Error **errp) + { + BlockJob *job = *jobptr; +@@ -483,7 +265,7 @@ void block_job_cancel(BlockJob *job, bool force) + if (!job_started(&job->job)) { + block_job_completed(job, -ECANCELED); + } else if (job->job.deferred_to_main_loop) { +- block_job_completed_txn_abort(job); ++ job_completed_txn_abort(&job->job); + } else { + block_job_enter(job); + } +@@ -656,7 +438,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + return NULL; + } + +- job = job_create(job_id, &driver->job_driver, blk_get_aio_context(blk), ++ job = job_create(job_id, &driver->job_driver, txn, blk_get_aio_context(blk), + flags, cb, opaque, errp); + if (job == NULL) { + blk_unref(blk); +@@ -703,30 +485,20 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + } + } + +- /* Single jobs are modeled as single-job transactions for sake of +- * consolidating the job management logic */ +- if (!txn) { +- txn = block_job_txn_new(); +- block_job_txn_add_job(txn, job); +- block_job_txn_unref(txn); +- } else { +- block_job_txn_add_job(txn, job); +- } +- + return job; + } + + void block_job_completed(BlockJob *job, int ret) + { +- assert(job && job->txn && !job_is_completed(&job->job)); ++ assert(job && job->job.txn && !job_is_completed(&job->job)); + assert(blk_bs(job->blk)->job == job); + job->job.ret = ret; + job_update_rc(&job->job); + trace_block_job_completed(job, ret, job->job.ret); + if (job->job.ret) { +- block_job_completed_txn_abort(job); ++ job_completed_txn_abort(&job->job); + } else { +- block_job_completed_txn_success(job); ++ job_completed_txn_success(&job->job); + } + } + +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 44df025..09e6bb4 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -33,7 +33,6 @@ + #define BLOCK_JOB_SLICE_TIME 100000000ULL /* ns */ + + typedef struct BlockJobDriver BlockJobDriver; +-typedef struct JobTxn JobTxn; + + /** + * BlockJob: +@@ -84,8 +83,6 @@ typedef struct BlockJob { + + /** BlockDriverStates that are involved in this block job */ + GSList *nodes; +- +- JobTxn *txn; + } BlockJob; + + /** +@@ -153,22 +150,6 @@ void block_job_set_speed(BlockJob *job, int64_t speed, Error **errp); + void block_job_cancel(BlockJob *job, bool force); + + /** +- * block_job_finalize: +- * @job: The job to fully commit and finish. +- * @errp: Error object. +- * +- * For jobs that have finished their work and are pending +- * awaiting explicit acknowledgement to commit their work, +- * This will commit that work. +- * +- * FIXME: Make the below statement universally true: +- * For jobs that support the manual workflow mode, all graph +- * changes that occur as a result will occur after this command +- * and before a successful reply. +- */ +-void block_job_finalize(BlockJob *job, Error **errp); +- +-/** + * block_job_dismiss: + * @job: The job to be dismissed. + * @errp: Error object. +@@ -260,41 +241,6 @@ int block_job_complete_sync(BlockJob *job, Error **errp); + void block_job_iostatus_reset(BlockJob *job); + + /** +- * block_job_txn_new: +- * +- * Allocate and return a new block job transaction. Jobs can be added to the +- * transaction using block_job_txn_add_job(). +- * +- * The transaction is automatically freed when the last job completes or is +- * cancelled. +- * +- * All jobs in the transaction either complete successfully or fail/cancel as a +- * group. Jobs wait for each other before completing. Cancelling one job +- * cancels all jobs in the transaction. +- */ +-JobTxn *block_job_txn_new(void); +- +-/** +- * block_job_txn_unref: +- * +- * Release a reference that was previously acquired with block_job_txn_add_job +- * or block_job_txn_new. If it's the last reference to the object, it will be +- * freed. +- */ +-void block_job_txn_unref(JobTxn *txn); +- +-/** +- * block_job_txn_add_job: +- * @txn: The transaction (may be NULL) +- * @job: Job to add to the transaction +- * +- * Add @job to the transaction. The @job must not already be in a transaction. +- * The caller must call either block_job_txn_unref() or block_job_completed() +- * to release the reference that is automatically grabbed here. +- */ +-void block_job_txn_add_job(JobTxn *txn, BlockJob *job); +- +-/** + * block_job_is_internal: + * @job: The job to determine if it is user-visible or not. + * +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index ce66a9b..29a2802 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -38,16 +38,6 @@ struct BlockJobDriver { + /** Generic JobDriver callbacks and settings */ + JobDriver job_driver; + +- /** +- * If the callback is not NULL, prepare will be invoked when all the jobs +- * belonging to the same transaction complete; or upon this job's completion +- * if it is not in a transaction. +- * +- * This callback will not be invoked if the job has already failed. +- * If it fails, abort and then clean will be called. +- */ +- int (*prepare)(BlockJob *job); +- + /* + * If the callback is not NULL, it will be invoked before the job is + * resumed in a new AioContext. This is the place to move any resources +diff --git a/include/qemu/job.h b/include/qemu/job.h +index d4aa7fa..39d74ab 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -32,6 +32,8 @@ + #include "block/aio.h" + + typedef struct JobDriver JobDriver; ++typedef struct JobTxn JobTxn; ++ + + /** + * Long-running operation. +@@ -133,6 +135,9 @@ typedef struct Job { + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; + ++ /** Transaction this job is part of */ ++ JobTxn *txn; ++ + /** Element of the list of jobs in a job transaction */ + QLIST_ENTRY(Job) txn_list; + } Job; +@@ -184,6 +189,16 @@ struct JobDriver { + void (*drain)(Job *job); + + /** ++ * If the callback is not NULL, prepare will be invoked when all the jobs ++ * belonging to the same transaction complete; or upon this job's completion ++ * if it is not in a transaction. ++ * ++ * This callback will not be invoked if the job has already failed. ++ * If it fails, abort and then clean will be called. ++ */ ++ int (*prepare)(Job *job); ++ ++ /** + * If the callback is not NULL, it will be invoked when all the jobs + * belonging to the same transaction complete; or upon this job's + * completion if it is not in a transaction. Skipped if NULL. +@@ -227,20 +242,52 @@ typedef enum JobCreateFlags { + JOB_MANUAL_DISMISS = 0x04, + } JobCreateFlags; + ++/** ++ * Allocate and return a new job transaction. Jobs can be added to the ++ * transaction using job_txn_add_job(). ++ * ++ * The transaction is automatically freed when the last job completes or is ++ * cancelled. ++ * ++ * All jobs in the transaction either complete successfully or fail/cancel as a ++ * group. Jobs wait for each other before completing. Cancelling one job ++ * cancels all jobs in the transaction. ++ */ ++JobTxn *job_txn_new(void); ++ ++/** ++ * Release a reference that was previously acquired with job_txn_add_job or ++ * job_txn_new. If it's the last reference to the object, it will be freed. ++ */ ++void job_txn_unref(JobTxn *txn); ++ ++/** ++ * @txn: The transaction (may be NULL) ++ * @job: Job to add to the transaction ++ * ++ * Add @job to the transaction. The @job must not already be in a transaction. ++ * The caller must call either job_txn_unref() or block_job_completed() to ++ * release the reference that is automatically grabbed here. ++ * ++ * If @txn is NULL, the function does nothing. ++ */ ++void job_txn_add_job(JobTxn *txn, Job *job); + + /** + * Create a new long-running job and return it. + * + * @job_id: The id of the newly-created job, or %NULL for internal jobs + * @driver: The class object for the newly-created job. ++ * @txn: The transaction this job belongs to, if any. %NULL otherwise. + * @ctx: The AioContext to run the job coroutine in. + * @flags: Creation flags for the job. See @JobCreateFlags. + * @cb: Completion function for the job. + * @opaque: Opaque pointer value passed to @cb. + * @errp: Error object. + */ +-void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, +- int flags, BlockCompletionFunc *cb, void *opaque, Error **errp); ++void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, ++ AioContext *ctx, int flags, BlockCompletionFunc *cb, ++ void *opaque, Error **errp); + + /** + * Add a reference to Job refcnt, it will be decreased with job_unref, and then +@@ -260,9 +307,6 @@ void job_event_cancelled(Job *job); + /** To be called when a successfully completed job is finalised. */ + void job_event_completed(Job *job); + +-/** To be called when the job transitions to PENDING */ +-void job_event_pending(Job *job); +- + /** + * Conditionally enter the job coroutine if the job is ready to run, not + * already busy and fn() returns true. fn() is called while under the job_lock +@@ -375,6 +419,16 @@ void job_early_fail(Job *job); + /** Asynchronously complete the specified @job. */ + void job_complete(Job *job, Error **errp);; + ++/** ++ * For a @job that has finished its work and is pending awaiting explicit ++ * acknowledgement to commit its work, this will commit that work. ++ * ++ * FIXME: Make the below statement universally true: ++ * For jobs that support the manual workflow mode, all graph changes that occur ++ * as a result will occur after this command and before a successful reply. ++ */ ++void job_finalize(Job *job, Error **errp); ++ + typedef void JobDeferToMainLoopFn(Job *job, void *opaque); + + /** +@@ -407,10 +461,9 @@ void coroutine_fn job_do_yield(Job *job, uint64_t ns); + bool job_should_pause(Job *job); + bool job_started(Job *job); + void job_do_dismiss(Job *job); +-int job_finalize_single(Job *job); + void job_update_rc(Job *job); +- +-typedef struct BlockJob BlockJob; +-void block_job_txn_del_job(BlockJob *job); ++void job_cancel_async(Job *job, bool force); ++void job_completed_txn_abort(Job *job); ++void job_completed_txn_success(Job *job); + + #endif +diff --git a/job.c b/job.c +index aa74b4c..4f6fd73 100644 +--- a/job.c ++++ b/job.c +@@ -60,6 +60,19 @@ bool JobVerbTable[JOB_VERB__MAX][JOB_STATUS__MAX] = { + [JOB_VERB_DISMISS] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0}, + }; + ++/* Transactional group of jobs */ ++struct JobTxn { ++ ++ /* Is this txn being cancelled? */ ++ bool aborting; ++ ++ /* List of jobs */ ++ QLIST_HEAD(, Job) jobs; ++ ++ /* Reference count */ ++ int refcnt; ++}; ++ + /* Right now, this mutex is only needed to synchronize accesses to job->busy + * and job->sleep_timer, such as concurrent calls to job_do_yield and + * job_enter. */ +@@ -80,6 +93,71 @@ static void __attribute__((__constructor__)) job_init(void) + qemu_mutex_init(&job_mutex); + } + ++JobTxn *job_txn_new(void) ++{ ++ JobTxn *txn = g_new0(JobTxn, 1); ++ QLIST_INIT(&txn->jobs); ++ txn->refcnt = 1; ++ return txn; ++} ++ ++static void job_txn_ref(JobTxn *txn) ++{ ++ txn->refcnt++; ++} ++ ++void job_txn_unref(JobTxn *txn) ++{ ++ if (txn && --txn->refcnt == 0) { ++ g_free(txn); ++ } ++} ++ ++void job_txn_add_job(JobTxn *txn, Job *job) ++{ ++ if (!txn) { ++ return; ++ } ++ ++ assert(!job->txn); ++ job->txn = txn; ++ ++ QLIST_INSERT_HEAD(&txn->jobs, job, txn_list); ++ job_txn_ref(txn); ++} ++ ++static void job_txn_del_job(Job *job) ++{ ++ if (job->txn) { ++ QLIST_REMOVE(job, txn_list); ++ job_txn_unref(job->txn); ++ job->txn = NULL; ++ } ++} ++ ++static int job_txn_apply(JobTxn *txn, int fn(Job *), bool lock) ++{ ++ AioContext *ctx; ++ Job *job, *next; ++ int rc = 0; ++ ++ QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { ++ if (lock) { ++ ctx = job->aio_context; ++ aio_context_acquire(ctx); ++ } ++ rc = fn(job); ++ if (lock) { ++ aio_context_release(ctx); ++ } ++ if (rc) { ++ break; ++ } ++ } ++ return rc; ++} ++ ++ + /* TODO Make static once the whole state machine is in job.c */ + void job_state_transition(Job *job, JobStatus s1) + { +@@ -181,8 +259,9 @@ static void job_sleep_timer_cb(void *opaque) + job_enter(job); + } + +-void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, +- int flags, BlockCompletionFunc *cb, void *opaque, Error **errp) ++void *job_create(const char *job_id, const JobDriver *driver, JobTxn *txn, ++ AioContext *ctx, int flags, BlockCompletionFunc *cb, ++ void *opaque, Error **errp) + { + Job *job; + +@@ -228,6 +307,16 @@ void *job_create(const char *job_id, const JobDriver *driver, AioContext *ctx, + + QLIST_INSERT_HEAD(&jobs, job, job_list); + ++ /* Single jobs are modeled as single-job transactions for sake of ++ * consolidating the job management logic */ ++ if (!txn) { ++ txn = job_txn_new(); ++ job_txn_add_job(txn, job); ++ job_txn_unref(txn); ++ } else { ++ job_txn_add_job(txn, job); ++ } ++ + return job; + } + +@@ -241,6 +330,7 @@ void job_unref(Job *job) + if (--job->refcnt == 0) { + assert(job->status == JOB_STATUS_NULL); + assert(!timer_pending(&job->sleep_timer)); ++ assert(!job->txn); + + if (job->driver->free) { + job->driver->free(job); +@@ -263,7 +353,7 @@ void job_event_completed(Job *job) + notifier_list_notify(&job->on_finalize_completed, job); + } + +-void job_event_pending(Job *job) ++static void job_event_pending(Job *job) + { + notifier_list_notify(&job->on_pending, job); + } +@@ -469,8 +559,7 @@ void job_do_dismiss(Job *job) + job->paused = false; + job->deferred_to_main_loop = true; + +- /* TODO Don't assume it's a BlockJob */ +- block_job_txn_del_job((BlockJob*) job); ++ job_txn_del_job(job); + + job_state_transition(job, JOB_STATUS_NULL); + job_unref(job); +@@ -523,7 +612,7 @@ static void job_clean(Job *job) + } + } + +-int job_finalize_single(Job *job) ++static int job_finalize_single(Job *job) + { + assert(job_is_completed(job)); + +@@ -550,12 +639,141 @@ int job_finalize_single(Job *job) + } + } + +- /* TODO Don't assume it's a BlockJob */ +- block_job_txn_del_job((BlockJob*) job); ++ job_txn_del_job(job); + job_conclude(job); + return 0; + } + ++void job_cancel_async(Job *job, bool force) ++{ ++ if (job->user_paused) { ++ /* Do not call job_enter here, the caller will handle it. */ ++ job->user_paused = false; ++ if (job->driver->user_resume) { ++ job->driver->user_resume(job); ++ } ++ assert(job->pause_count > 0); ++ job->pause_count--; ++ } ++ job->cancelled = true; ++ /* To prevent 'force == false' overriding a previous 'force == true' */ ++ job->force_cancel |= force; ++} ++ ++void job_completed_txn_abort(Job *job) ++{ ++ AioContext *ctx; ++ JobTxn *txn = job->txn; ++ Job *other_job; ++ ++ if (txn->aborting) { ++ /* ++ * We are cancelled by another job, which will handle everything. ++ */ ++ return; ++ } ++ txn->aborting = true; ++ job_txn_ref(txn); ++ ++ /* We are the first failed job. Cancel other jobs. */ ++ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { ++ ctx = other_job->aio_context; ++ aio_context_acquire(ctx); ++ } ++ ++ /* Other jobs are effectively cancelled by us, set the status for ++ * them; this job, however, may or may not be cancelled, depending ++ * on the caller, so leave it. */ ++ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { ++ if (other_job != job) { ++ job_cancel_async(other_job, false); ++ } ++ } ++ while (!QLIST_EMPTY(&txn->jobs)) { ++ other_job = QLIST_FIRST(&txn->jobs); ++ ctx = other_job->aio_context; ++ if (!job_is_completed(other_job)) { ++ assert(job_is_cancelled(other_job)); ++ job_finish_sync(other_job, NULL, NULL); ++ } ++ job_finalize_single(other_job); ++ aio_context_release(ctx); ++ } ++ ++ job_txn_unref(txn); ++} ++ ++static int job_prepare(Job *job) ++{ ++ if (job->ret == 0 && job->driver->prepare) { ++ job->ret = job->driver->prepare(job); ++ } ++ return job->ret; ++} ++ ++static int job_needs_finalize(Job *job) ++{ ++ return !job->auto_finalize; ++} ++ ++static void job_do_finalize(Job *job) ++{ ++ int rc; ++ assert(job && job->txn); ++ ++ /* prepare the transaction to complete */ ++ rc = job_txn_apply(job->txn, job_prepare, true); ++ if (rc) { ++ job_completed_txn_abort(job); ++ } else { ++ job_txn_apply(job->txn, job_finalize_single, true); ++ } ++} ++ ++void job_finalize(Job *job, Error **errp) ++{ ++ assert(job && job->id); ++ if (job_apply_verb(job, JOB_VERB_FINALIZE, errp)) { ++ return; ++ } ++ job_do_finalize(job); ++} ++ ++static int job_transition_to_pending(Job *job) ++{ ++ job_state_transition(job, JOB_STATUS_PENDING); ++ if (!job->auto_finalize) { ++ job_event_pending(job); ++ } ++ return 0; ++} ++ ++void job_completed_txn_success(Job *job) ++{ ++ JobTxn *txn = job->txn; ++ Job *other_job; ++ ++ job_state_transition(job, JOB_STATUS_WAITING); ++ ++ /* ++ * Successful completion, see if there are other running jobs in this ++ * txn. ++ */ ++ QLIST_FOREACH(other_job, &txn->jobs, txn_list) { ++ if (!job_is_completed(other_job)) { ++ return; ++ } ++ assert(other_job->ret == 0); ++ } ++ ++ job_txn_apply(txn, job_transition_to_pending, false); ++ ++ /* If no jobs need manual finalization, automatically do so */ ++ if (job_txn_apply(txn, job_needs_finalize, false) == 0) { ++ job_do_finalize(job); ++ } ++} ++ + void job_complete(Job *job, Error **errp) + { + /* Should not be reachable via external interface for internal jobs */ +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index ec5d592..6ee31d5 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -125,7 +125,7 @@ static void test_single_job(int expected) + JobTxn *txn; + int result = -EINPROGRESS; + +- txn = block_job_txn_new(); ++ txn = job_txn_new(); + job = test_block_job_start(1, true, expected, &result, txn); + job_start(&job->job); + +@@ -138,7 +138,7 @@ static void test_single_job(int expected) + } + g_assert_cmpint(result, ==, expected); + +- block_job_txn_unref(txn); ++ job_txn_unref(txn); + } + + static void test_single_job_success(void) +@@ -164,7 +164,7 @@ static void test_pair_jobs(int expected1, int expected2) + int result1 = -EINPROGRESS; + int result2 = -EINPROGRESS; + +- txn = block_job_txn_new(); ++ txn = job_txn_new(); + job1 = test_block_job_start(1, true, expected1, &result1, txn); + job2 = test_block_job_start(2, true, expected2, &result2, txn); + job_start(&job1->job); +@@ -173,7 +173,7 @@ static void test_pair_jobs(int expected1, int expected2) + /* Release our reference now to trigger as many nice + * use-after-free bugs as possible. + */ +- block_job_txn_unref(txn); ++ job_txn_unref(txn); + + if (expected1 == -ECANCELED) { + block_job_cancel(job1, false); +@@ -226,7 +226,7 @@ static void test_pair_jobs_fail_cancel_race(void) + int result1 = -EINPROGRESS; + int result2 = -EINPROGRESS; + +- txn = block_job_txn_new(); ++ txn = job_txn_new(); + job1 = test_block_job_start(1, true, -ECANCELED, &result1, txn); + job2 = test_block_job_start(2, false, 0, &result2, txn); + job_start(&job1->job); +@@ -247,7 +247,7 @@ static void test_pair_jobs_fail_cancel_race(void) + g_assert_cmpint(result1, ==, -ECANCELED); + g_assert_cmpint(result2, ==, -ECANCELED); + +- block_job_txn_unref(txn); ++ job_txn_unref(txn); + } + + int main(int argc, char **argv) +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index e44c608..1e052c2 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -364,7 +364,7 @@ static void test_cancel_concluded(void) + } + assert(job->job.status == JOB_STATUS_PENDING); + +- block_job_finalize(job, &error_abort); ++ job_finalize(&job->job, &error_abort); + assert(job->job.status == JOB_STATUS_CONCLUDED); + + cancel_common(s); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Rename-BlockJobType-into-JobType.patch b/SOURCES/kvm-job-Rename-BlockJobType-into-JobType.patch new file mode 100644 index 0000000..ed71096 --- /dev/null +++ b/SOURCES/kvm-job-Rename-BlockJobType-into-JobType.patch @@ -0,0 +1,204 @@ +From a8fa5721ae332c4f16f96e72898a938ed178843b Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:00 +0200 +Subject: [PATCH 092/268] job: Rename BlockJobType into JobType + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-18-kwolf@redhat.com> +Patchwork-id: 81112 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 17/73] job: Rename BlockJobType into JobType +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +QAPI types aren't externally visible, so we can rename them without +causing problems. Before we add a job type to Job, rename the enum +so it can be used for more than just block jobs. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit 8e4c87000fc515f8f65f7c8f18afb1e9270b11d6) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + block/commit.c | 2 +- + block/mirror.c | 4 ++-- + block/stream.c | 2 +- + blockjob.c | 6 +++--- + include/block/blockjob_int.h | 2 +- + qapi/block-core.json | 14 +++++++------- + 7 files changed, 16 insertions(+), 16 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 9e672bb..c49ea92 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -526,7 +526,7 @@ static const BlockJobDriver backup_job_driver = { + .job_driver = { + .instance_size = sizeof(BackupBlockJob), + }, +- .job_type = BLOCK_JOB_TYPE_BACKUP, ++ .job_type = JOB_TYPE_BACKUP, + .start = backup_run, + .commit = backup_commit, + .abort = backup_abort, +diff --git a/block/commit.c b/block/commit.c +index 18cbb2f..afa2b2b 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -218,7 +218,7 @@ static const BlockJobDriver commit_job_driver = { + .job_driver = { + .instance_size = sizeof(CommitBlockJob), + }, +- .job_type = BLOCK_JOB_TYPE_COMMIT, ++ .job_type = JOB_TYPE_COMMIT, + .start = commit_run, + }; + +diff --git a/block/mirror.c b/block/mirror.c +index aa1d6b7..ed72656 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -989,7 +989,7 @@ static const BlockJobDriver mirror_job_driver = { + .job_driver = { + .instance_size = sizeof(MirrorBlockJob), + }, +- .job_type = BLOCK_JOB_TYPE_MIRROR, ++ .job_type = JOB_TYPE_MIRROR, + .start = mirror_run, + .complete = mirror_complete, + .pause = mirror_pause, +@@ -1001,7 +1001,7 @@ static const BlockJobDriver commit_active_job_driver = { + .job_driver = { + .instance_size = sizeof(MirrorBlockJob), + }, +- .job_type = BLOCK_JOB_TYPE_COMMIT, ++ .job_type = JOB_TYPE_COMMIT, + .start = mirror_run, + .complete = mirror_complete, + .pause = mirror_pause, +diff --git a/block/stream.c b/block/stream.c +index f88fc75..048bceb 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -212,7 +212,7 @@ static const BlockJobDriver stream_job_driver = { + .job_driver = { + .instance_size = sizeof(StreamBlockJob), + }, +- .job_type = BLOCK_JOB_TYPE_STREAM, ++ .job_type = JOB_TYPE_STREAM, + .start = stream_run, + }; + +diff --git a/blockjob.c b/blockjob.c +index 1464856..2a38447 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -310,7 +310,7 @@ static char *child_job_get_parent_desc(BdrvChild *c) + { + BlockJob *job = c->opaque; + return g_strdup_printf("%s job '%s'", +- BlockJobType_str(job->driver->job_type), ++ JobType_str(job->driver->job_type), + job->job.id); + } + +@@ -847,7 +847,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp) + return NULL; + } + info = g_new0(BlockJobInfo, 1); +- info->type = g_strdup(BlockJobType_str(job->driver->job_type)); ++ info->type = g_strdup(JobType_str(job->driver->job_type)); + info->device = g_strdup(job->job.id); + info->len = job->len; + info->busy = atomic_read(&job->busy); +@@ -980,7 +980,7 @@ void *block_job_create(const char *job_id, const BlockJobDriver *driver, + block_job_sleep_timer_cb, job); + + error_setg(&job->blocker, "block device is in use by block job: %s", +- BlockJobType_str(driver->job_type)); ++ JobType_str(driver->job_type)); + block_job_add_bdrv(job, "main node", bs, 0, BLK_PERM_ALL, &error_abort); + bs->job = job; + +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index e8eca44..8e7e0a2 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -39,7 +39,7 @@ struct BlockJobDriver { + JobDriver job_driver; + + /** String describing the operation, part of query-block-jobs QMP API */ +- BlockJobType job_type; ++ JobType job_type; + + /** Mandatory: Entrypoint for the Coroutine. */ + CoroutineEntry *start; +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 2b4566f..cab0697 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1050,9 +1050,9 @@ + 'data': ['top', 'full', 'none', 'incremental'] } + + ## +-# @BlockJobType: ++# @JobType: + # +-# Type of a block job. ++# Type of a background job. + # + # @commit: block commit job type, see "block-commit" + # +@@ -1064,7 +1064,7 @@ + # + # Since: 1.7 + ## +-{ 'enum': 'BlockJobType', ++{ 'enum': 'JobType', + 'data': ['commit', 'stream', 'mirror', 'backup'] } + + ## +@@ -4513,7 +4513,7 @@ + # + ## + { 'event': 'BLOCK_JOB_COMPLETED', +- 'data': { 'type' : 'BlockJobType', ++ 'data': { 'type' : 'JobType', + 'device': 'str', + 'len' : 'int', + 'offset': 'int', +@@ -4549,7 +4549,7 @@ + # + ## + { 'event': 'BLOCK_JOB_CANCELLED', +- 'data': { 'type' : 'BlockJobType', ++ 'data': { 'type' : 'JobType', + 'device': 'str', + 'len' : 'int', + 'offset': 'int', +@@ -4614,7 +4614,7 @@ + # + ## + { 'event': 'BLOCK_JOB_READY', +- 'data': { 'type' : 'BlockJobType', ++ 'data': { 'type' : 'JobType', + 'device': 'str', + 'len' : 'int', + 'offset': 'int', +@@ -4641,7 +4641,7 @@ + # + ## + { 'event': 'BLOCK_JOB_PENDING', +- 'data': { 'type' : 'BlockJobType', ++ 'data': { 'type' : 'JobType', + 'id' : 'str' } } + + ## +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Replace-BlockJob.completed-with-job_is_completed.patch b/SOURCES/kvm-job-Replace-BlockJob.completed-with-job_is_completed.patch new file mode 100644 index 0000000..3662cf2 --- /dev/null +++ b/SOURCES/kvm-job-Replace-BlockJob.completed-with-job_is_completed.patch @@ -0,0 +1,186 @@ +From 1b81ccdb3a4726180e54a0a427d6728c39d43a26 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:12 +0200 +Subject: [PATCH 104/268] job: Replace BlockJob.completed with + job_is_completed() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-30-kwolf@redhat.com> +Patchwork-id: 81094 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 29/73] job: Replace BlockJob.completed with job_is_completed() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Since we introduced an explicit status to block job, BlockJob.completed +is redundant because it can be derived from the status. Remove the field +from BlockJob and add a function to derive it from the status at the Job +level. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: John Snow +(cherry picked from commit dbe5e6c1f73b41282624b78a2375a5c3ee59e905) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + blockjob.c | 16 +++++++--------- + include/block/blockjob.h | 3 --- + include/qemu/job.h | 3 +++ + job.c | 22 ++++++++++++++++++++++ + qemu-img.c | 4 ++-- + 5 files changed, 34 insertions(+), 14 deletions(-) + +diff --git a/blockjob.c b/blockjob.c +index 6334a54..a1d1f48 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -193,7 +193,7 @@ static void block_job_detach_aio_context(void *opaque) + + job_pause(&job->job); + +- while (!job->job.paused && !job->completed) { ++ while (!job->job.paused && !job_is_completed(&job->job)) { + block_job_drain(job); + } + +@@ -269,7 +269,6 @@ const BlockJobDriver *block_job_driver(BlockJob *job) + static void block_job_decommission(BlockJob *job) + { + assert(job); +- job->completed = true; + job->job.busy = false; + job->job.paused = false; + job->job.deferred_to_main_loop = true; +@@ -334,7 +333,7 @@ static void block_job_clean(BlockJob *job) + + static int block_job_finalize_single(BlockJob *job) + { +- assert(job->completed); ++ assert(job_is_completed(&job->job)); + + /* Ensure abort is called for late-transactional failures */ + block_job_update_rc(job); +@@ -427,10 +426,10 @@ static int block_job_finish_sync(BlockJob *job, + /* block_job_drain calls block_job_enter, and it should be enough to + * induce progress until the job completes or moves to the main thread. + */ +- while (!job->job.deferred_to_main_loop && !job->completed) { ++ while (!job->job.deferred_to_main_loop && !job_is_completed(&job->job)) { + block_job_drain(job); + } +- while (!job->completed) { ++ while (!job_is_completed(&job->job)) { + aio_poll(qemu_get_aio_context(), true); + } + ret = (job_is_cancelled(&job->job) && job->ret == 0) +@@ -471,7 +470,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + while (!QLIST_EMPTY(&txn->jobs)) { + other_job = QLIST_FIRST(&txn->jobs); + ctx = blk_get_aio_context(other_job->blk); +- if (!other_job->completed) { ++ if (!job_is_completed(&other_job->job)) { + assert(job_is_cancelled(&other_job->job)); + block_job_finish_sync(other_job, NULL, NULL); + } +@@ -513,7 +512,7 @@ static void block_job_completed_txn_success(BlockJob *job) + * txn. + */ + QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- if (!other_job->completed) { ++ if (!job_is_completed(&other_job->job)) { + return; + } + assert(other_job->ret == 0); +@@ -847,9 +846,8 @@ void block_job_early_fail(BlockJob *job) + + void block_job_completed(BlockJob *job, int ret) + { +- assert(job && job->txn && !job->completed); ++ assert(job && job->txn && !job_is_completed(&job->job)); + assert(blk_bs(job->blk)->job == job); +- job->completed = true; + job->ret = ret; + block_job_update_rc(job); + trace_block_job_completed(job, ret, job->ret); +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 556a8f6..3e94e18 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -88,9 +88,6 @@ typedef struct BlockJob { + /** The opaque value that is passed to the completion function. */ + void *opaque; + +- /** True when job has reported completion by calling block_job_completed. */ +- bool completed; +- + /** ret code passed to block_job_completed. */ + int ret; + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index bc63985..858f3be 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -214,6 +214,9 @@ const char *job_type_str(const Job *job); + /** Returns whether the job is scheduled for cancellation. */ + bool job_is_cancelled(Job *job); + ++/** Returns whether the job is in a completed state. */ ++bool job_is_completed(Job *job); ++ + /** + * Request @job to pause at the next pause point. Must be paired with + * job_resume(). If the job is supposed to be resumed by user action, call +diff --git a/job.c b/job.c +index fd10b1d..aaacfcc 100644 +--- a/job.c ++++ b/job.c +@@ -121,6 +121,28 @@ bool job_is_cancelled(Job *job) + return job->cancelled; + } + ++bool job_is_completed(Job *job) ++{ ++ switch (job->status) { ++ case JOB_STATUS_UNDEFINED: ++ case JOB_STATUS_CREATED: ++ case JOB_STATUS_RUNNING: ++ case JOB_STATUS_PAUSED: ++ case JOB_STATUS_READY: ++ case JOB_STATUS_STANDBY: ++ return false; ++ case JOB_STATUS_WAITING: ++ case JOB_STATUS_PENDING: ++ case JOB_STATUS_ABORTING: ++ case JOB_STATUS_CONCLUDED: ++ case JOB_STATUS_NULL: ++ return true; ++ default: ++ g_assert_not_reached(); ++ } ++ return false; ++} ++ + bool job_started(Job *job) + { + return job->co; +diff --git a/qemu-img.c b/qemu-img.c +index f745919..f5bb0ef 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -878,9 +878,9 @@ static void run_block_job(BlockJob *job, Error **errp) + aio_poll(aio_context, true); + qemu_progress_print(job->len ? + ((float)job->offset / job->len * 100.f) : 0.0f, 0); +- } while (!job->ready && !job->completed); ++ } while (!job->ready && !job_is_completed(&job->job)); + +- if (!job->completed) { ++ if (!job_is_completed(&job->job)) { + ret = block_job_complete_sync(job, errp); + } else { + ret = job->ret; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Switch-transactions-to-JobTxn.patch b/SOURCES/kvm-job-Switch-transactions-to-JobTxn.patch new file mode 100644 index 0000000..4dd6be8 --- /dev/null +++ b/SOURCES/kvm-job-Switch-transactions-to-JobTxn.patch @@ -0,0 +1,423 @@ +From 7586908f543ae6e8981ed4bc17b60adec3b7456d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:21 +0200 +Subject: [PATCH 113/268] job: Switch transactions to JobTxn + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-39-kwolf@redhat.com> +Patchwork-id: 81086 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 38/73] job: Switch transactions to JobTxn +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This doesn't actually move any transaction code to Job yet, but it +renames the type for transactions from BlockJobTxn to JobTxn and makes +them contain Jobs rather than BlockJobs + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 62c9e4162a7bc26a1389e50d17d3b2637028bbc3) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/backup.c | 2 +- + blockdev.c | 14 +++++------ + blockjob.c | 60 +++++++++++++++++++++++--------------------- + include/block/block_int.h | 2 +- + include/block/blockjob.h | 11 ++++---- + include/block/blockjob_int.h | 2 +- + include/qemu/job.h | 3 +++ + tests/test-blockjob-txn.c | 8 +++--- + 8 files changed, 54 insertions(+), 48 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index ca7d990..6172f90 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -547,7 +547,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + BlockdevOnError on_target_error, + int creation_flags, + BlockCompletionFunc *cb, void *opaque, +- BlockJobTxn *txn, Error **errp) ++ JobTxn *txn, Error **errp) + { + int64_t len; + BlockDriverInfo bdi; +diff --git a/blockdev.c b/blockdev.c +index dd9a080..6efdb30 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -1447,7 +1447,7 @@ typedef struct BlkActionOps { + struct BlkActionState { + TransactionAction *action; + const BlkActionOps *ops; +- BlockJobTxn *block_job_txn; ++ JobTxn *block_job_txn; + TransactionProperties *txn_props; + QSIMPLEQ_ENTRY(BlkActionState) entry; + }; +@@ -1865,7 +1865,7 @@ typedef struct DriveBackupState { + BlockJob *job; + } DriveBackupState; + +-static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, ++static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn, + Error **errp); + + static void drive_backup_prepare(BlkActionState *common, Error **errp) +@@ -1955,7 +1955,7 @@ typedef struct BlockdevBackupState { + BlockJob *job; + } BlockdevBackupState; + +-static BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, ++static BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + Error **errp); + + static void blockdev_backup_prepare(BlkActionState *common, Error **errp) +@@ -2244,7 +2244,7 @@ void qmp_transaction(TransactionActionList *dev_list, + Error **errp) + { + TransactionActionList *dev_entry = dev_list; +- BlockJobTxn *block_job_txn = NULL; ++ JobTxn *block_job_txn = NULL; + BlkActionState *state, *next; + Error *local_err = NULL; + +@@ -2252,7 +2252,7 @@ void qmp_transaction(TransactionActionList *dev_list, + QSIMPLEQ_INIT(&snap_bdrv_states); + + /* Does this transaction get canceled as a group on failure? +- * If not, we don't really need to make a BlockJobTxn. ++ * If not, we don't really need to make a JobTxn. + */ + props = get_transaction_properties(props); + if (props->completion_mode != ACTION_COMPLETION_MODE_INDIVIDUAL) { +@@ -3265,7 +3265,7 @@ out: + aio_context_release(aio_context); + } + +-static BlockJob *do_drive_backup(DriveBackup *backup, BlockJobTxn *txn, ++static BlockJob *do_drive_backup(DriveBackup *backup, JobTxn *txn, + Error **errp) + { + BlockDriverState *bs; +@@ -3435,7 +3435,7 @@ BlockDeviceInfoList *qmp_query_named_block_nodes(Error **errp) + return bdrv_named_nodes_list(errp); + } + +-BlockJob *do_blockdev_backup(BlockdevBackup *backup, BlockJobTxn *txn, ++BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn, + Error **errp) + { + BlockDriverState *bs; +diff --git a/blockjob.c b/blockjob.c +index 1ed3e9c..bd35c4f 100644 +--- a/blockjob.c ++++ b/blockjob.c +@@ -37,13 +37,13 @@ + #include "qemu/timer.h" + + /* Transactional group of block jobs */ +-struct BlockJobTxn { ++struct JobTxn { + + /* Is this txn being cancelled? */ + bool aborting; + + /* List of jobs */ +- QLIST_HEAD(, BlockJob) jobs; ++ QLIST_HEAD(, Job) jobs; + + /* Reference count */ + int refcnt; +@@ -94,27 +94,27 @@ BlockJob *block_job_get(const char *id) + } + } + +-BlockJobTxn *block_job_txn_new(void) ++JobTxn *block_job_txn_new(void) + { +- BlockJobTxn *txn = g_new0(BlockJobTxn, 1); ++ JobTxn *txn = g_new0(JobTxn, 1); + QLIST_INIT(&txn->jobs); + txn->refcnt = 1; + return txn; + } + +-static void block_job_txn_ref(BlockJobTxn *txn) ++static void block_job_txn_ref(JobTxn *txn) + { + txn->refcnt++; + } + +-void block_job_txn_unref(BlockJobTxn *txn) ++void block_job_txn_unref(JobTxn *txn) + { + if (txn && --txn->refcnt == 0) { + g_free(txn); + } + } + +-void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job) ++void block_job_txn_add_job(JobTxn *txn, BlockJob *job) + { + if (!txn) { + return; +@@ -123,14 +123,14 @@ void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job) + assert(!job->txn); + job->txn = txn; + +- QLIST_INSERT_HEAD(&txn->jobs, job, txn_list); ++ QLIST_INSERT_HEAD(&txn->jobs, &job->job, txn_list); + block_job_txn_ref(txn); + } + + void block_job_txn_del_job(BlockJob *job) + { + if (job->txn) { +- QLIST_REMOVE(job, txn_list); ++ QLIST_REMOVE(&job->job, txn_list); + block_job_txn_unref(job->txn); + job->txn = NULL; + } +@@ -285,18 +285,22 @@ static void job_cancel_async(Job *job, bool force) + job->force_cancel |= force; + } + +-static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *), bool lock) ++static int block_job_txn_apply(JobTxn *txn, int fn(BlockJob *), bool lock) + { + AioContext *ctx; +- BlockJob *job, *next; ++ Job *job, *next; ++ BlockJob *bjob; + int rc = 0; + + QLIST_FOREACH_SAFE(job, &txn->jobs, txn_list, next) { ++ assert(is_block_job(job)); ++ bjob = container_of(job, BlockJob, job); ++ + if (lock) { +- ctx = blk_get_aio_context(job->blk); ++ ctx = job->aio_context; + aio_context_acquire(ctx); + } +- rc = fn(job); ++ rc = fn(bjob); + if (lock) { + aio_context_release(ctx); + } +@@ -310,8 +314,8 @@ static int block_job_txn_apply(BlockJobTxn *txn, int fn(BlockJob *), bool lock) + static void block_job_completed_txn_abort(BlockJob *job) + { + AioContext *ctx; +- BlockJobTxn *txn = job->txn; +- BlockJob *other_job; ++ JobTxn *txn = job->txn; ++ Job *other_job; + + if (txn->aborting) { + /* +@@ -324,7 +328,7 @@ static void block_job_completed_txn_abort(BlockJob *job) + + /* We are the first failed job. Cancel other jobs. */ + QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- ctx = blk_get_aio_context(other_job->blk); ++ ctx = other_job->aio_context; + aio_context_acquire(ctx); + } + +@@ -332,18 +336,18 @@ static void block_job_completed_txn_abort(BlockJob *job) + * them; this job, however, may or may not be cancelled, depending + * on the caller, so leave it. */ + QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- if (other_job != job) { +- job_cancel_async(&other_job->job, false); ++ if (other_job != &job->job) { ++ job_cancel_async(other_job, false); + } + } + while (!QLIST_EMPTY(&txn->jobs)) { + other_job = QLIST_FIRST(&txn->jobs); +- ctx = blk_get_aio_context(other_job->blk); +- if (!job_is_completed(&other_job->job)) { +- assert(job_is_cancelled(&other_job->job)); +- job_finish_sync(&other_job->job, NULL, NULL); ++ ctx = other_job->aio_context; ++ if (!job_is_completed(other_job)) { ++ assert(job_is_cancelled(other_job)); ++ job_finish_sync(other_job, NULL, NULL); + } +- job_finalize_single(&other_job->job); ++ job_finalize_single(other_job); + aio_context_release(ctx); + } + +@@ -385,8 +389,8 @@ static int block_job_transition_to_pending(BlockJob *job) + + static void block_job_completed_txn_success(BlockJob *job) + { +- BlockJobTxn *txn = job->txn; +- BlockJob *other_job; ++ JobTxn *txn = job->txn; ++ Job *other_job; + + job_state_transition(&job->job, JOB_STATUS_WAITING); + +@@ -395,10 +399,10 @@ static void block_job_completed_txn_success(BlockJob *job) + * txn. + */ + QLIST_FOREACH(other_job, &txn->jobs, txn_list) { +- if (!job_is_completed(&other_job->job)) { ++ if (!job_is_completed(other_job)) { + return; + } +- assert(other_job->job.ret == 0); ++ assert(other_job->ret == 0); + } + + block_job_txn_apply(txn, block_job_transition_to_pending, false); +@@ -628,7 +632,7 @@ static void block_job_event_pending(Notifier *n, void *opaque) + */ + + void *block_job_create(const char *job_id, const BlockJobDriver *driver, +- BlockJobTxn *txn, BlockDriverState *bs, uint64_t perm, ++ JobTxn *txn, BlockDriverState *bs, uint64_t perm, + uint64_t shared_perm, int64_t speed, int flags, + BlockCompletionFunc *cb, void *opaque, Error **errp) + { +diff --git a/include/block/block_int.h b/include/block/block_int.h +index d913ed1..ad2b852 100644 +--- a/include/block/block_int.h ++++ b/include/block/block_int.h +@@ -1018,7 +1018,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs, + BlockdevOnError on_target_error, + int creation_flags, + BlockCompletionFunc *cb, void *opaque, +- BlockJobTxn *txn, Error **errp); ++ JobTxn *txn, Error **errp); + + void hmp_drive_add_node(Monitor *mon, const char *optstr); + +diff --git a/include/block/blockjob.h b/include/block/blockjob.h +index 85ce18a..44df025 100644 +--- a/include/block/blockjob.h ++++ b/include/block/blockjob.h +@@ -33,7 +33,7 @@ + #define BLOCK_JOB_SLICE_TIME 100000000ULL /* ns */ + + typedef struct BlockJobDriver BlockJobDriver; +-typedef struct BlockJobTxn BlockJobTxn; ++typedef struct JobTxn JobTxn; + + /** + * BlockJob: +@@ -85,8 +85,7 @@ typedef struct BlockJob { + /** BlockDriverStates that are involved in this block job */ + GSList *nodes; + +- BlockJobTxn *txn; +- QLIST_ENTRY(BlockJob) txn_list; ++ JobTxn *txn; + } BlockJob; + + /** +@@ -273,7 +272,7 @@ void block_job_iostatus_reset(BlockJob *job); + * group. Jobs wait for each other before completing. Cancelling one job + * cancels all jobs in the transaction. + */ +-BlockJobTxn *block_job_txn_new(void); ++JobTxn *block_job_txn_new(void); + + /** + * block_job_txn_unref: +@@ -282,7 +281,7 @@ BlockJobTxn *block_job_txn_new(void); + * or block_job_txn_new. If it's the last reference to the object, it will be + * freed. + */ +-void block_job_txn_unref(BlockJobTxn *txn); ++void block_job_txn_unref(JobTxn *txn); + + /** + * block_job_txn_add_job: +@@ -293,7 +292,7 @@ void block_job_txn_unref(BlockJobTxn *txn); + * The caller must call either block_job_txn_unref() or block_job_completed() + * to release the reference that is automatically grabbed here. + */ +-void block_job_txn_add_job(BlockJobTxn *txn, BlockJob *job); ++void block_job_txn_add_job(JobTxn *txn, BlockJob *job); + + /** + * block_job_is_internal: +diff --git a/include/block/blockjob_int.h b/include/block/blockjob_int.h +index b8ca7bb..ce66a9b 100644 +--- a/include/block/blockjob_int.h ++++ b/include/block/blockjob_int.h +@@ -91,7 +91,7 @@ struct BlockJobDriver { + * called from a wrapper that is specific to the job type. + */ + void *block_job_create(const char *job_id, const BlockJobDriver *driver, +- BlockJobTxn *txn, BlockDriverState *bs, uint64_t perm, ++ JobTxn *txn, BlockDriverState *bs, uint64_t perm, + uint64_t shared_perm, int64_t speed, int flags, + BlockCompletionFunc *cb, void *opaque, Error **errp); + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 17e2cec..d4aa7fa 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -132,6 +132,9 @@ typedef struct Job { + + /** Element of the list of jobs */ + QLIST_ENTRY(Job) job_list; ++ ++ /** Element of the list of jobs in a job transaction */ ++ QLIST_ENTRY(Job) txn_list; + } Job; + + /** +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 1572f8d..ec5d592 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -93,7 +93,7 @@ static const BlockJobDriver test_block_job_driver = { + */ + static BlockJob *test_block_job_start(unsigned int iterations, + bool use_timer, +- int rc, int *result, BlockJobTxn *txn) ++ int rc, int *result, JobTxn *txn) + { + BlockDriverState *bs; + TestBlockJob *s; +@@ -122,7 +122,7 @@ static BlockJob *test_block_job_start(unsigned int iterations, + static void test_single_job(int expected) + { + BlockJob *job; +- BlockJobTxn *txn; ++ JobTxn *txn; + int result = -EINPROGRESS; + + txn = block_job_txn_new(); +@@ -160,7 +160,7 @@ static void test_pair_jobs(int expected1, int expected2) + { + BlockJob *job1; + BlockJob *job2; +- BlockJobTxn *txn; ++ JobTxn *txn; + int result1 = -EINPROGRESS; + int result2 = -EINPROGRESS; + +@@ -222,7 +222,7 @@ static void test_pair_jobs_fail_cancel_race(void) + { + BlockJob *job1; + BlockJob *job2; +- BlockJobTxn *txn; ++ JobTxn *txn; + int result1 = -EINPROGRESS; + int result2 = -EINPROGRESS; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-job-Use-AIO_WAIT_WHILE-in-job_finish_sync.patch b/SOURCES/kvm-job-Use-AIO_WAIT_WHILE-in-job_finish_sync.patch new file mode 100644 index 0000000..b4838e3 --- /dev/null +++ b/SOURCES/kvm-job-Use-AIO_WAIT_WHILE-in-job_finish_sync.patch @@ -0,0 +1,77 @@ +From 011fbd28d5f69f770357bcca6a767605de84fe95 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:00 +0100 +Subject: [PATCH 34/49] job: Use AIO_WAIT_WHILE() in job_finish_sync() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-22-kwolf@redhat.com> +Patchwork-id: 82612 +O-Subject: [RHEL-8 qemu-kvm PATCH 31/44] job: Use AIO_WAIT_WHILE() in job_finish_sync() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +job_finish_sync() needs to release the AioContext lock of the job before +calling aio_poll(). Otherwise, callbacks called by aio_poll() would +possibly take the lock a second time and run into a deadlock with a +nested AIO_WAIT_WHILE() call. + +Also, job_drain() without aio_poll() isn't necessarily enough to make +progress on a job, it could depend on bottom halves to be executed. + +Combine both open-coded while loops into a single AIO_WAIT_WHILE() call +that solves both of these problems. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +Reviewed-by: Max Reitz +(cherry picked from commit de0fbe64806321fc3e6399bfab360553db87a41d) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + job.c | 14 ++++++-------- + 1 file changed, 6 insertions(+), 8 deletions(-) + +diff --git a/job.c b/job.c +index 5a0ccc7..47b5a11 100644 +--- a/job.c ++++ b/job.c +@@ -29,6 +29,7 @@ + #include "qemu/job.h" + #include "qemu/id.h" + #include "qemu/main-loop.h" ++#include "block/aio-wait.h" + #include "trace-root.h" + #include "qapi/qapi-events-job.h" + +@@ -957,6 +958,7 @@ void job_complete(Job *job, Error **errp) + int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + { + Error *local_err = NULL; ++ AioWait dummy_wait = {}; + int ret; + + job_ref(job); +@@ -969,14 +971,10 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + job_unref(job); + return -EBUSY; + } +- /* job_drain calls job_enter, and it should be enough to induce progress +- * until the job completes or moves to the main thread. */ +- while (!job->deferred_to_main_loop && !job_is_completed(job)) { +- job_drain(job); +- } +- while (!job_is_completed(job)) { +- aio_poll(qemu_get_aio_context(), true); +- } ++ ++ AIO_WAIT_WHILE(&dummy_wait, job->aio_context, ++ (job_drain(job), !job_is_completed(job))); ++ + ret = (job_is_cancelled(job) && job->ret == 0) ? -ECANCELED : job->ret; + job_unref(job); + return ret; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-add-exit-shim.patch b/SOURCES/kvm-jobs-add-exit-shim.patch new file mode 100644 index 0000000..abaa9d4 --- /dev/null +++ b/SOURCES/kvm-jobs-add-exit-shim.patch @@ -0,0 +1,108 @@ +From 005ea02d982ce5701b065e1e241bc62133c407b4 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:09 +0100 +Subject: [PATCH 06/28] jobs: add exit shim + +RH-Author: John Snow +Message-id: <20180925223431.24791-4-jsnow@redhat.com> +Patchwork-id: 82273 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 03/25] jobs: add exit shim +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +All jobs do the same thing when they leave their running loop: +- Store the return code in a structure +- wait to receive this structure in the main thread +- signal job completion via job_completed + +Few jobs do anything beyond exactly this. Consolidate this exit +logic for a net reduction in SLOC. + +More seriously, when we utilize job_defer_to_main_loop_bh to call +a function that calls job_completed, job_finalize_single will run +in a context where it has recursively taken the aio_context lock, +which can cause hangs if it puts down a reference that causes a flush. + +You can observe this in practice by looking at mirror_exit's careful +placement of job_completed and bdrv_unref calls. + +If we centralize job exiting, we can signal job completion from outside +of the aio_context, which should allow for job cleanup code to run with +only one lock, which makes cleanup callbacks less tricky to write. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180830015734.19765-4-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 00359a71d45a414ee47d8e423104dc0afd24ec65) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/job.h | 11 +++++++++++ + job.c | 18 ++++++++++++++++++ + 2 files changed, 29 insertions(+) + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 905dfdd..24b5f3f 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -209,6 +209,17 @@ struct JobDriver { + void (*drain)(Job *job); + + /** ++ * If the callback is not NULL, exit will be invoked from the main thread ++ * when the job's coroutine has finished, but before transactional ++ * convergence; before @prepare or @abort. ++ * ++ * FIXME TODO: This callback is only temporary to transition remaining jobs ++ * to prepare/commit/abort/clean callbacks and will be removed before 3.1. ++ * is released. ++ */ ++ void (*exit)(Job *job); ++ ++ /** + * If the callback is not NULL, prepare will be invoked when all the jobs + * belonging to the same transaction complete; or upon this job's completion + * if it is not in a transaction. +diff --git a/job.c b/job.c +index 17b4fad..03b5d65 100644 +--- a/job.c ++++ b/job.c +@@ -530,6 +530,18 @@ void job_drain(Job *job) + } + } + ++static void job_exit(void *opaque) ++{ ++ Job *job = (Job *)opaque; ++ AioContext *aio_context = job->aio_context; ++ ++ if (job->driver->exit) { ++ aio_context_acquire(aio_context); ++ job->driver->exit(job); ++ aio_context_release(aio_context); ++ } ++ job_completed(job, job->ret); ++} + + /** + * All jobs must allow a pause point before entering their job proper. This +@@ -542,6 +554,12 @@ static void coroutine_fn job_co_entry(void *opaque) + assert(job && job->driver && job->driver->run); + job_pause_point(job); + job->ret = job->driver->run(job, &job->err); ++ if (!job->deferred_to_main_loop) { ++ job->deferred_to_main_loop = true; ++ aio_bh_schedule_oneshot(qemu_get_aio_context(), ++ job_exit, ++ job); ++ } + } + + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-canonize-Error-object.patch b/SOURCES/kvm-jobs-canonize-Error-object.patch new file mode 100644 index 0000000..790cc32 --- /dev/null +++ b/SOURCES/kvm-jobs-canonize-Error-object.patch @@ -0,0 +1,285 @@ +From dfed24e446a733eeadf0ee85f3ce8aec63b2b3bf Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:08 +0100 +Subject: [PATCH 05/28] jobs: canonize Error object + +RH-Author: John Snow +Message-id: <20180925223431.24791-3-jsnow@redhat.com> +Patchwork-id: 82262 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 02/25] jobs: canonize Error object +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Jobs presently use both an Error object in the case of the create job, +and char strings in the case of generic errors elsewhere. + +Unify the two paths as just j->err, and remove the extra argument from +job_completed. The integer error code for job_completed is kept for now, +to be removed shortly in a separate patch. + +Signed-off-by: John Snow +Message-id: 20180830015734.19765-3-jsnow@redhat.com +[mreitz: Dropped a superfluous g_strdup()] +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 3d1f8b07a4c241f81949eff507d9f3a8fd73b87b) +Signed-off-by: John Snow + +Signed-off-by: Danilo C. L. de Paula +--- + block/backup.c | 2 +- + block/commit.c | 2 +- + block/create.c | 5 ++--- + block/mirror.c | 2 +- + block/stream.c | 2 +- + include/qemu/job.h | 14 ++++++++------ + job-qmp.c | 5 +++-- + job.c | 18 ++++++------------ + tests/test-bdrv-drain.c | 2 +- + tests/test-blockjob-txn.c | 2 +- + tests/test-blockjob.c | 2 +- + 11 files changed, 26 insertions(+), 30 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index a142518..a5bf699 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -388,7 +388,7 @@ static void backup_complete(Job *job, void *opaque) + { + BackupCompleteData *data = opaque; + +- job_completed(job, data->ret, NULL); ++ job_completed(job, data->ret); + g_free(data); + } + +diff --git a/block/commit.c b/block/commit.c +index 905a1c5..af7579d 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -117,7 +117,7 @@ static void commit_complete(Job *job, void *opaque) + * bdrv_set_backing_hd() to fail. */ + block_job_remove_all_bdrv(bjob); + +- job_completed(job, ret, NULL); ++ job_completed(job, ret); + g_free(data); + + /* If bdrv_drop_intermediate() didn't already do that, remove the commit +diff --git a/block/create.c b/block/create.c +index 04733c3..26a385c 100644 +--- a/block/create.c ++++ b/block/create.c +@@ -35,14 +35,13 @@ typedef struct BlockdevCreateJob { + BlockDriver *drv; + BlockdevCreateOptions *opts; + int ret; +- Error *err; + } BlockdevCreateJob; + + static void blockdev_create_complete(Job *job, void *opaque) + { + BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common); + +- job_completed(job, s->ret, s->err); ++ job_completed(job, s->ret); + } + + static int coroutine_fn blockdev_create_run(Job *job, Error **errp) +@@ -50,7 +49,7 @@ static int coroutine_fn blockdev_create_run(Job *job, Error **errp) + BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common); + + job_progress_set_remaining(&s->common, 1); +- s->ret = s->drv->bdrv_co_create(s->opts, &s->err); ++ s->ret = s->drv->bdrv_co_create(s->opts, errp); + job_progress_update(&s->common, 1); + + qapi_free_BlockdevCreateOptions(s->opts); +diff --git a/block/mirror.c b/block/mirror.c +index 89a92c2..3c330ee 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -581,7 +581,7 @@ static void mirror_exit(Job *job, void *opaque) + blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort); + blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort); + +- job_completed(job, data->ret, NULL); ++ job_completed(job, data->ret); + + g_free(data); + bdrv_drained_end(src); +diff --git a/block/stream.c b/block/stream.c +index b4b987d..26a7753 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -93,7 +93,7 @@ out: + } + + g_free(s->backing_file_str); +- job_completed(job, data->ret, NULL); ++ job_completed(job, data->ret); + g_free(data); + } + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index e81cc34..905dfdd 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -124,12 +124,16 @@ typedef struct Job { + /** Estimated progress_current value at the completion of the job */ + int64_t progress_total; + +- /** Error string for a failed job (NULL if, and only if, job->ret == 0) */ +- char *error; +- + /** ret code passed to job_completed. */ + int ret; + ++ /** ++ * Error object for a failed job. ++ * If job->ret is nonzero and an error object was not set, it will be set ++ * to strerror(-job->ret) during job_completed. ++ */ ++ Error *err; ++ + /** The completion function that will be called when the job completes. */ + BlockCompletionFunc *cb; + +@@ -469,15 +473,13 @@ void job_transition_to_ready(Job *job); + /** + * @job: The job being completed. + * @ret: The status code. +- * @error: The error message for a failing job (only with @ret < 0). If @ret is +- * negative, but NULL is given for @error, strerror() is used. + * + * Marks @job as completed. If @ret is non-zero, the job transaction it is part + * of is aborted. If @ret is zero, the job moves into the WAITING state. If it + * is the last job to complete in its transaction, all jobs in the transaction + * move from WAITING to PENDING. + */ +-void job_completed(Job *job, int ret, Error *error); ++void job_completed(Job *job, int ret); + + /** Asynchronously complete the specified @job. */ + void job_complete(Job *job, Error **errp); +diff --git a/job-qmp.c b/job-qmp.c +index 410775d..a969b2b 100644 +--- a/job-qmp.c ++++ b/job-qmp.c +@@ -146,8 +146,9 @@ static JobInfo *job_query_single(Job *job, Error **errp) + .status = job->status, + .current_progress = job->progress_current, + .total_progress = job->progress_total, +- .has_error = !!job->error, +- .error = g_strdup(job->error), ++ .has_error = !!job->err, ++ .error = job->err ? \ ++ g_strdup(error_get_pretty(job->err)) : NULL, + }; + + return info; +diff --git a/job.c b/job.c +index d4e3041..17b4fad 100644 +--- a/job.c ++++ b/job.c +@@ -369,7 +369,7 @@ void job_unref(Job *job) + + QLIST_REMOVE(job, job_list); + +- g_free(job->error); ++ error_free(job->err); + g_free(job->id); + g_free(job); + } +@@ -541,7 +541,7 @@ static void coroutine_fn job_co_entry(void *opaque) + + assert(job && job->driver && job->driver->run); + job_pause_point(job); +- job->ret = job->driver->run(job, NULL); ++ job->ret = job->driver->run(job, &job->err); + } + + +@@ -661,8 +661,8 @@ static void job_update_rc(Job *job) + job->ret = -ECANCELED; + } + if (job->ret) { +- if (!job->error) { +- job->error = g_strdup(strerror(-job->ret)); ++ if (!job->err) { ++ error_setg(&job->err, "%s", strerror(-job->ret)); + } + job_state_transition(job, JOB_STATUS_ABORTING); + } +@@ -860,17 +860,11 @@ static void job_completed_txn_success(Job *job) + } + } + +-void job_completed(Job *job, int ret, Error *error) ++void job_completed(Job *job, int ret) + { + assert(job && job->txn && !job_is_completed(job)); + + job->ret = ret; +- if (error) { +- assert(job->ret < 0); +- job->error = g_strdup(error_get_pretty(error)); +- error_free(error); +- } +- + job_update_rc(job); + trace_job_completed(job, ret, job->ret); + if (job->ret) { +@@ -888,7 +882,7 @@ void job_cancel(Job *job, bool force) + } + job_cancel_async(job, force); + if (!job_started(job)) { +- job_completed(job, -ECANCELED, NULL); ++ job_completed(job, -ECANCELED); + } else if (job->deferred_to_main_loop) { + job_completed_txn_abort(job); + } else { +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 798445a..8937894 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -498,7 +498,7 @@ typedef struct TestBlockJob { + + static void test_job_completed(Job *job, void *opaque) + { +- job_completed(job, 0, NULL); ++ job_completed(job, 0); + } + + static int coroutine_fn test_job_run(Job *job, Error **errp) +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 3194924..82cedee 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -34,7 +34,7 @@ static void test_block_job_complete(Job *job, void *opaque) + rc = -ECANCELED; + } + +- job_completed(job, rc, NULL); ++ job_completed(job, rc); + bdrv_unref(bs); + } + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index b0462bf..408a226 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -167,7 +167,7 @@ static void cancel_job_completed(Job *job, void *opaque) + { + CancelJob *s = opaque; + s->completed = true; +- job_completed(job, 0, NULL); ++ job_completed(job, 0); + } + + static void cancel_job_complete(Job *job, Error **errp) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-change-start-callback-to-run-callback.patch b/SOURCES/kvm-jobs-change-start-callback-to-run-callback.patch new file mode 100644 index 0000000..8a4bab9 --- /dev/null +++ b/SOURCES/kvm-jobs-change-start-callback-to-run-callback.patch @@ -0,0 +1,372 @@ +From 660f34302ebc9474e821346003f2e0f46bc6507c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:07 +0100 +Subject: [PATCH 04/28] jobs: change start callback to run callback + +RH-Author: John Snow +Message-id: <20180925223431.24791-2-jsnow@redhat.com> +Patchwork-id: 82261 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 01/25] jobs: change start callback to run callback +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Presently we codify the entry point for a job as the "start" callback, +but a more apt name would be "run" to clarify the idea that when this +function returns we consider the job to have "finished," except for +any cleanup which occurs in separate callbacks later. + +As part of this clarification, change the signature to include an error +object and a return code. The error ptr is not yet used, and the return +code while captured, will be overwritten by actions in the job_completed +function. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180830015734.19765-2-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit f67432a2019caf05b57a146bf45c1024a5cb608e) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/backup.c | 7 ++++--- + block/commit.c | 7 ++++--- + block/create.c | 8 +++++--- + block/mirror.c | 10 ++++++---- + block/stream.c | 7 ++++--- + include/qemu/job.h | 2 +- + job.c | 6 +++--- + tests/test-bdrv-drain.c | 7 ++++--- + tests/test-blockjob-txn.c | 16 ++++++++-------- + tests/test-blockjob.c | 7 ++++--- + 10 files changed, 43 insertions(+), 34 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index 4ba1a6a..a142518 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -480,9 +480,9 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) + bdrv_dirty_iter_free(dbi); + } + +-static void coroutine_fn backup_run(void *opaque) ++static int coroutine_fn backup_run(Job *opaque_job, Error **errp) + { +- BackupBlockJob *job = opaque; ++ BackupBlockJob *job = container_of(opaque_job, BackupBlockJob, common.job); + BackupCompleteData *data; + BlockDriverState *bs = blk_bs(job->common.blk); + int64_t offset, nb_clusters; +@@ -587,6 +587,7 @@ static void coroutine_fn backup_run(void *opaque) + data = g_malloc(sizeof(*data)); + data->ret = ret; + job_defer_to_main_loop(&job->common.job, backup_complete, data); ++ return ret; + } + + static const BlockJobDriver backup_job_driver = { +@@ -596,7 +597,7 @@ static const BlockJobDriver backup_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = backup_run, ++ .run = backup_run, + .commit = backup_commit, + .abort = backup_abort, + .clean = backup_clean, +diff --git a/block/commit.c b/block/commit.c +index e1814d9..905a1c5 100644 +--- a/block/commit.c ++++ b/block/commit.c +@@ -134,9 +134,9 @@ static void commit_complete(Job *job, void *opaque) + bdrv_unref(top); + } + +-static void coroutine_fn commit_run(void *opaque) ++static int coroutine_fn commit_run(Job *job, Error **errp) + { +- CommitBlockJob *s = opaque; ++ CommitBlockJob *s = container_of(job, CommitBlockJob, common.job); + CommitCompleteData *data; + int64_t offset; + uint64_t delay_ns = 0; +@@ -213,6 +213,7 @@ out: + data = g_malloc(sizeof(*data)); + data->ret = ret; + job_defer_to_main_loop(&s->common.job, commit_complete, data); ++ return ret; + } + + static const BlockJobDriver commit_job_driver = { +@@ -222,7 +223,7 @@ static const BlockJobDriver commit_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = commit_run, ++ .run = commit_run, + }, + }; + +diff --git a/block/create.c b/block/create.c +index 915cd41..04733c3 100644 +--- a/block/create.c ++++ b/block/create.c +@@ -45,9 +45,9 @@ static void blockdev_create_complete(Job *job, void *opaque) + job_completed(job, s->ret, s->err); + } + +-static void coroutine_fn blockdev_create_run(void *opaque) ++static int coroutine_fn blockdev_create_run(Job *job, Error **errp) + { +- BlockdevCreateJob *s = opaque; ++ BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common); + + job_progress_set_remaining(&s->common, 1); + s->ret = s->drv->bdrv_co_create(s->opts, &s->err); +@@ -55,12 +55,14 @@ static void coroutine_fn blockdev_create_run(void *opaque) + + qapi_free_BlockdevCreateOptions(s->opts); + job_defer_to_main_loop(&s->common, blockdev_create_complete, NULL); ++ ++ return s->ret; + } + + static const JobDriver blockdev_create_job_driver = { + .instance_size = sizeof(BlockdevCreateJob), + .job_type = JOB_TYPE_CREATE, +- .start = blockdev_create_run, ++ .run = blockdev_create_run, + }; + + void qmp_blockdev_create(const char *job_id, BlockdevCreateOptions *options, +diff --git a/block/mirror.c b/block/mirror.c +index 435268b..89a92c2 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -683,9 +683,9 @@ static int mirror_flush(MirrorBlockJob *s) + return ret; + } + +-static void coroutine_fn mirror_run(void *opaque) ++static int coroutine_fn mirror_run(Job *job, Error **errp) + { +- MirrorBlockJob *s = opaque; ++ MirrorBlockJob *s = container_of(job, MirrorBlockJob, common.job); + MirrorExitData *data; + BlockDriverState *bs = s->source; + BlockDriverState *target_bs = blk_bs(s->target); +@@ -902,7 +902,9 @@ immediate_exit: + if (need_drain) { + bdrv_drained_begin(bs); + } ++ + job_defer_to_main_loop(&s->common.job, mirror_exit, data); ++ return ret; + } + + static void mirror_complete(Job *job, Error **errp) +@@ -993,7 +995,7 @@ static const BlockJobDriver mirror_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = mirror_run, ++ .run = mirror_run, + .pause = mirror_pause, + .complete = mirror_complete, + }, +@@ -1008,7 +1010,7 @@ static const BlockJobDriver commit_active_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = mirror_run, ++ .run = mirror_run, + .pause = mirror_pause, + .complete = mirror_complete, + }, +diff --git a/block/stream.c b/block/stream.c +index 9264b68..b4b987d 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -97,9 +97,9 @@ out: + g_free(data); + } + +-static void coroutine_fn stream_run(void *opaque) ++static int coroutine_fn stream_run(Job *job, Error **errp) + { +- StreamBlockJob *s = opaque; ++ StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); + StreamCompleteData *data; + BlockBackend *blk = s->common.blk; + BlockDriverState *bs = blk_bs(blk); +@@ -206,6 +206,7 @@ out: + data = g_malloc(sizeof(*data)); + data->ret = ret; + job_defer_to_main_loop(&s->common.job, stream_complete, data); ++ return ret; + } + + static const BlockJobDriver stream_job_driver = { +@@ -213,7 +214,7 @@ static const BlockJobDriver stream_job_driver = { + .instance_size = sizeof(StreamBlockJob), + .job_type = JOB_TYPE_STREAM, + .free = block_job_free, +- .start = stream_run, ++ .run = stream_run, + .user_resume = block_job_user_resume, + .drain = block_job_drain, + }, +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 1d82053..e81cc34 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -169,7 +169,7 @@ struct JobDriver { + JobType job_type; + + /** Mandatory: Entrypoint for the Coroutine. */ +- CoroutineEntry *start; ++ int coroutine_fn (*run)(Job *job, Error **errp); + + /** + * If the callback is not NULL, it will be invoked when the job transitions +diff --git a/job.c b/job.c +index 84e1402..d4e3041 100644 +--- a/job.c ++++ b/job.c +@@ -539,16 +539,16 @@ static void coroutine_fn job_co_entry(void *opaque) + { + Job *job = opaque; + +- assert(job && job->driver && job->driver->start); ++ assert(job && job->driver && job->driver->run); + job_pause_point(job); +- job->driver->start(job); ++ job->ret = job->driver->run(job, NULL); + } + + + void job_start(Job *job) + { + assert(job && !job_started(job) && job->paused && +- job->driver && job->driver->start); ++ job->driver && job->driver->run); + job->co = qemu_coroutine_create(job_co_entry, job); + job->pause_count--; + job->busy = true; +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index a11c4cf..798445a 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -501,9 +501,9 @@ static void test_job_completed(Job *job, void *opaque) + job_completed(job, 0, NULL); + } + +-static void coroutine_fn test_job_start(void *opaque) ++static int coroutine_fn test_job_run(Job *job, Error **errp) + { +- TestBlockJob *s = opaque; ++ TestBlockJob *s = container_of(job, TestBlockJob, common.job); + + job_transition_to_ready(&s->common.job); + while (!s->should_complete) { +@@ -511,6 +511,7 @@ static void coroutine_fn test_job_start(void *opaque) + } + + job_defer_to_main_loop(&s->common.job, test_job_completed, NULL); ++ return 0; + } + + static void test_job_complete(Job *job, Error **errp) +@@ -525,7 +526,7 @@ BlockJobDriver test_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = test_job_start, ++ .run = test_job_run, + .complete = test_job_complete, + }, + }; +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 58d9b87..3194924 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -38,25 +38,25 @@ static void test_block_job_complete(Job *job, void *opaque) + bdrv_unref(bs); + } + +-static void coroutine_fn test_block_job_run(void *opaque) ++static int coroutine_fn test_block_job_run(Job *job, Error **errp) + { +- TestBlockJob *s = opaque; +- BlockJob *job = &s->common; ++ TestBlockJob *s = container_of(job, TestBlockJob, common.job); + + while (s->iterations--) { + if (s->use_timer) { +- job_sleep_ns(&job->job, 0); ++ job_sleep_ns(job, 0); + } else { +- job_yield(&job->job); ++ job_yield(job); + } + +- if (job_is_cancelled(&job->job)) { ++ if (job_is_cancelled(job)) { + break; + } + } + +- job_defer_to_main_loop(&job->job, test_block_job_complete, ++ job_defer_to_main_loop(job, test_block_job_complete, + (void *)(intptr_t)s->rc); ++ return s->rc; + } + + typedef struct { +@@ -80,7 +80,7 @@ static const BlockJobDriver test_block_job_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = test_block_job_run, ++ .run = test_block_job_run, + }, + }; + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index cb42f06..b0462bf 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -176,9 +176,9 @@ static void cancel_job_complete(Job *job, Error **errp) + s->should_complete = true; + } + +-static void coroutine_fn cancel_job_start(void *opaque) ++static int coroutine_fn cancel_job_run(Job *job, Error **errp) + { +- CancelJob *s = opaque; ++ CancelJob *s = container_of(job, CancelJob, common.job); + + while (!s->should_complete) { + if (job_is_cancelled(&s->common.job)) { +@@ -194,6 +194,7 @@ static void coroutine_fn cancel_job_start(void *opaque) + + defer: + job_defer_to_main_loop(&s->common.job, cancel_job_completed, s); ++ return 0; + } + + static const BlockJobDriver test_cancel_driver = { +@@ -202,7 +203,7 @@ static const BlockJobDriver test_cancel_driver = { + .free = block_job_free, + .user_resume = block_job_user_resume, + .drain = block_job_drain, +- .start = cancel_job_start, ++ .run = cancel_job_run, + .complete = cancel_job_complete, + }, + }; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-fix-stale-wording.patch b/SOURCES/kvm-jobs-fix-stale-wording.patch new file mode 100644 index 0000000..bcc7df1 --- /dev/null +++ b/SOURCES/kvm-jobs-fix-stale-wording.patch @@ -0,0 +1,62 @@ +From c0c27707f886ddbf8bc853bf8f798278b1bb713d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:53 +0200 +Subject: [PATCH 145/268] jobs: fix stale wording + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-71-kwolf@redhat.com> +Patchwork-id: 81107 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 70/73] jobs: fix stale wording +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: John Snow + +During the design for manual completion, we decided not to use the +"manual" property as a shorthand for both auto-dismiss and auto-finalize. + +Fix the wording. + +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Reviewed-by: Markus Armbruster +Signed-off-by: Kevin Wolf +(cherry picked from commit c5b09f3f2c4db32f39ecbde2cf2f78e7a657a85d) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + qapi/job.json | 11 ++++++----- + 1 file changed, 6 insertions(+), 5 deletions(-) + +diff --git a/qapi/job.json b/qapi/job.json +index 17d1003..2264435 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -50,16 +50,17 @@ + # the last job in a transaction. + # + # @pending: The job has finished its work, but has finalization steps that it +-# needs to make prior to completing. These changes may require +-# manual intervention by the management process if manual was set +-# to true. These changes may still fail. ++# needs to make prior to completing. These changes will require ++# manual intervention via @job-finalize if auto-finalize was set to ++# false. These pending changes may still fail. + # + # @aborting: The job is in the process of being aborted, and will finish with + # an error. The job will afterwards report that it is @concluded. + # This status may not be visible to the management process. + # +-# @concluded: The job has finished all work. If manual was set to true, the job +-# will remain in the query list until it is dismissed. ++# @concluded: The job has finished all work. If auto-dismiss was set to false, ++# the job will remain in the query list until it is dismissed via ++# @job-dismiss. + # + # @null: The job is in the process of being dismantled. This state should not + # ever be visible externally. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-fix-verb-references-in-docs.patch b/SOURCES/kvm-jobs-fix-verb-references-in-docs.patch new file mode 100644 index 0000000..b175e3c --- /dev/null +++ b/SOURCES/kvm-jobs-fix-verb-references-in-docs.patch @@ -0,0 +1,66 @@ +From a7d31b5b668307b4ce48344e4aab8afb413e8c50 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:54 +0200 +Subject: [PATCH 146/268] jobs: fix verb references in docs + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-72-kwolf@redhat.com> +Patchwork-id: 81108 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 71/73] jobs: fix verb references in docs +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: John Snow + +These point to the job versions now, not the blockjob versions which +don't really exist anymore. + +Except set-speed, which does. It sticks out like a sore thumb. This +patch doesn't fix that, but it doesn't make it any worse, either. + +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Reviewed-by: Markus Armbruster +Signed-off-by: Kevin Wolf +(cherry picked from commit b8a366feb2aa2200c42fa983a9c607fb78a501db) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + qapi/job.json | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) + +diff --git a/qapi/job.json b/qapi/job.json +index 2264435..9d074eb 100644 +--- a/qapi/job.json ++++ b/qapi/job.json +@@ -76,19 +76,19 @@ + # + # Represents command verbs that can be applied to a job. + # +-# @cancel: see @block-job-cancel ++# @cancel: see @job-cancel + # +-# @pause: see @block-job-pause ++# @pause: see @job-pause + # +-# @resume: see @block-job-resume ++# @resume: see @job-resume + # + # @set-speed: see @block-job-set-speed + # +-# @complete: see @block-job-complete ++# @complete: see @job-complete + # +-# @dismiss: see @block-job-dismiss ++# @dismiss: see @job-dismiss + # +-# @finalize: see @block-job-finalize ++# @finalize: see @job-finalize + # + # Since: 2.12 + ## +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-remove-.exit-callback.patch b/SOURCES/kvm-jobs-remove-.exit-callback.patch new file mode 100644 index 0000000..940c55d --- /dev/null +++ b/SOURCES/kvm-jobs-remove-.exit-callback.patch @@ -0,0 +1,156 @@ +From f7507b149eb7470ce9f9f1d2e04050a82b4c94de Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:26 +0100 +Subject: [PATCH 23/28] jobs: remove .exit callback + +RH-Author: John Snow +Message-id: <20180925223431.24791-21-jsnow@redhat.com> +Patchwork-id: 82283 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 20/25] jobs: remove .exit callback +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Now that all of the jobs use the component finalization callbacks, +there's no use for the heavy-hammer .exit callback anymore. + +job_exit becomes a glorified type shim so that we can call +job_completed from aio_bh_schedule_oneshot. + +Move these three functions down into job.c to eliminate a +forward reference. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-12-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit ccbfb3319aa265e71c16dac976ff857d0a5bcb4b) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/job.h | 11 -------- + job.c | 77 ++++++++++++++++++++++++------------------------------ + 2 files changed, 34 insertions(+), 54 deletions(-) + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 04090ba..fdaa06f 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -222,17 +222,6 @@ struct JobDriver { + void (*drain)(Job *job); + + /** +- * If the callback is not NULL, exit will be invoked from the main thread +- * when the job's coroutine has finished, but before transactional +- * convergence; before @prepare or @abort. +- * +- * FIXME TODO: This callback is only temporary to transition remaining jobs +- * to prepare/commit/abort/clean callbacks and will be removed before 3.1. +- * is released. +- */ +- void (*exit)(Job *job); +- +- /** + * If the callback is not NULL, prepare will be invoked when all the jobs + * belonging to the same transaction complete; or upon this job's completion + * if it is not in a transaction. +diff --git a/job.c b/job.c +index edcae98..3ab8ce1 100644 +--- a/job.c ++++ b/job.c +@@ -530,49 +530,6 @@ void job_drain(Job *job) + } + } + +-static void job_completed(Job *job); +- +-static void job_exit(void *opaque) +-{ +- Job *job = (Job *)opaque; +- AioContext *aio_context = job->aio_context; +- +- if (job->driver->exit) { +- aio_context_acquire(aio_context); +- job->driver->exit(job); +- aio_context_release(aio_context); +- } +- job_completed(job); +-} +- +-/** +- * All jobs must allow a pause point before entering their job proper. This +- * ensures that jobs can be paused prior to being started, then resumed later. +- */ +-static void coroutine_fn job_co_entry(void *opaque) +-{ +- Job *job = opaque; +- +- assert(job && job->driver && job->driver->run); +- job_pause_point(job); +- job->ret = job->driver->run(job, &job->err); +- job->deferred_to_main_loop = true; +- aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); +-} +- +- +-void job_start(Job *job) +-{ +- assert(job && !job_started(job) && job->paused && +- job->driver && job->driver->run); +- job->co = qemu_coroutine_create(job_co_entry, job); +- job->pause_count--; +- job->busy = true; +- job->paused = false; +- job_state_transition(job, JOB_STATUS_RUNNING); +- aio_co_enter(job->aio_context, job->co); +-} +- + /* Assumes the block_job_mutex is held */ + static bool job_timer_not_pending(Job *job) + { +@@ -889,6 +846,40 @@ static void job_completed(Job *job) + } + } + ++/** Useful only as a type shim for aio_bh_schedule_oneshot. */ ++static void job_exit(void *opaque) ++{ ++ Job *job = (Job *)opaque; ++ job_completed(job); ++} ++ ++/** ++ * All jobs must allow a pause point before entering their job proper. This ++ * ensures that jobs can be paused prior to being started, then resumed later. ++ */ ++static void coroutine_fn job_co_entry(void *opaque) ++{ ++ Job *job = opaque; ++ ++ assert(job && job->driver && job->driver->run); ++ job_pause_point(job); ++ job->ret = job->driver->run(job, &job->err); ++ job->deferred_to_main_loop = true; ++ aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); ++} ++ ++void job_start(Job *job) ++{ ++ assert(job && !job_started(job) && job->paused && ++ job->driver && job->driver->run); ++ job->co = qemu_coroutine_create(job_co_entry, job); ++ job->pause_count--; ++ job->busy = true; ++ job->paused = false; ++ job_state_transition(job, JOB_STATUS_RUNNING); ++ aio_co_enter(job->aio_context, job->co); ++} ++ + void job_cancel(Job *job, bool force) + { + if (job->status == JOB_STATUS_CONCLUDED) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-remove-job_defer_to_main_loop.patch b/SOURCES/kvm-jobs-remove-job_defer_to_main_loop.patch new file mode 100644 index 0000000..0096e15 --- /dev/null +++ b/SOURCES/kvm-jobs-remove-job_defer_to_main_loop.patch @@ -0,0 +1,119 @@ +From ba435686f6f81197fe01a47ee8962b5ea60fce6c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:15 +0100 +Subject: [PATCH 12/28] jobs: remove job_defer_to_main_loop + +RH-Author: John Snow +Message-id: <20180925223431.24791-10-jsnow@redhat.com> +Patchwork-id: 82275 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 09/25] jobs: remove job_defer_to_main_loop +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Now that the job infrastructure is handling the job_completed call for +all implemented jobs, we can remove the interface that allowed jobs to +schedule their own completion. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180830015734.19765-10-jsnow@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit e21a1c9831fc80ae3f3c1affdfa43350035d8588) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/job.h | 17 ----------------- + job.c | 40 ++-------------------------------------- + 2 files changed, 2 insertions(+), 55 deletions(-) + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 952ac3a..04090ba 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -553,23 +553,6 @@ void job_finalize(Job *job, Error **errp); + */ + void job_dismiss(Job **job, Error **errp); + +-typedef void JobDeferToMainLoopFn(Job *job, void *opaque); +- +-/** +- * @job: The job +- * @fn: The function to run in the main loop +- * @opaque: The opaque value that is passed to @fn +- * +- * This function must be called by the main job coroutine just before it +- * returns. @fn is executed in the main loop with the job AioContext acquired. +- * +- * Block jobs must call bdrv_unref(), bdrv_close(), and anything that uses +- * bdrv_drain_all() in the main loop. +- * +- * The @job AioContext is held while @fn executes. +- */ +-void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque); +- + /** + * Synchronously finishes the given @job. If @finish is given, it is called to + * trigger completion or cancellation of the job. +diff --git a/job.c b/job.c +index e935a36..edcae98 100644 +--- a/job.c ++++ b/job.c +@@ -556,12 +556,8 @@ static void coroutine_fn job_co_entry(void *opaque) + assert(job && job->driver && job->driver->run); + job_pause_point(job); + job->ret = job->driver->run(job, &job->err); +- if (!job->deferred_to_main_loop) { +- job->deferred_to_main_loop = true; +- aio_bh_schedule_oneshot(qemu_get_aio_context(), +- job_exit, +- job); +- } ++ job->deferred_to_main_loop = true; ++ aio_bh_schedule_oneshot(qemu_get_aio_context(), job_exit, job); + } + + +@@ -964,38 +960,6 @@ void job_complete(Job *job, Error **errp) + job->driver->complete(job, errp); + } + +- +-typedef struct { +- Job *job; +- JobDeferToMainLoopFn *fn; +- void *opaque; +-} JobDeferToMainLoopData; +- +-static void job_defer_to_main_loop_bh(void *opaque) +-{ +- JobDeferToMainLoopData *data = opaque; +- Job *job = data->job; +- AioContext *aio_context = job->aio_context; +- +- aio_context_acquire(aio_context); +- data->fn(data->job, data->opaque); +- aio_context_release(aio_context); +- +- g_free(data); +-} +- +-void job_defer_to_main_loop(Job *job, JobDeferToMainLoopFn *fn, void *opaque) +-{ +- JobDeferToMainLoopData *data = g_malloc(sizeof(*data)); +- data->job = job; +- data->fn = fn; +- data->opaque = opaque; +- job->deferred_to_main_loop = true; +- +- aio_bh_schedule_oneshot(qemu_get_aio_context(), +- job_defer_to_main_loop_bh, data); +-} +- + int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp) + { + Error *local_err = NULL; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-remove-ret-argument-to-job_completed-privatize-.patch b/SOURCES/kvm-jobs-remove-ret-argument-to-job_completed-privatize-.patch new file mode 100644 index 0000000..0622839 --- /dev/null +++ b/SOURCES/kvm-jobs-remove-ret-argument-to-job_completed-privatize-.patch @@ -0,0 +1,154 @@ +From 4a583cc4bcba75b26f06efed94869555bf527ce0 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:14 +0100 +Subject: [PATCH 11/28] jobs: remove ret argument to job_completed; privatize + it + +RH-Author: John Snow +Message-id: <20180925223431.24791-9-jsnow@redhat.com> +Patchwork-id: 82271 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 08/25] jobs: remove ret argument to job_completed; privatize it +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Jobs are now expected to return their retcode on the stack, from the +.run callback, so we can remove that argument. + +job_cancel does not need to set -ECANCELED because job_completed will +update the return code itself if the job was canceled. + +While we're here, make job_completed static to job.c and remove it from +job.h; move the documentation of return code to the .run() callback and +to the job->ret property, accordingly. + +Signed-off-by: John Snow +Message-id: 20180830015734.19765-9-jsnow@redhat.com +Reviewed-by: Max Reitz +Signed-off-by: Max Reitz +(cherry picked from commit 404ff28d6ae59fc1c24d631710d4063fc68aed03) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/job.h | 28 +++++++++++++++------------- + job.c | 11 ++++++----- + trace-events | 2 +- + 3 files changed, 22 insertions(+), 19 deletions(-) + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 24b5f3f..952ac3a 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -124,7 +124,11 @@ typedef struct Job { + /** Estimated progress_current value at the completion of the job */ + int64_t progress_total; + +- /** ret code passed to job_completed. */ ++ /** ++ * Return code from @run and/or @prepare callback(s). ++ * Not final until the job has reached the CONCLUDED status. ++ * 0 on success, -errno on failure. ++ */ + int ret; + + /** +@@ -172,7 +176,16 @@ struct JobDriver { + /** Enum describing the operation */ + JobType job_type; + +- /** Mandatory: Entrypoint for the Coroutine. */ ++ /** ++ * Mandatory: Entrypoint for the Coroutine. ++ * ++ * This callback will be invoked when moving from CREATED to RUNNING. ++ * ++ * If this callback returns nonzero, the job transaction it is part of is ++ * aborted. If it returns zero, the job moves into the WAITING state. If it ++ * is the last job to complete in its transaction, all jobs in the ++ * transaction move from WAITING to PENDING. ++ */ + int coroutine_fn (*run)(Job *job, Error **errp); + + /** +@@ -481,17 +494,6 @@ void job_early_fail(Job *job); + /** Moves the @job from RUNNING to READY */ + void job_transition_to_ready(Job *job); + +-/** +- * @job: The job being completed. +- * @ret: The status code. +- * +- * Marks @job as completed. If @ret is non-zero, the job transaction it is part +- * of is aborted. If @ret is zero, the job moves into the WAITING state. If it +- * is the last job to complete in its transaction, all jobs in the transaction +- * move from WAITING to PENDING. +- */ +-void job_completed(Job *job, int ret); +- + /** Asynchronously complete the specified @job. */ + void job_complete(Job *job, Error **errp); + +diff --git a/job.c b/job.c +index 03b5d65..e935a36 100644 +--- a/job.c ++++ b/job.c +@@ -530,6 +530,8 @@ void job_drain(Job *job) + } + } + ++static void job_completed(Job *job); ++ + static void job_exit(void *opaque) + { + Job *job = (Job *)opaque; +@@ -540,7 +542,7 @@ static void job_exit(void *opaque) + job->driver->exit(job); + aio_context_release(aio_context); + } +- job_completed(job, job->ret); ++ job_completed(job); + } + + /** +@@ -878,13 +880,12 @@ static void job_completed_txn_success(Job *job) + } + } + +-void job_completed(Job *job, int ret) ++static void job_completed(Job *job) + { + assert(job && job->txn && !job_is_completed(job)); + +- job->ret = ret; + job_update_rc(job); +- trace_job_completed(job, ret, job->ret); ++ trace_job_completed(job, job->ret); + if (job->ret) { + job_completed_txn_abort(job); + } else { +@@ -900,7 +901,7 @@ void job_cancel(Job *job, bool force) + } + job_cancel_async(job, force); + if (!job_started(job)) { +- job_completed(job, -ECANCELED); ++ job_completed(job); + } else if (job->deferred_to_main_loop) { + job_completed_txn_abort(job); + } else { +diff --git a/trace-events b/trace-events +index c445f54..4fd2cb4 100644 +--- a/trace-events ++++ b/trace-events +@@ -107,7 +107,7 @@ gdbstub_err_checksum_incorrect(uint8_t expected, uint8_t got) "got command packe + # job.c + job_state_transition(void *job, int ret, const char *legal, const char *s0, const char *s1) "job %p (ret: %d) attempting %s transition (%s-->%s)" + job_apply_verb(void *job, const char *state, const char *verb, const char *legal) "job %p in state %s; applying verb %s (%s)" +-job_completed(void *job, int ret, int jret) "job %p ret %d corrected ret %d" ++job_completed(void *job, int ret) "job %p ret %d" + + # job-qmp.c + qmp_job_cancel(void *job) "job %p" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-jobs-utilize-job_exit-shim.patch b/SOURCES/kvm-jobs-utilize-job_exit-shim.patch new file mode 100644 index 0000000..2ff6708 --- /dev/null +++ b/SOURCES/kvm-jobs-utilize-job_exit-shim.patch @@ -0,0 +1,307 @@ +From 98c027dc5d49edb499749092d647edd620ca7d68 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:12 +0100 +Subject: [PATCH 09/28] jobs: utilize job_exit shim + +RH-Author: John Snow +Message-id: <20180925223431.24791-7-jsnow@redhat.com> +Patchwork-id: 82267 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 06/25] jobs: utilize job_exit shim +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Utilize the job_exit shim by not calling job_defer_to_main_loop, and +where applicable, converting the deferred callback into the job_exit +callback. + +This converts backup, stream, create, and the unit tests all at once. +Most of these jobs do not see any changes to the order in which they +clean up their resources, except the test-blockjob-txn test, which +now puts down its bs before job_completed is called. + +This is safe for the same reason the reordering in the mirror job is +safe, because job_completed no longer runs under two locks, making +the unref safe even if it causes a flush. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180830015734.19765-7-jsnow@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit eb23654dbe43b549ea2a9ebff9d8edf544d34a73) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + block/backup.c | 16 ---------------- + block/create.c | 14 +++----------- + block/stream.c | 22 +++++++--------------- + tests/test-bdrv-drain.c | 6 ------ + tests/test-blockjob-txn.c | 11 ++--------- + tests/test-blockjob.c | 10 ++++------ + 6 files changed, 16 insertions(+), 63 deletions(-) + +diff --git a/block/backup.c b/block/backup.c +index a5bf699..08a5b74 100644 +--- a/block/backup.c ++++ b/block/backup.c +@@ -380,18 +380,6 @@ static BlockErrorAction backup_error_action(BackupBlockJob *job, + } + } + +-typedef struct { +- int ret; +-} BackupCompleteData; +- +-static void backup_complete(Job *job, void *opaque) +-{ +- BackupCompleteData *data = opaque; +- +- job_completed(job, data->ret); +- g_free(data); +-} +- + static bool coroutine_fn yield_and_check(BackupBlockJob *job) + { + uint64_t delay_ns; +@@ -483,7 +471,6 @@ static void backup_incremental_init_copy_bitmap(BackupBlockJob *job) + static int coroutine_fn backup_run(Job *opaque_job, Error **errp) + { + BackupBlockJob *job = container_of(opaque_job, BackupBlockJob, common.job); +- BackupCompleteData *data; + BlockDriverState *bs = blk_bs(job->common.blk); + int64_t offset, nb_clusters; + int ret = 0; +@@ -584,9 +571,6 @@ static int coroutine_fn backup_run(Job *opaque_job, Error **errp) + qemu_co_rwlock_unlock(&job->flush_rwlock); + hbitmap_free(job->copy_bitmap); + +- data = g_malloc(sizeof(*data)); +- data->ret = ret; +- job_defer_to_main_loop(&job->common.job, backup_complete, data); + return ret; + } + +diff --git a/block/create.c b/block/create.c +index 26a385c..9534121 100644 +--- a/block/create.c ++++ b/block/create.c +@@ -34,28 +34,20 @@ typedef struct BlockdevCreateJob { + Job common; + BlockDriver *drv; + BlockdevCreateOptions *opts; +- int ret; + } BlockdevCreateJob; + +-static void blockdev_create_complete(Job *job, void *opaque) +-{ +- BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common); +- +- job_completed(job, s->ret); +-} +- + static int coroutine_fn blockdev_create_run(Job *job, Error **errp) + { + BlockdevCreateJob *s = container_of(job, BlockdevCreateJob, common); ++ int ret; + + job_progress_set_remaining(&s->common, 1); +- s->ret = s->drv->bdrv_co_create(s->opts, errp); ++ ret = s->drv->bdrv_co_create(s->opts, errp); + job_progress_update(&s->common, 1); + + qapi_free_BlockdevCreateOptions(s->opts); +- job_defer_to_main_loop(&s->common, blockdev_create_complete, NULL); + +- return s->ret; ++ return ret; + } + + static const JobDriver blockdev_create_job_driver = { +diff --git a/block/stream.c b/block/stream.c +index 26a7753..67e1e72 100644 +--- a/block/stream.c ++++ b/block/stream.c +@@ -54,20 +54,16 @@ static int coroutine_fn stream_populate(BlockBackend *blk, + return blk_co_preadv(blk, offset, qiov.size, &qiov, BDRV_REQ_COPY_ON_READ); + } + +-typedef struct { +- int ret; +-} StreamCompleteData; +- +-static void stream_complete(Job *job, void *opaque) ++static void stream_exit(Job *job) + { + StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); + BlockJob *bjob = &s->common; +- StreamCompleteData *data = opaque; + BlockDriverState *bs = blk_bs(bjob->blk); + BlockDriverState *base = s->base; + Error *local_err = NULL; ++ int ret = job->ret; + +- if (!job_is_cancelled(job) && bs->backing && data->ret == 0) { ++ if (!job_is_cancelled(job) && bs->backing && ret == 0) { + const char *base_id = NULL, *base_fmt = NULL; + if (base) { + base_id = s->backing_file_str; +@@ -75,11 +71,11 @@ static void stream_complete(Job *job, void *opaque) + base_fmt = base->drv->format_name; + } + } +- data->ret = bdrv_change_backing_file(bs, base_id, base_fmt); ++ ret = bdrv_change_backing_file(bs, base_id, base_fmt); + bdrv_set_backing_hd(bs, base, &local_err); + if (local_err) { + error_report_err(local_err); +- data->ret = -EPERM; ++ ret = -EPERM; + goto out; + } + } +@@ -93,14 +89,12 @@ out: + } + + g_free(s->backing_file_str); +- job_completed(job, data->ret); +- g_free(data); ++ job->ret = ret; + } + + static int coroutine_fn stream_run(Job *job, Error **errp) + { + StreamBlockJob *s = container_of(job, StreamBlockJob, common.job); +- StreamCompleteData *data; + BlockBackend *blk = s->common.blk; + BlockDriverState *bs = blk_bs(blk); + BlockDriverState *base = s->base; +@@ -203,9 +197,6 @@ static int coroutine_fn stream_run(Job *job, Error **errp) + + out: + /* Modify backing chain and close BDSes in main loop */ +- data = g_malloc(sizeof(*data)); +- data->ret = ret; +- job_defer_to_main_loop(&s->common.job, stream_complete, data); + return ret; + } + +@@ -215,6 +206,7 @@ static const BlockJobDriver stream_job_driver = { + .job_type = JOB_TYPE_STREAM, + .free = block_job_free, + .run = stream_run, ++ .exit = stream_exit, + .user_resume = block_job_user_resume, + .drain = block_job_drain, + }, +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 8937894..403446e 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -496,11 +496,6 @@ typedef struct TestBlockJob { + bool should_complete; + } TestBlockJob; + +-static void test_job_completed(Job *job, void *opaque) +-{ +- job_completed(job, 0); +-} +- + static int coroutine_fn test_job_run(Job *job, Error **errp) + { + TestBlockJob *s = container_of(job, TestBlockJob, common.job); +@@ -510,7 +505,6 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) + job_sleep_ns(&s->common.job, 100000); + } + +- job_defer_to_main_loop(&s->common.job, test_job_completed, NULL); + return 0; + } + +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index 82cedee..ef29f35 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -24,17 +24,11 @@ typedef struct { + int *result; + } TestBlockJob; + +-static void test_block_job_complete(Job *job, void *opaque) ++static void test_block_job_exit(Job *job) + { + BlockJob *bjob = container_of(job, BlockJob, job); + BlockDriverState *bs = blk_bs(bjob->blk); +- int rc = (intptr_t)opaque; + +- if (job_is_cancelled(job)) { +- rc = -ECANCELED; +- } +- +- job_completed(job, rc); + bdrv_unref(bs); + } + +@@ -54,8 +48,6 @@ static int coroutine_fn test_block_job_run(Job *job, Error **errp) + } + } + +- job_defer_to_main_loop(job, test_block_job_complete, +- (void *)(intptr_t)s->rc); + return s->rc; + } + +@@ -81,6 +73,7 @@ static const BlockJobDriver test_block_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = test_block_job_run, ++ .exit = test_block_job_exit, + }, + }; + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 408a226..ad4a65b 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -163,11 +163,10 @@ typedef struct CancelJob { + bool completed; + } CancelJob; + +-static void cancel_job_completed(Job *job, void *opaque) ++static void cancel_job_exit(Job *job) + { +- CancelJob *s = opaque; ++ CancelJob *s = container_of(job, CancelJob, common.job); + s->completed = true; +- job_completed(job, 0); + } + + static void cancel_job_complete(Job *job, Error **errp) +@@ -182,7 +181,7 @@ static int coroutine_fn cancel_job_run(Job *job, Error **errp) + + while (!s->should_complete) { + if (job_is_cancelled(&s->common.job)) { +- goto defer; ++ return 0; + } + + if (!job_is_ready(&s->common.job) && s->should_converge) { +@@ -192,8 +191,6 @@ static int coroutine_fn cancel_job_run(Job *job, Error **errp) + job_sleep_ns(&s->common.job, 100000); + } + +- defer: +- job_defer_to_main_loop(&s->common.job, cancel_job_completed, s); + return 0; + } + +@@ -204,6 +201,7 @@ static const BlockJobDriver test_cancel_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = cancel_job_run, ++ .exit = cancel_job_exit, + .complete = cancel_job_complete, + }, + }; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-libvhost-user-support-host-notifier.patch b/SOURCES/kvm-libvhost-user-support-host-notifier.patch new file mode 100644 index 0000000..c40664c --- /dev/null +++ b/SOURCES/kvm-libvhost-user-support-host-notifier.patch @@ -0,0 +1,239 @@ +From 672f445642c474d54500ca8d080d64d8a8d4dbd9 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:45 +0200 +Subject: [PATCH 166/268] libvhost-user: support host notifier + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-11-git-send-email-plai@redhat.com> +Patchwork-id: 80939 +O-Subject: [RHEL7.6 PATCH BZ 1526645 10/10] libvhost-user: support host notifier +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +This patch introduces the host notifier support in +libvhost-user. A new API is added to support setting +host notifier for each queue. + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit d84599f56c820d8c1ac9928a76500dcdfbbf194d) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + contrib/libvhost-user/libvhost-user.c | 81 +++++++++++++++++++++++++++++++---- + contrib/libvhost-user/libvhost-user.h | 32 ++++++++++++++ + 2 files changed, 105 insertions(+), 8 deletions(-) + +diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost-user.c +index beeed0c..fa06eee 100644 +--- a/contrib/libvhost-user/libvhost-user.c ++++ b/contrib/libvhost-user/libvhost-user.c +@@ -314,11 +314,6 @@ vu_message_write(VuDev *dev, int conn_fd, VhostUserMsg *vmsg) + msg.msg_controllen = 0; + } + +- /* Set the version in the flags when sending the reply */ +- vmsg->flags &= ~VHOST_USER_VERSION_MASK; +- vmsg->flags |= VHOST_USER_VERSION; +- vmsg->flags |= VHOST_USER_REPLY_MASK; +- + do { + rc = sendmsg(conn_fd, &msg, 0); + } while (rc < 0 && (errno == EINTR || errno == EAGAIN)); +@@ -339,6 +334,39 @@ vu_message_write(VuDev *dev, int conn_fd, VhostUserMsg *vmsg) + return true; + } + ++static bool ++vu_send_reply(VuDev *dev, int conn_fd, VhostUserMsg *vmsg) ++{ ++ /* Set the version in the flags when sending the reply */ ++ vmsg->flags &= ~VHOST_USER_VERSION_MASK; ++ vmsg->flags |= VHOST_USER_VERSION; ++ vmsg->flags |= VHOST_USER_REPLY_MASK; ++ ++ return vu_message_write(dev, conn_fd, vmsg); ++} ++ ++static bool ++vu_process_message_reply(VuDev *dev, const VhostUserMsg *vmsg) ++{ ++ VhostUserMsg msg_reply; ++ ++ if ((vmsg->flags & VHOST_USER_NEED_REPLY_MASK) == 0) { ++ return true; ++ } ++ ++ if (!vu_message_read(dev, dev->slave_fd, &msg_reply)) { ++ return false; ++ } ++ ++ if (msg_reply.request != vmsg->request) { ++ DPRINT("Received unexpected msg type. Expected %d received %d", ++ vmsg->request, msg_reply.request); ++ return false; ++ } ++ ++ return msg_reply.payload.u64 == 0; ++} ++ + /* Kick the log_call_fd if required. */ + static void + vu_log_kick(VuDev *dev) +@@ -534,7 +562,7 @@ vu_set_mem_table_exec_postcopy(VuDev *dev, VhostUserMsg *vmsg) + + /* Send the message back to qemu with the addresses filled in */ + vmsg->fd_num = 0; +- if (!vu_message_write(dev, dev->sock, vmsg)) { ++ if (!vu_send_reply(dev, dev->sock, vmsg)) { + vu_panic(dev, "failed to respond to set-mem-table for postcopy"); + return false; + } +@@ -914,6 +942,41 @@ void vu_set_queue_handler(VuDev *dev, VuVirtq *vq, + } + } + ++bool vu_set_queue_host_notifier(VuDev *dev, VuVirtq *vq, int fd, ++ int size, int offset) ++{ ++ int qidx = vq - dev->vq; ++ int fd_num = 0; ++ VhostUserMsg vmsg = { ++ .request = VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG, ++ .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY_MASK, ++ .size = sizeof(vmsg.payload.area), ++ .payload.area = { ++ .u64 = qidx & VHOST_USER_VRING_IDX_MASK, ++ .size = size, ++ .offset = offset, ++ }, ++ }; ++ ++ if (fd == -1) { ++ vmsg.payload.area.u64 |= VHOST_USER_VRING_NOFD_MASK; ++ } else { ++ vmsg.fds[fd_num++] = fd; ++ } ++ ++ vmsg.fd_num = fd_num; ++ ++ if ((dev->protocol_features & VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD) == 0) { ++ return false; ++ } ++ ++ if (!vu_message_write(dev, dev->slave_fd, &vmsg)) { ++ return false; ++ } ++ ++ return vu_process_message_reply(dev, &vmsg); ++} ++ + static bool + vu_set_vring_call_exec(VuDev *dev, VhostUserMsg *vmsg) + { +@@ -966,7 +1029,9 @@ static bool + vu_get_protocol_features_exec(VuDev *dev, VhostUserMsg *vmsg) + { + uint64_t features = 1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD | +- 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ; ++ 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_REQ | ++ 1ULL << VHOST_USER_PROTOCOL_F_HOST_NOTIFIER | ++ 1ULL << VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD; + + if (have_userfault()) { + features |= 1ULL << VHOST_USER_PROTOCOL_F_PAGEFAULT; +@@ -1250,7 +1315,7 @@ vu_dispatch(VuDev *dev) + goto end; + } + +- if (!vu_message_write(dev, dev->sock, &vmsg)) { ++ if (!vu_send_reply(dev, dev->sock, &vmsg)) { + goto end; + } + +diff --git a/contrib/libvhost-user/libvhost-user.h b/contrib/libvhost-user/libvhost-user.h +index b27075e..4aa55b4 100644 +--- a/contrib/libvhost-user/libvhost-user.h ++++ b/contrib/libvhost-user/libvhost-user.h +@@ -51,6 +51,8 @@ enum VhostUserProtocolFeature { + VHOST_USER_PROTOCOL_F_CRYPTO_SESSION = 7, + VHOST_USER_PROTOCOL_F_PAGEFAULT = 8, + VHOST_USER_PROTOCOL_F_CONFIG = 9, ++ VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD = 10, ++ VHOST_USER_PROTOCOL_F_HOST_NOTIFIER = 11, + + VHOST_USER_PROTOCOL_F_MAX + }; +@@ -92,6 +94,14 @@ typedef enum VhostUserRequest { + VHOST_USER_MAX + } VhostUserRequest; + ++typedef enum VhostUserSlaveRequest { ++ VHOST_USER_SLAVE_NONE = 0, ++ VHOST_USER_SLAVE_IOTLB_MSG = 1, ++ VHOST_USER_SLAVE_CONFIG_CHANGE_MSG = 2, ++ VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG = 3, ++ VHOST_USER_SLAVE_MAX ++} VhostUserSlaveRequest; ++ + typedef struct VhostUserMemoryRegion { + uint64_t guest_phys_addr; + uint64_t memory_size; +@@ -122,6 +132,12 @@ static VhostUserConfig c __attribute__ ((unused)); + + sizeof(c.size) \ + + sizeof(c.flags)) + ++typedef struct VhostUserVringArea { ++ uint64_t u64; ++ uint64_t size; ++ uint64_t offset; ++} VhostUserVringArea; ++ + #if defined(_WIN32) + # define VU_PACKED __attribute__((gcc_struct, packed)) + #else +@@ -133,6 +149,7 @@ typedef struct VhostUserMsg { + + #define VHOST_USER_VERSION_MASK (0x3) + #define VHOST_USER_REPLY_MASK (0x1 << 2) ++#define VHOST_USER_NEED_REPLY_MASK (0x1 << 3) + uint32_t flags; + uint32_t size; /* the following payload size */ + +@@ -145,6 +162,7 @@ typedef struct VhostUserMsg { + VhostUserMemory memory; + VhostUserLog log; + VhostUserConfig config; ++ VhostUserVringArea area; + } payload; + + int fds[VHOST_MEMORY_MAX_NREGIONS]; +@@ -368,6 +386,20 @@ VuVirtq *vu_get_queue(VuDev *dev, int qidx); + void vu_set_queue_handler(VuDev *dev, VuVirtq *vq, + vu_queue_handler_cb handler); + ++/** ++ * vu_set_queue_host_notifier: ++ * @dev: a VuDev context ++ * @vq: a VuVirtq queue ++ * @fd: a file descriptor ++ * @size: host page size ++ * @offset: notifier offset in @fd file ++ * ++ * Set queue's host notifier. This function may be called several ++ * times for the same queue. If called with -1 @fd, the notifier ++ * is removed. ++ */ ++bool vu_set_queue_host_notifier(VuDev *dev, VuVirtq *vq, int fd, ++ int size, int offset); + + /** + * vu_queue_set_notification: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-linux-headers-Update-for-nested-KVM-HV-downstream-on.patch b/SOURCES/kvm-linux-headers-Update-for-nested-KVM-HV-downstream-on.patch new file mode 100644 index 0000000..17c6a09 --- /dev/null +++ b/SOURCES/kvm-linux-headers-Update-for-nested-KVM-HV-downstream-on.patch @@ -0,0 +1,55 @@ +From d514010d4b0abc4eeae499f06c9c9443f85273c7 Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Mon, 12 Nov 2018 01:28:33 +0000 +Subject: [PATCH 02/16] linux-headers: Update for nested KVM-HV [downstream + only] + +RH-Author: David Gibson +Message-id: <20181112012835.21863-3-dgibson@redhat.com> +Patchwork-id: 82977 +O-Subject: [RHEL-8 qemu-kvm PATCH 2/4] linux-headers: Update for nested KVM-HV [downstream only] +Bugzilla: 1639069 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych +RH-Acked-by: Thomas Huth + +Update for the necessary pieces for nested KVM HV on POWER. Not a full +header update, just selected pieces, since that seems to be the norm +downstream. + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1639069 + +Signed-off-by: David Gibson +Signed-off-by: Danilo C. L. de Paula +--- + linux-headers/asm-powerpc/kvm.h | 1 + + linux-headers/linux/kvm.h | 1 + + 2 files changed, 2 insertions(+) + +diff --git a/linux-headers/asm-powerpc/kvm.h b/linux-headers/asm-powerpc/kvm.h +index 833ed9a..83a50df 100644 +--- a/linux-headers/asm-powerpc/kvm.h ++++ b/linux-headers/asm-powerpc/kvm.h +@@ -633,6 +633,7 @@ struct kvm_ppc_cpu_char { + #define KVM_REG_PPC_PSSCR (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xbd) + + #define KVM_REG_PPC_DEC_EXPIRY (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xbe) ++#define KVM_REG_PPC_PTCR (KVM_REG_PPC | KVM_REG_SIZE_U64 | 0xc0) + + /* Transactional Memory checkpointed state: + * This is all GPRs, all VSX regs and a subset of SPRs +diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h +index 9d2e44d..8be1232 100644 +--- a/linux-headers/linux/kvm.h ++++ b/linux-headers/linux/kvm.h +@@ -940,6 +940,7 @@ struct kvm_ppc_resize_hpt { + #define KVM_CAP_NESTED_STATE 157 + #define KVM_CAP_ARM_INJECT_SERROR_ESR 158 + #define KVM_CAP_MSR_PLATFORM_INFO 159 ++#define KVM_CAP_PPC_NESTED_HV 160 + + #ifdef KVM_CAP_IRQ_ROUTING + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-linux-headers-Update-to-include-KVM_CAP_S390_HPAGE_1.patch b/SOURCES/kvm-linux-headers-Update-to-include-KVM_CAP_S390_HPAGE_1.patch new file mode 100644 index 0000000..0d3dcc5 --- /dev/null +++ b/SOURCES/kvm-linux-headers-Update-to-include-KVM_CAP_S390_HPAGE_1.patch @@ -0,0 +1,42 @@ +From 961a7b811a649be9b48c2d061042d690ada7561a Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Mon, 6 Aug 2018 14:18:40 +0100 +Subject: [PATCH 1/3] linux-headers: Update to include KVM_CAP_S390_HPAGE_1M + +RH-Author: David Hildenbrand +Message-id: <20180806141842.23963-2-david@redhat.com> +Patchwork-id: 81643 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 1/3] linux-headers: Update to include KVM_CAP_S390_HPAGE_1M +Bugzilla: 1610906 +RH-Acked-by: Thomas Huth +RH-Acked-by: Cornelia Huck +RH-Acked-by: Paolo Bonzini + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1610906 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=17624600 +Upstream: N/A + +Kernel part is in kvm/next, scheduled for 4.19. No full header sync, only +import the new capability. + +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + linux-headers/linux/kvm.h | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h +index a167be8..454a0f7 100644 +--- a/linux-headers/linux/kvm.h ++++ b/linux-headers/linux/kvm.h +@@ -936,6 +936,7 @@ struct kvm_ppc_resize_hpt { + #define KVM_CAP_PPC_GET_CPU_CHAR 151 + #define KVM_CAP_S390_BPB 152 + #define KVM_CAP_GET_MSR_FEATURES 153 ++#define KVM_CAP_S390_HPAGE_1M 156 + + #ifdef KVM_CAP_IRQ_ROUTING + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-linux-headers-asm-s390-kvm.h-header-sync.patch b/SOURCES/kvm-linux-headers-asm-s390-kvm.h-header-sync.patch new file mode 100644 index 0000000..413e87a --- /dev/null +++ b/SOURCES/kvm-linux-headers-asm-s390-kvm.h-header-sync.patch @@ -0,0 +1,56 @@ +From 694b95dcb7fc97118fc05c885a7561b0702bf6bd Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Thu, 9 Aug 2018 10:15:08 +0000 +Subject: [PATCH 20/21] linux-headers: asm-s390/kvm.h header sync + +RH-Author: Thomas Huth +Message-id: <1533813309-9643-2-git-send-email-thuth@redhat.com> +Patchwork-id: 81688 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 1/2] linux-headers: asm-s390/kvm.h header sync +Bugzilla: 1612938 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +This is a header sync with the linux uapi header. The corresponding +kernel commit id is a3da7b4a3be51f37f434f14e11e60491f098b6ea (in +the kvm/next branch) + +Signed-off-by: Thomas Huth +--- + linux-headers/asm-s390/kvm.h | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h +index 11def14..1ab9901 100644 +--- a/linux-headers/asm-s390/kvm.h ++++ b/linux-headers/asm-s390/kvm.h +@@ -4,7 +4,7 @@ + /* + * KVM s390 specific structures and definitions + * +- * Copyright IBM Corp. 2008 ++ * Copyright IBM Corp. 2008, 2018 + * + * Author(s): Carsten Otte + * Christian Borntraeger +@@ -225,6 +225,7 @@ struct kvm_guest_debug_arch { + #define KVM_SYNC_FPRS (1UL << 8) + #define KVM_SYNC_GSCB (1UL << 9) + #define KVM_SYNC_BPBC (1UL << 10) ++#define KVM_SYNC_ETOKEN (1UL << 11) + /* length and alignment of the sdnx as a power of two */ + #define SDNXC 8 + #define SDNXL (1UL << SDNXC) +@@ -258,6 +259,8 @@ struct kvm_sync_regs { + struct { + __u64 reserved1[2]; + __u64 gscb[4]; ++ __u64 etoken; ++ __u64 etoken_extension; + }; + }; + }; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-linux-headers-update.patch b/SOURCES/kvm-linux-headers-update.patch new file mode 100644 index 0000000..62af815 --- /dev/null +++ b/SOURCES/kvm-linux-headers-update.patch @@ -0,0 +1,202 @@ +From be13671e5df3d44fe0e29736e625a53f4d451d9f Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Mon, 15 Oct 2018 10:19:26 +0100 +Subject: [PATCH 1/6] linux-headers: update + +RH-Author: Thomas Huth +Message-id: <1539598771-16223-2-git-send-email-thuth@redhat.com> +Patchwork-id: 82696 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/6] linux-headers: update +Bugzilla: 1508142 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Cornelia Huck + +Update to kvm/next commit dd5bd0a65ff6 ("Merge tag 'kvm-s390-next-4.20-1' +of git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux into HEAD") + +Signed-off-by: Cornelia Huck +(cherry picked from commit 8f3cd250a897213d39e621e3d824507b48158d42) +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + linux-headers/linux/kvm.h + linux-headers/linux/vhost.h + (simple contextual conflicts due to some missing patches in downstream) + +Signed-off-by: Thomas Huth +--- + include/standard-headers/linux/input.h | 9 +++++---- + linux-headers/asm-arm/kvm.h | 13 +++++++++++++ + linux-headers/asm-arm64/kvm.h | 13 +++++++++++++ + linux-headers/asm-s390/kvm.h | 2 ++ + linux-headers/asm-x86/kvm.h | 1 + + linux-headers/linux/kvm.h | 3 +++ + linux-headers/linux/vfio.h | 2 ++ + linux-headers/linux/vhost.h | 8 ++++++++ + 8 files changed, 47 insertions(+), 4 deletions(-) + +diff --git a/include/standard-headers/linux/input.h b/include/standard-headers/linux/input.h +index 939b627..f791b1b 100644 +--- a/include/standard-headers/linux/input.h ++++ b/include/standard-headers/linux/input.h +@@ -267,10 +267,11 @@ struct input_mask { + /* + * MT_TOOL types + */ +-#define MT_TOOL_FINGER 0 +-#define MT_TOOL_PEN 1 +-#define MT_TOOL_PALM 2 +-#define MT_TOOL_MAX 2 ++#define MT_TOOL_FINGER 0x00 ++#define MT_TOOL_PEN 0x01 ++#define MT_TOOL_PALM 0x02 ++#define MT_TOOL_DIAL 0x0a ++#define MT_TOOL_MAX 0x0f + + /* + * Values describing the status of a force-feedback effect +diff --git a/linux-headers/asm-arm/kvm.h b/linux-headers/asm-arm/kvm.h +index 4392955..3097290 100644 +--- a/linux-headers/asm-arm/kvm.h ++++ b/linux-headers/asm-arm/kvm.h +@@ -27,6 +27,7 @@ + #define __KVM_HAVE_GUEST_DEBUG + #define __KVM_HAVE_IRQ_LINE + #define __KVM_HAVE_READONLY_MEM ++#define __KVM_HAVE_VCPU_EVENTS + + #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 + +@@ -124,6 +125,18 @@ struct kvm_sync_regs { + struct kvm_arch_memory_slot { + }; + ++/* for KVM_GET/SET_VCPU_EVENTS */ ++struct kvm_vcpu_events { ++ struct { ++ __u8 serror_pending; ++ __u8 serror_has_esr; ++ /* Align it to 8 bytes */ ++ __u8 pad[6]; ++ __u64 serror_esr; ++ } exception; ++ __u32 reserved[12]; ++}; ++ + /* If you need to interpret the index values, here is the key: */ + #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 + #define KVM_REG_ARM_COPROC_SHIFT 16 +diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h +index 4e80651..208de22 100644 +--- a/linux-headers/asm-arm64/kvm.h ++++ b/linux-headers/asm-arm64/kvm.h +@@ -39,6 +39,7 @@ + #define __KVM_HAVE_GUEST_DEBUG + #define __KVM_HAVE_IRQ_LINE + #define __KVM_HAVE_READONLY_MEM ++#define __KVM_HAVE_VCPU_EVENTS + + #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 + +@@ -153,6 +154,18 @@ struct kvm_sync_regs { + struct kvm_arch_memory_slot { + }; + ++/* for KVM_GET/SET_VCPU_EVENTS */ ++struct kvm_vcpu_events { ++ struct { ++ __u8 serror_pending; ++ __u8 serror_has_esr; ++ /* Align it to 8 bytes */ ++ __u8 pad[6]; ++ __u64 serror_esr; ++ } exception; ++ __u32 reserved[12]; ++}; ++ + /* If you need to interpret the index values, here is the key: */ + #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 + #define KVM_REG_ARM_COPROC_SHIFT 16 +diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h +index 1ab9901..0265482 100644 +--- a/linux-headers/asm-s390/kvm.h ++++ b/linux-headers/asm-s390/kvm.h +@@ -160,6 +160,8 @@ struct kvm_s390_vm_cpu_subfunc { + #define KVM_S390_VM_CRYPTO_ENABLE_DEA_KW 1 + #define KVM_S390_VM_CRYPTO_DISABLE_AES_KW 2 + #define KVM_S390_VM_CRYPTO_DISABLE_DEA_KW 3 ++#define KVM_S390_VM_CRYPTO_ENABLE_APIE 4 ++#define KVM_S390_VM_CRYPTO_DISABLE_APIE 5 + + /* kvm attributes for migration mode */ + #define KVM_S390_VM_MIGRATION_STOP 0 +diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h +index f3a9604..dcf4dc9 100644 +--- a/linux-headers/asm-x86/kvm.h ++++ b/linux-headers/asm-x86/kvm.h +@@ -360,5 +360,6 @@ struct kvm_sync_regs { + + #define KVM_X86_QUIRK_LINT0_REENABLED (1 << 0) + #define KVM_X86_QUIRK_CD_NW_CLEARED (1 << 1) ++#define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2) + + #endif /* _ASM_X86_KVM_H */ +diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h +index 454a0f7..9d2e44d 100644 +--- a/linux-headers/linux/kvm.h ++++ b/linux-headers/linux/kvm.h +@@ -937,6 +937,9 @@ struct kvm_ppc_resize_hpt { + #define KVM_CAP_S390_BPB 152 + #define KVM_CAP_GET_MSR_FEATURES 153 + #define KVM_CAP_S390_HPAGE_1M 156 ++#define KVM_CAP_NESTED_STATE 157 ++#define KVM_CAP_ARM_INJECT_SERROR_ESR 158 ++#define KVM_CAP_MSR_PLATFORM_INFO 159 + + #ifdef KVM_CAP_IRQ_ROUTING + +diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h +index 3a0a305..25c7b7d 100644 +--- a/linux-headers/linux/vfio.h ++++ b/linux-headers/linux/vfio.h +@@ -200,6 +200,7 @@ struct vfio_device_info { + #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2) /* vfio-platform device */ + #define VFIO_DEVICE_FLAGS_AMBA (1 << 3) /* vfio-amba device */ + #define VFIO_DEVICE_FLAGS_CCW (1 << 4) /* vfio-ccw device */ ++#define VFIO_DEVICE_FLAGS_AP (1 << 5) /* vfio-ap device */ + __u32 num_regions; /* Max region index + 1 */ + __u32 num_irqs; /* Max IRQ index + 1 */ + }; +@@ -215,6 +216,7 @@ struct vfio_device_info { + #define VFIO_DEVICE_API_PLATFORM_STRING "vfio-platform" + #define VFIO_DEVICE_API_AMBA_STRING "vfio-amba" + #define VFIO_DEVICE_API_CCW_STRING "vfio-ccw" ++#define VFIO_DEVICE_API_AP_STRING "vfio-ap" + + /** + * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8, +diff --git a/linux-headers/linux/vhost.h b/linux-headers/linux/vhost.h +index e336395..3421624 100644 +--- a/linux-headers/linux/vhost.h ++++ b/linux-headers/linux/vhost.h +@@ -160,6 +160,14 @@ struct vhost_memory { + #define VHOST_GET_VRING_BUSYLOOP_TIMEOUT _IOW(VHOST_VIRTIO, 0x24, \ + struct vhost_vring_state) + ++/* Set or get vhost backend capability */ ++ ++/* Use message type V2 */ ++#define VHOST_BACKEND_F_IOTLB_MSG_V2 0x1 ++ ++#define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64) ++#define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64) ++ + /* VHOST_NET specific defines */ + + /* Attach virtio net ring to a raw socket, or tap device. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-luks-Allow-share-rw-on.patch b/SOURCES/kvm-luks-Allow-share-rw-on.patch new file mode 100644 index 0000000..29d9e60 --- /dev/null +++ b/SOURCES/kvm-luks-Allow-share-rw-on.patch @@ -0,0 +1,52 @@ +From f864b6b5b9dd5bc7c853c002657d8497d285cee4 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 28 Sep 2018 06:09:52 +0100 +Subject: [PATCH 1/3] luks: Allow share-rw=on +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Fam Zheng +Message-id: <20180928060952.8616-2-famz@redhat.com> +Patchwork-id: 82311 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] luks: Allow share-rw=on +Bugzilla: 1629701 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Format drivers such as qcow2 don't allow sharing the same image between +two QEMU instances in order to prevent image corruptions, because of +metadata cache. LUKS driver don't modify metadata except for when +creating image, so it is safe to relax the permission. This makes +share-rw=on property work on virtual devices. + +Suggested-by: Daniel P. Berrangé +Signed-off-by: Fam Zheng +Reviewed-by: Daniel P. Berrangé +Signed-off-by: Kevin Wolf +(cherry picked from commit 497da8236ab2663a8108858ba7ea59aac21c5fe6) +Signed-off-by: Fam Zheng +Signed-off-by: Danilo C. L. de Paula +--- + block/crypto.c | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/block/crypto.c b/block/crypto.c +index 02f04f3..0bb0db6 100644 +--- a/block/crypto.c ++++ b/block/crypto.c +@@ -698,7 +698,9 @@ BlockDriver bdrv_crypto_luks = { + .bdrv_probe = block_crypto_probe_luks, + .bdrv_open = block_crypto_open_luks, + .bdrv_close = block_crypto_close, +- .bdrv_child_perm = bdrv_format_default_perms, ++ /* This driver doesn't modify LUKS metadata except when creating image. ++ * Allow share-rw=on as a special case. */ ++ .bdrv_child_perm = bdrv_filter_default_perms, + .bdrv_co_create = block_crypto_co_create_luks, + .bdrv_co_create_opts = block_crypto_co_create_opts_luks, + .bdrv_co_truncate = block_crypto_co_truncate, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-main-loop-drop-spin_counter.patch b/SOURCES/kvm-main-loop-drop-spin_counter.patch new file mode 100644 index 0000000..05ed1ee --- /dev/null +++ b/SOURCES/kvm-main-loop-drop-spin_counter.patch @@ -0,0 +1,106 @@ +From 473673fd0f834adf72f80caddab0d942a99dfe03 Mon Sep 17 00:00:00 2001 +From: Jeffrey Cody +Date: Wed, 20 Jun 2018 17:44:46 +0200 +Subject: [PATCH 059/268] main-loop: drop spin_counter + +RH-Author: Jeffrey Cody +Message-id: <835200bb37bc622f0f6f9c471c809a1b73a08482.1529516334.git.jcody@redhat.com> +Patchwork-id: 80906 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 1/1] main-loop: drop spin_counter +Bugzilla: 1168213 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Markus Armbruster + +From: Stefan Hajnoczi + +Commit d759c951f3287fad04210a52f2dc93f94cf58c7f ("replay: push +replay_mutex_lock up the call tree") removed the !timeout lock +optimization in the main loop. + +The idea of the optimization was to avoid ping-pongs between threads by +keeping the Big QEMU Lock held across non-blocking (!timeout) main loop +iterations. + +A warning is printed when the main loop spins without releasing BQL for +long periods of time. These warnings were supposed to aid debugging but +in practice they just alarm users. They are considered noise because +the cause of spinning is not shown and is hard to find. + +Now that the lock optimization has been removed, there is no danger of +hogging the BQL. Drop the spin counter and the infamous warning. + +Signed-off-by: Stefan Hajnoczi +Reviewed-by: Jeff Cody +(cherry picked from commit 21891a5a3011608845b5d7f1f9cce60cdc2bcc62) +Note: The original patch series from Stefan makes note that the + patch can be dropped because the lock optimization changed + upstream. + + However, even without that optimization change upstream, this + warning / debug message is not very useful, and is just noise that + does not aid in debugging. Go ahead and remove it anyway. +Signed-off-by: Jeff Cody + +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/common.filter | 1 - + util/main-loop.c | 25 ------------------------- + 2 files changed, 26 deletions(-) + +diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter +index c5f4bcf..7acb454 100644 +--- a/tests/qemu-iotests/common.filter ++++ b/tests/qemu-iotests/common.filter +@@ -77,7 +77,6 @@ _filter_qemu() + { + sed -e "s#\\(^\\|(qemu) \\)$(basename $QEMU_PROG):#\1QEMU_PROG:#" \ + -e 's#^QEMU [0-9]\+\.[0-9]\+\.[0-9]\+ monitor#QEMU X.Y.Z monitor#' \ +- -e '/main-loop: WARNING: I\/O thread spun for [0-9]\+ iterations/d' \ + -e $'s#\r##' # QEMU monitor uses \r\n line endings + } + +diff --git a/util/main-loop.c b/util/main-loop.c +index 992f9b0..affe040 100644 +--- a/util/main-loop.c ++++ b/util/main-loop.c +@@ -222,36 +222,11 @@ static int os_host_main_loop_wait(int64_t timeout) + { + GMainContext *context = g_main_context_default(); + int ret; +- static int spin_counter; + + g_main_context_acquire(context); + + glib_pollfds_fill(&timeout); + +- /* If the I/O thread is very busy or we are incorrectly busy waiting in +- * the I/O thread, this can lead to starvation of the BQL such that the +- * VCPU threads never run. To make sure we can detect the later case, +- * print a message to the screen. If we run into this condition, create +- * a fake timeout in order to give the VCPU threads a chance to run. +- */ +- if (!timeout && (spin_counter > MAX_MAIN_LOOP_SPIN)) { +- static bool notified; +- +- if (!notified && !qtest_enabled() && !qtest_driver()) { +- warn_report("I/O thread spun for %d iterations", +- MAX_MAIN_LOOP_SPIN); +- notified = true; +- } +- +- timeout = SCALE_MS; +- } +- +- +- if (timeout) { +- spin_counter = 0; +- } else { +- spin_counter++; +- } + qemu_mutex_unlock_iothread(); + replay_mutex_unlock(); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-mem-nvdimm-ensure-write-persistence-to-PMEM-in-label.patch b/SOURCES/kvm-mem-nvdimm-ensure-write-persistence-to-PMEM-in-label.patch new file mode 100644 index 0000000..44c3860 --- /dev/null +++ b/SOURCES/kvm-mem-nvdimm-ensure-write-persistence-to-PMEM-in-label.patch @@ -0,0 +1,108 @@ +From 3fe466338a4ab4a119933768462d62c6d7b9024e Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:20 +0000 +Subject: [PATCH 19/22] mem/nvdimm: ensure write persistence to PMEM in label + emulation + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-8-git-send-email-plai@redhat.com> +Patchwork-id: 83893 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 07/10] mem/nvdimm: ensure write persistence to PMEM in label emulation +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +Guest writes to vNVDIMM labels are intercepted and performed on the +backend by QEMU. When the backend is a real persistent memort, QEMU +needs to take proper operations to ensure its write persistence on the +persistent memory. Otherwise, a host power failure may result in the +loss of guest label configurations. + +Signed-off-by: Haozhong Zhang +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +Reviewed-by: Richard Henderson +(cherry picked from commit faf8a13d80de98b43342a7ec9878b4fd76b18327) +Signed-off-by: Paul Lai +Signed-off-by: Danilo C. L. de Paula +--- + hw/mem/nvdimm.c | 9 ++++++++- + include/qemu/pmem.h | 30 ++++++++++++++++++++++++++++++ + 2 files changed, 38 insertions(+), 1 deletion(-) + create mode 100644 include/qemu/pmem.h + +diff --git a/hw/mem/nvdimm.c b/hw/mem/nvdimm.c +index acb656b..0c962fd 100644 +--- a/hw/mem/nvdimm.c ++++ b/hw/mem/nvdimm.c +@@ -23,6 +23,7 @@ + */ + + #include "qemu/osdep.h" ++#include "qemu/pmem.h" + #include "qapi/error.h" + #include "qapi/visitor.h" + #include "hw/mem/nvdimm.h" +@@ -155,11 +156,17 @@ static void nvdimm_write_label_data(NVDIMMDevice *nvdimm, const void *buf, + { + MemoryRegion *mr; + PCDIMMDevice *dimm = PC_DIMM(nvdimm); ++ bool is_pmem = object_property_get_bool(OBJECT(dimm->hostmem), ++ "pmem", NULL); + uint64_t backend_offset; + + nvdimm_validate_rw_label_data(nvdimm, size, offset); + +- memcpy(nvdimm->label_data + offset, buf, size); ++ if (!is_pmem) { ++ memcpy(nvdimm->label_data + offset, buf, size); ++ } else { ++ pmem_memcpy_persist(nvdimm->label_data + offset, buf, size); ++ } + + mr = host_memory_backend_get_memory(dimm->hostmem, &error_abort); + backend_offset = memory_region_size(mr) - nvdimm->label_size + offset; +diff --git a/include/qemu/pmem.h b/include/qemu/pmem.h +new file mode 100644 +index 0000000..ebdb070 +--- /dev/null ++++ b/include/qemu/pmem.h +@@ -0,0 +1,30 @@ ++/* ++ * QEMU header file for libpmem. ++ * ++ * Copyright (c) 2018 Intel Corporation. ++ * ++ * Author: Haozhong Zhang ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#ifndef QEMU_PMEM_H ++#define QEMU_PMEM_H ++ ++#ifdef CONFIG_LIBPMEM ++#include ++#else /* !CONFIG_LIBPMEM */ ++ ++static inline void * ++pmem_memcpy_persist(void *pmemdest, const void *src, size_t len) ++{ ++ /* If 'pmem' option is 'on', we should always have libpmem support, ++ or qemu will report a error and exit, never come here. */ ++ g_assert_not_reached(); ++ return NULL; ++} ++ ++#endif /* CONFIG_LIBPMEM */ ++ ++#endif /* !QEMU_PMEM_H */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-memory-cleanup-side-effects-of-memory_region_init_fo.patch b/SOURCES/kvm-memory-cleanup-side-effects-of-memory_region_init_fo.patch new file mode 100644 index 0000000..fae56ec --- /dev/null +++ b/SOURCES/kvm-memory-cleanup-side-effects-of-memory_region_init_fo.patch @@ -0,0 +1,186 @@ +From cfa472eb467f8e5139a2a7d30aa85c9affbccd4f Mon Sep 17 00:00:00 2001 +From: Igor Mammedov +Date: Fri, 5 Oct 2018 12:59:47 +0100 +Subject: [PATCH 6/6] memory: cleanup side effects of memory_region_init_foo() + on failure + +RH-Author: Igor Mammedov +Message-id: <1538744387-84898-1-git-send-email-imammedo@redhat.com> +Patchwork-id: 82391 +O-Subject: [RHEL-8 qemu-kvm PATCH] memory: cleanup side effects of memory_region_init_foo() on failure +Bugzilla: 1600365 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Laszlo Ersek + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1600365 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18658506 + +if MemoryRegion intialization fails it's left in semi-initialized state, +where it's size is not 0 and attached as child to owner object. +And this leds to crash in following use-case: + (monitor) object_add memory-backend-file,id=mem1,size=99999G,mem-path=/tmp/foo,discard-data=yes + memory.c:2083: memory_region_get_ram_ptr: Assertion `mr->ram_block' failed + Aborted (core dumped) +it happens due to assumption that memory region is intialized when + memory_region_size() != 0 +and therefore it's ok to access it in + file_backend_unparent() + if (memory_region_size() != 0) + memory_region_get_ram_ptr() + +which happens when object_add fails and unparents failed backend making +file_backend_unparent() access invalid memory region. + +Fix it by making sure that memory_region_init_foo() APIs cleanup externally +visible side effects on failure (like set size to 0 and unparenting object) + +Signed-off-by: Igor Mammedov +Message-Id: <1536064777-42312-1-git-send-email-imammedo@redhat.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit 1cd3d492624da399d66c4c3e6a5eabb8f96bb0a2) +Signed-off-by: Igor Mammedov +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + memory.c + due missing (cbfc01710 "memory, exec: switch file ram allocation functions to 'flags' parameters") + not related to the patch signature mismatch of + qemu_ram_alloc_from_file()/qemu_ram_alloc_from_fd() +--- + memory.c | 48 ++++++++++++++++++++++++++++++++++++++++++------ + 1 file changed, 42 insertions(+), 6 deletions(-) + +diff --git a/memory.c b/memory.c +index e70b64b..1a99b9c 100644 +--- a/memory.c ++++ b/memory.c +@@ -1519,12 +1519,18 @@ void memory_region_init_ram_shared_nomigrate(MemoryRegion *mr, + bool share, + Error **errp) + { ++ Error *err = NULL; + memory_region_init(mr, owner, name, size); + mr->ram = true; + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; +- mr->ram_block = qemu_ram_alloc(size, share, mr, errp); ++ mr->ram_block = qemu_ram_alloc(size, share, mr, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; ++ if (err) { ++ mr->size = int128_zero(); ++ object_unparent(OBJECT(mr)); ++ error_propagate(errp, err); ++ } + } + + void memory_region_init_resizeable_ram(MemoryRegion *mr, +@@ -1537,13 +1543,19 @@ void memory_region_init_resizeable_ram(MemoryRegion *mr, + void *host), + Error **errp) + { ++ Error *err = NULL; + memory_region_init(mr, owner, name, size); + mr->ram = true; + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; + mr->ram_block = qemu_ram_alloc_resizeable(size, max_size, resized, +- mr, errp); ++ mr, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; ++ if (err) { ++ mr->size = int128_zero(); ++ object_unparent(OBJECT(mr)); ++ error_propagate(errp, err); ++ } + } + + #ifdef __linux__ +@@ -1556,13 +1568,19 @@ void memory_region_init_ram_from_file(MemoryRegion *mr, + const char *path, + Error **errp) + { ++ Error *err = NULL; + memory_region_init(mr, owner, name, size); + mr->ram = true; + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; + mr->align = align; +- mr->ram_block = qemu_ram_alloc_from_file(size, mr, share, path, errp); ++ mr->ram_block = qemu_ram_alloc_from_file(size, mr, share, path, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; ++ if (err) { ++ mr->size = int128_zero(); ++ object_unparent(OBJECT(mr)); ++ error_propagate(errp, err); ++ } + } + + void memory_region_init_ram_from_fd(MemoryRegion *mr, +@@ -1573,12 +1591,18 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr, + int fd, + Error **errp) + { ++ Error *err = NULL; + memory_region_init(mr, owner, name, size); + mr->ram = true; + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; +- mr->ram_block = qemu_ram_alloc_from_fd(size, mr, share, fd, errp); ++ mr->ram_block = qemu_ram_alloc_from_fd(size, mr, share, fd, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; ++ if (err) { ++ mr->size = int128_zero(); ++ object_unparent(OBJECT(mr)); ++ error_propagate(errp, err); ++ } + } + #endif + +@@ -1629,13 +1653,19 @@ void memory_region_init_rom_nomigrate(MemoryRegion *mr, + uint64_t size, + Error **errp) + { ++ Error *err = NULL; + memory_region_init(mr, owner, name, size); + mr->ram = true; + mr->readonly = true; + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; +- mr->ram_block = qemu_ram_alloc(size, false, mr, errp); ++ mr->ram_block = qemu_ram_alloc(size, false, mr, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; ++ if (err) { ++ mr->size = int128_zero(); ++ object_unparent(OBJECT(mr)); ++ error_propagate(errp, err); ++ } + } + + void memory_region_init_rom_device_nomigrate(MemoryRegion *mr, +@@ -1646,6 +1676,7 @@ void memory_region_init_rom_device_nomigrate(MemoryRegion *mr, + uint64_t size, + Error **errp) + { ++ Error *err = NULL; + assert(ops); + memory_region_init(mr, owner, name, size); + mr->ops = ops; +@@ -1653,7 +1684,12 @@ void memory_region_init_rom_device_nomigrate(MemoryRegion *mr, + mr->terminates = true; + mr->rom_device = true; + mr->destructor = memory_region_destructor_ram; +- mr->ram_block = qemu_ram_alloc(size, false, mr, errp); ++ mr->ram_block = qemu_ram_alloc(size, false, mr, &err); ++ if (err) { ++ mr->size = int128_zero(); ++ object_unparent(OBJECT(mr)); ++ error_propagate(errp, err); ++ } + } + + void memory_region_init_iommu(void *_iommu_mr, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-memory-exec-Expose-all-memory-block-related-flags.patch b/SOURCES/kvm-memory-exec-Expose-all-memory-block-related-flags.patch new file mode 100644 index 0000000..405f418 --- /dev/null +++ b/SOURCES/kvm-memory-exec-Expose-all-memory-block-related-flags.patch @@ -0,0 +1,99 @@ +From 65df9633df636fa14fad0dde4915582f53ec00c3 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:16 +0000 +Subject: [PATCH 15/22] memory, exec: Expose all memory block related flags. + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-4-git-send-email-plai@redhat.com> +Patchwork-id: 83887 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 03/10] memory, exec: Expose all memory block related flags. +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +We need to use these flags in other files rather than just in exec.c, +For example, RAM_SHARED should be used when create a ram block from file. +We expose them the exec/memory.h + +Signed-off-by: Junyan He +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Richard Henderson +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit b0e5de93811077254a536c23b713b49e12efb742) +Signed-off-by: Paul Lai +Signed-off-by: Danilo C. L. de Paula +--- + exec.c | 20 -------------------- + include/exec/memory.h | 20 ++++++++++++++++++++ + 2 files changed, 20 insertions(+), 20 deletions(-) + +diff --git a/exec.c b/exec.c +index fff49ba..b66377c 100644 +--- a/exec.c ++++ b/exec.c +@@ -87,26 +87,6 @@ AddressSpace address_space_memory; + + MemoryRegion io_mem_rom, io_mem_notdirty; + static MemoryRegion io_mem_unassigned; +- +-/* RAM is pre-allocated and passed into qemu_ram_alloc_from_ptr */ +-#define RAM_PREALLOC (1 << 0) +- +-/* RAM is mmap-ed with MAP_SHARED */ +-#define RAM_SHARED (1 << 1) +- +-/* Only a portion of RAM (used_length) is actually used, and migrated. +- * This used_length size can change across reboots. +- */ +-#define RAM_RESIZEABLE (1 << 2) +- +-/* UFFDIO_ZEROPAGE is available on this RAMBlock to atomically +- * zero the page and wake waiting processes. +- * (Set during postcopy) +- */ +-#define RAM_UF_ZEROPAGE (1 << 3) +- +-/* RAM can be migrated */ +-#define RAM_MIGRATABLE (1 << 4) + #endif + + #ifdef TARGET_PAGE_BITS_VARY +diff --git a/include/exec/memory.h b/include/exec/memory.h +index 31eae0a..db46501 100644 +--- a/include/exec/memory.h ++++ b/include/exec/memory.h +@@ -102,6 +102,26 @@ struct IOMMUNotifier { + }; + typedef struct IOMMUNotifier IOMMUNotifier; + ++/* RAM is pre-allocated and passed into qemu_ram_alloc_from_ptr */ ++#define RAM_PREALLOC (1 << 0) ++ ++/* RAM is mmap-ed with MAP_SHARED */ ++#define RAM_SHARED (1 << 1) ++ ++/* Only a portion of RAM (used_length) is actually used, and migrated. ++ * This used_length size can change across reboots. ++ */ ++#define RAM_RESIZEABLE (1 << 2) ++ ++/* UFFDIO_ZEROPAGE is available on this RAMBlock to atomically ++ * zero the page and wake waiting processes. ++ * (Set during postcopy) ++ */ ++#define RAM_UF_ZEROPAGE (1 << 3) ++ ++/* RAM can be migrated */ ++#define RAM_MIGRATABLE (1 << 4) ++ + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, + IOMMUNotifierFlag flags, + hwaddr start, hwaddr end) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-memory-exec-switch-file-ram-allocation-functions-to-.patch b/SOURCES/kvm-memory-exec-switch-file-ram-allocation-functions-to-.patch new file mode 100644 index 0000000..baf591f --- /dev/null +++ b/SOURCES/kvm-memory-exec-switch-file-ram-allocation-functions-to-.patch @@ -0,0 +1,235 @@ +From 6a5885e65ce9daac0464a78c93a10c36f41b28ce Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:17 +0000 +Subject: [PATCH 16/22] memory, exec: switch file ram allocation functions to + 'flags' parameters + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-5-git-send-email-plai@redhat.com> +Patchwork-id: 83889 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 04/10] memory, exec: switch file ram allocation functions to 'flags' parameters +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +As more flag parameters besides the existing 'share' are going to be +added to following functions +memory_region_init_ram_from_file +qemu_ram_alloc_from_fd +qemu_ram_alloc_from_file +let's switch them to use the 'flags' parameters so as to ease future +flag additions. + +The existing 'share' flag is converted to the RAM_SHARED bit in ram_flags, +and other flag bits are ignored by above functions right now. + +Signed-off-by: Junyan He +Signed-off-by: Haozhong Zhang +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +Reviewed-by: Richard Henderson +(cherry picked from commit cbfc01710362f3de6fca3010a17b0e1c866fc181) +Signed-off-by: Paul Lai + +Resovled Conflicts: + include/exec/ram_addr.h + memory.c + +Signed-off-by: Danilo C. L. de Paula +--- + backends/hostmem-file.c | 3 ++- + exec.c | 10 +++++----- + include/exec/memory.h | 7 +++++-- + include/exec/ram_addr.h | 25 +++++++++++++++++++++++-- + memory.c | 8 +++++--- + numa.c | 2 +- + 6 files changed, 41 insertions(+), 14 deletions(-) + +diff --git a/backends/hostmem-file.c b/backends/hostmem-file.c +index 134b08d..34c68bb 100644 +--- a/backends/hostmem-file.c ++++ b/backends/hostmem-file.c +@@ -58,7 +58,8 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp) + path = object_get_canonical_path(OBJECT(backend)); + memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), + path, +- backend->size, fb->align, backend->share, ++ backend->size, fb->align, ++ backend->share ? RAM_SHARED : 0, + fb->mem_path, errp); + g_free(path); + } +diff --git a/exec.c b/exec.c +index b66377c..8d58e8f 100644 +--- a/exec.c ++++ b/exec.c +@@ -2042,7 +2042,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared) + + #ifdef __linux__ + RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, +- bool share, int fd, ++ uint32_t ram_flags, int fd, + Error **errp) + { + RAMBlock *new_block; +@@ -2084,14 +2084,14 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, + new_block->mr = mr; + new_block->used_length = size; + new_block->max_length = size; +- new_block->flags = share ? RAM_SHARED : 0; ++ new_block->flags = ram_flags; + new_block->host = file_ram_alloc(new_block, size, fd, !file_size, errp); + if (!new_block->host) { + g_free(new_block); + return NULL; + } + +- ram_block_add(new_block, &local_err, share); ++ ram_block_add(new_block, &local_err, ram_flags & RAM_SHARED); + if (local_err) { + g_free(new_block); + error_propagate(errp, local_err); +@@ -2103,7 +2103,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, + + + RAMBlock *qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr, +- bool share, const char *mem_path, ++ uint32_t ram_flags, const char *mem_path, + Error **errp) + { + int fd; +@@ -2115,7 +2115,7 @@ RAMBlock *qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr, + return NULL; + } + +- block = qemu_ram_alloc_from_fd(size, mr, share, fd, errp); ++ block = qemu_ram_alloc_from_fd(size, mr, ram_flags, fd, errp); + if (!block) { + if (created) { + unlink(mem_path); +diff --git a/include/exec/memory.h b/include/exec/memory.h +index db46501..b3abe61 100644 +--- a/include/exec/memory.h ++++ b/include/exec/memory.h +@@ -527,6 +527,7 @@ void memory_region_init_resizeable_ram(MemoryRegion *mr, + void *host), + Error **errp); + #ifdef __linux__ ++ + /** + * memory_region_init_ram_from_file: Initialize RAM memory region with a + * mmap-ed backend. +@@ -538,7 +539,9 @@ void memory_region_init_resizeable_ram(MemoryRegion *mr, + * @size: size of the region. + * @align: alignment of the region base address; if 0, the default alignment + * (getpagesize()) will be used. +- * @share: %true if memory must be mmaped with the MAP_SHARED flag ++ * @ram_flags: Memory region features: ++ * - RAM_SHARED: memory must be mmaped with the MAP_SHARED flag ++ * Other bits are ignored now. + * @path: the path in which to allocate the RAM. + * @errp: pointer to Error*, to store an error if it happens. + * +@@ -550,7 +553,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr, + const char *name, + uint64_t size, + uint64_t align, +- bool share, ++ uint32_t ram_flags, + const char *path, + Error **errp); + +diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h +index cf2446a..67e163e 100644 +--- a/include/exec/ram_addr.h ++++ b/include/exec/ram_addr.h +@@ -72,12 +72,33 @@ static inline unsigned long int ramblock_recv_bitmap_offset(void *host_addr, + + long qemu_getrampagesize(void); + unsigned long last_ram_page(void); ++ ++/** ++ * qemu_ram_alloc_from_file, ++ * qemu_ram_alloc_from_fd: Allocate a ram block from the specified backing ++ * file or device ++ * ++ * Parameters: ++ * @size: the size in bytes of the ram block ++ * @mr: the memory region where the ram block is ++ * @ram_flags: specify the properties of the ram block, which can be one ++ * or bit-or of following values ++ * - RAM_SHARED: mmap the backing file or device with MAP_SHARED ++ * Other bits are ignored. ++ * @mem_path or @fd: specify the backing file or device ++ * @errp: pointer to Error*, to store an error if it happens ++ * ++ * Return: ++ * On success, return a pointer to the ram block. ++ * On failure, return NULL. ++ */ + RAMBlock *qemu_ram_alloc_from_file(ram_addr_t size, MemoryRegion *mr, +- bool share, const char *mem_path, ++ uint32_t ram_flags, const char *mem_path, + Error **errp); + RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, +- bool share, int fd, ++ uint32_t ram_flags, int fd, + Error **errp); ++ + RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host, + MemoryRegion *mr, Error **errp); + RAMBlock *qemu_ram_alloc(ram_addr_t size, bool share, MemoryRegion *mr, +diff --git a/memory.c b/memory.c +index 1a99b9c..4974f97 100644 +--- a/memory.c ++++ b/memory.c +@@ -1564,7 +1564,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr, + const char *name, + uint64_t size, + uint64_t align, +- bool share, ++ uint32_t ram_flags, + const char *path, + Error **errp) + { +@@ -1574,7 +1574,7 @@ void memory_region_init_ram_from_file(MemoryRegion *mr, + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; + mr->align = align; +- mr->ram_block = qemu_ram_alloc_from_file(size, mr, share, path, &err); ++ mr->ram_block = qemu_ram_alloc_from_file(size, mr, ram_flags, path, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; + if (err) { + mr->size = int128_zero(); +@@ -1596,7 +1596,9 @@ void memory_region_init_ram_from_fd(MemoryRegion *mr, + mr->ram = true; + mr->terminates = true; + mr->destructor = memory_region_destructor_ram; +- mr->ram_block = qemu_ram_alloc_from_fd(size, mr, share, fd, &err); ++ mr->ram_block = qemu_ram_alloc_from_fd(size, mr, ++ share ? RAM_SHARED : 0, ++ fd, &err); + mr->dirty_log_mask = tcg_enabled() ? (1 << DIRTY_MEMORY_CODE) : 0; + if (err) { + mr->size = int128_zero(); +diff --git a/numa.c b/numa.c +index a767a9d..6ea8a86 100644 +--- a/numa.c ++++ b/numa.c +@@ -456,7 +456,7 @@ static void allocate_system_memory_nonnuma(MemoryRegion *mr, Object *owner, + if (mem_path) { + #ifdef __linux__ + Error *err = NULL; +- memory_region_init_ram_from_file(mr, owner, name, ram_size, 0, false, ++ memory_region_init_ram_from_file(mr, owner, name, ram_size, 0, 0, + mem_path, &err); + if (err) { + error_report_err(err); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-Don-t-activate-block-devices-if-using-S.patch b/SOURCES/kvm-migration-Don-t-activate-block-devices-if-using-S.patch new file mode 100644 index 0000000..414f52c --- /dev/null +++ b/SOURCES/kvm-migration-Don-t-activate-block-devices-if-using-S.patch @@ -0,0 +1,113 @@ +From 1669a71fd8a4bd570f09fc2df67279d6936cc5ad Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Fri, 22 Jun 2018 19:00:03 +0200 +Subject: [PATCH 15/21] migration: Don't activate block devices if using -S + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-17-dgilbert@redhat.com> +Patchwork-id: 81576 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 16/18] migration: Don't activate block devices if using -S +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: "Dr. David Alan Gilbert" + +Activating the block devices causes the locks to be taken on +the backing file. If we're running with -S and the destination libvirt +hasn't started the destination with 'cont', it's expecting the locks are +still untaken. + +Don't activate the block devices if we're not going to autostart the VM; +'cont' already will do that anyway. This change is tied to the new +migration capability 'late-block-activate' that defaults to off, keeping +the old behaviour by default. + +bz: https://bugzilla.redhat.com/show_bug.cgi?id=1560854 +Signed-off-by: Dr. David Alan Gilbert +Reviewed-by: Juan Quintela +Signed-off-by: Juan Quintela +(cherry picked from commit 0f073f44df109ea0910d67caede70dec95956ff6) +--- + migration/migration.c | 34 +++++++++++++++++++++++++++------- + qapi/migration.json | 5 ++++- + 2 files changed, 31 insertions(+), 8 deletions(-) + +diff --git a/migration/migration.c b/migration/migration.c +index b6294f6..edf1c06 100644 +--- a/migration/migration.c ++++ b/migration/migration.c +@@ -199,6 +199,16 @@ static void migrate_generate_event(int new_state) + } + } + ++static bool migrate_late_block_activate(void) ++{ ++ MigrationState *s; ++ ++ s = migrate_get_current(); ++ ++ return s->enabled_capabilities[ ++ MIGRATION_CAPABILITY_LATE_BLOCK_ACTIVATE]; ++} ++ + /* + * Called on -incoming with a defer: uri. + * The migration can be started later after any parameters have been +@@ -308,13 +318,23 @@ static void process_incoming_migration_bh(void *opaque) + Error *local_err = NULL; + MigrationIncomingState *mis = opaque; + +- /* Make sure all file formats flush their mutable metadata. +- * If we get an error here, just don't restart the VM yet. */ +- bdrv_invalidate_cache_all(&local_err); +- if (local_err) { +- error_report_err(local_err); +- local_err = NULL; +- autostart = false; ++ /* If capability late_block_activate is set: ++ * Only fire up the block code now if we're going to restart the ++ * VM, else 'cont' will do it. ++ * This causes file locking to happen; so we don't want it to happen ++ * unless we really are starting the VM. ++ */ ++ if (!migrate_late_block_activate() || ++ (autostart && (!global_state_received() || ++ global_state_get_runstate() == RUN_STATE_RUNNING))) { ++ /* Make sure all file formats flush their mutable metadata. ++ * If we get an error here, just don't restart the VM yet. */ ++ bdrv_invalidate_cache_all(&local_err); ++ if (local_err) { ++ error_report_err(local_err); ++ local_err = NULL; ++ autostart = false; ++ } + } + + /* +diff --git a/qapi/migration.json b/qapi/migration.json +index 9d0bf82..618d2de 100644 +--- a/qapi/migration.json ++++ b/qapi/migration.json +@@ -357,13 +357,16 @@ + # @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps. + # (since 2.12) + # ++# @late-block-activate: If enabled, the destination will not activate block ++# devices (and thus take locks) immediately at the end of migration. ++# (since 3.0) + # Since: 1.2 + ## + { 'enum': 'MigrationCapability', + 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks', + 'compress', 'events', 'postcopy-ram', 'x-colo', 'release-ram', + 'block', 'return-path', 'pause-before-switchover', 'x-multifd', +- 'dirty-bitmaps' ] } ++ 'dirty-bitmaps', 'late-block-activate' ] } + + ## + # @MigrationCapabilityStatus: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-block-dirty-bitmap-fix-dirty_bitmap_load.patch b/SOURCES/kvm-migration-block-dirty-bitmap-fix-dirty_bitmap_load.patch new file mode 100644 index 0000000..35ed021 --- /dev/null +++ b/SOURCES/kvm-migration-block-dirty-bitmap-fix-dirty_bitmap_load.patch @@ -0,0 +1,50 @@ +From d8d7edf965db996cd6105cc9d550374af9bb6521 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Fri, 22 Jun 2018 19:00:05 +0200 +Subject: [PATCH 17/21] migration/block-dirty-bitmap: fix dirty_bitmap_load + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-19-dgilbert@redhat.com> +Patchwork-id: 81573 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 18/18] migration/block-dirty-bitmap: fix dirty_bitmap_load +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Vladimir Sementsov-Ogievskiy + +dirty_bitmap_load_header return code is obtained but not handled. Fix +this. + +Bug was introduced in b35ebdf076d697bc +"migration: add postcopy migration of dirty bitmaps" with the whole +function. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180530112424.204835-1-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +Reviewed-by: John Snow +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit a36f6ff46f115672cf86d0e1e7cdb1c2fa4d304b) +--- + migration/block-dirty-bitmap.c | 3 +++ + 1 file changed, 3 insertions(+) + +diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c +index b3b31ba..fefbc6a 100644 +--- a/migration/block-dirty-bitmap.c ++++ b/migration/block-dirty-bitmap.c +@@ -672,6 +672,9 @@ static int dirty_bitmap_load(QEMUFile *f, void *opaque, int version_id) + + do { + ret = dirty_bitmap_load_header(f, &s); ++ if (ret < 0) { ++ return ret; ++ } + + if (s.flags & DIRTY_BITMAP_MIG_FLAG_START) { + ret = dirty_bitmap_load_start(f, &s); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-block-dirty-bitmap-fix-memory-leak-in-dirt.patch b/SOURCES/kvm-migration-block-dirty-bitmap-fix-memory-leak-in-dirt.patch new file mode 100644 index 0000000..27abe9e --- /dev/null +++ b/SOURCES/kvm-migration-block-dirty-bitmap-fix-memory-leak-in-dirt.patch @@ -0,0 +1,48 @@ +From 2bbbfcc52eb4c82714a8570fe217d80354687186 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:15 +0100 +Subject: [PATCH 11/21] migration/block-dirty-bitmap: fix memory leak in + dirty_bitmap_load_bits + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-12-dgilbert@redhat.com> +Patchwork-id: 81567 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 11/18] migration/block-dirty-bitmap: fix memory leak in dirty_bitmap_load_bits +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Vladimir Sementsov-Ogievskiy + +Release buf on error path too. + +Bug was introduced in b35ebdf076d697bc "migration: add postcopy +migration of dirty bitmaps" with the whole function. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180427142002.21930-3-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +CC: qemu-stable@nongnu.org +Signed-off-by: Eric Blake +(cherry picked from commit 16a2227893dc1d5cad78ed376ad1d7e300978fbe) +Signed-off-by: Danilo C. L. de Paula +--- + migration/block-dirty-bitmap.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c +index abba0b6..b3b31ba 100644 +--- a/migration/block-dirty-bitmap.c ++++ b/migration/block-dirty-bitmap.c +@@ -600,6 +600,7 @@ static int dirty_bitmap_load_bits(QEMUFile *f, DirtyBitmapLoadState *s) + ret = qemu_get_buffer(f, buf, buf_size); + if (ret != buf_size) { + error_report("Failed to read bitmap bits"); ++ g_free(buf); + return -EIO; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-calculate-expected_downtime-with-ram_bytes.patch b/SOURCES/kvm-migration-calculate-expected_downtime-with-ram_bytes.patch new file mode 100644 index 0000000..ebfe1d5 --- /dev/null +++ b/SOURCES/kvm-migration-calculate-expected_downtime-with-ram_bytes.patch @@ -0,0 +1,68 @@ +From 1c9e86aac83e91498772885f5223e212337a3ff9 Mon Sep 17 00:00:00 2001 +From: Laurent Vivier +Date: Thu, 21 Jun 2018 09:45:26 +0200 +Subject: [PATCH 075/268] migration: calculate expected_downtime with + ram_bytes_remaining() + +RH-Author: Laurent Vivier +Message-id: <20180621094526.5714-1-lvivier@redhat.com> +Patchwork-id: 80931 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH] migration: calculate expected_downtime with ram_bytes_remaining() +Bugzilla: 1564576 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Thomas Huth +RH-Acked-by: David Gibson + +From: Balamuruhan S + +expected_downtime value is not accurate with dirty_pages_rate * page_size, +using ram_bytes_remaining() would yeild it resonable. + +consider to read the remaining ram just after having updated the dirty +pages count later migration_bitmap_sync_range() in migration_bitmap_sync() +and reuse the `remaining` field in ram_counters to hold ram_bytes_remaining() +for calculating expected_downtime. + +Reported-by: Michael Roth +Signed-off-by: Balamuruhan S +Signed-off-by: Laurent Vivier +Message-Id: <20180612085009.17594-2-bala24@linux.vnet.ibm.com> +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 650af8907bd567db914b7ce3a7e9df4c323f4619) +Signed-off-by: Laurent Vivier +Signed-off-by: Miroslav Rezanina +--- + migration/migration.c | 3 +-- + migration/ram.c | 1 + + 2 files changed, 2 insertions(+), 2 deletions(-) + +diff --git a/migration/migration.c b/migration/migration.c +index ef4bb42..43d8a64 100644 +--- a/migration/migration.c ++++ b/migration/migration.c +@@ -2239,8 +2239,7 @@ static void migration_update_counters(MigrationState *s, + * recalculate. 10000 is a small enough number for our purposes + */ + if (ram_counters.dirty_pages_rate && transferred > 10000) { +- s->expected_downtime = ram_counters.dirty_pages_rate * +- qemu_target_page_size() / bandwidth; ++ s->expected_downtime = ram_counters.remaining / bandwidth; + } + + qemu_file_reset_rate_limit(s->to_dst_file); +diff --git a/migration/ram.c b/migration/ram.c +index 0e90efa..00c06b5 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -852,6 +852,7 @@ static void migration_bitmap_sync(RAMState *rs) + RAMBLOCK_FOREACH(block) { + migration_bitmap_sync_range(rs, block, 0, block->used_length); + } ++ ram_counters.remaining = ram_bytes_remaining(); + rcu_read_unlock(); + qemu_mutex_unlock(&rs->bitmap_mutex); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-cleanup-in-error-paths-in-loadvm.patch b/SOURCES/kvm-migration-cleanup-in-error-paths-in-loadvm.patch new file mode 100644 index 0000000..85a300e --- /dev/null +++ b/SOURCES/kvm-migration-cleanup-in-error-paths-in-loadvm.patch @@ -0,0 +1,52 @@ +From 020674a9569df103bdd6a8cef29ce8013c92a8b8 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Mon, 1 Oct 2018 10:54:49 +0100 +Subject: [PATCH 03/28] migration: cleanup in error paths in loadvm + +RH-Author: Dr. David Alan Gilbert +Message-id: <20181001105449.41090-3-dgilbert@redhat.com> +Patchwork-id: 82325 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 2/2] migration: cleanup in error paths in loadvm +Bugzilla: 1608765 +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Laurent Vivier + +From: "Dr. David Alan Gilbert" + +There's a couple of error paths in qemu_loadvm_state +which happen early on but after we've initialised the +load state; that needs to be cleaned up otherwise +we can hit asserts if the state gets reinitialised later. + +Signed-off-by: Dr. David Alan Gilbert +Message-Id: <20180914170430.54271-3-dgilbert@redhat.com> +Reviewed-by: Peter Xu +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 096c83b7219c5a2145435afc8be750281e9cb447) +Signed-off-by: Danilo C. L. de Paula +--- + migration/savevm.c | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/migration/savevm.c b/migration/savevm.c +index 6a8d363..edb3b94 100644 +--- a/migration/savevm.c ++++ b/migration/savevm.c +@@ -2145,11 +2145,13 @@ int qemu_loadvm_state(QEMUFile *f) + if (migrate_get_current()->send_configuration) { + if (qemu_get_byte(f) != QEMU_VM_CONFIGURATION) { + error_report("Configuration section missing"); ++ qemu_loadvm_state_cleanup(); + return -EINVAL; + } + ret = vmstate_load_state(f, &vmstate_configuration, &savevm_state, 0); + + if (ret) { ++ qemu_loadvm_state_cleanup(); + return ret; + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-detect-compression-and-decompression-error.patch b/SOURCES/kvm-migration-detect-compression-and-decompression-error.patch new file mode 100644 index 0000000..9d09e8d --- /dev/null +++ b/SOURCES/kvm-migration-detect-compression-and-decompression-error.patch @@ -0,0 +1,235 @@ +From aa3254bca93fb1702f0aa236b70d705ee8bf121c Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:08 +0100 +Subject: [PATCH 04/21] migration: detect compression and decompression errors + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-5-dgilbert@redhat.com> +Patchwork-id: 81583 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 04/18] migration: detect compression and decompression errors +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Currently the page being compressed is allowed to be updated by +the VM on the source QEMU, correspondingly the destination QEMU +just ignores the decompression error. However, we completely miss +the chance to catch real errors, then the VM is corrupted silently + +To make the migration more robuster, we copy the page to a buffer +first to avoid it being written by VM, then detect and handle the +errors of both compression and decompression errors properly + +Reviewed-by: Peter Xu +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-5-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 34ab9e9743aeaf265929d930747f101fa5c76fea) +Signed-off-by: Danilo C. L. de Paula +--- + migration/qemu-file.c | 4 ++-- + migration/ram.c | 56 +++++++++++++++++++++++++++++++++++---------------- + 2 files changed, 41 insertions(+), 19 deletions(-) + +diff --git a/migration/qemu-file.c b/migration/qemu-file.c +index bafe3a0..0463f4c 100644 +--- a/migration/qemu-file.c ++++ b/migration/qemu-file.c +@@ -710,9 +710,9 @@ ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, + blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t), + blen, p, size); + if (blen < 0) { +- error_report("Compress Failed!"); +- return 0; ++ return -1; + } ++ + qemu_put_be32(f, blen); + if (f->ops->writev_buffer) { + add_to_iovec(f, f->buf + f->buf_index, blen, false); +diff --git a/migration/ram.c b/migration/ram.c +index be89cd8..cd6d98a 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -269,7 +269,10 @@ struct CompressParam { + QemuCond cond; + RAMBlock *block; + ram_addr_t offset; ++ ++ /* internally used fields */ + z_stream stream; ++ uint8_t *originbuf; + }; + typedef struct CompressParam CompressParam; + +@@ -296,13 +299,14 @@ static QemuCond comp_done_cond; + /* The empty QEMUFileOps will be used by file in CompressParam */ + static const QEMUFileOps empty_ops = { }; + ++static QEMUFile *decomp_file; + static DecompressParam *decomp_param; + static QemuThread *decompress_threads; + static QemuMutex decomp_done_lock; + static QemuCond decomp_done_cond; + + static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, +- ram_addr_t offset); ++ ram_addr_t offset, uint8_t *source_buf); + + static void *do_data_compress(void *opaque) + { +@@ -318,7 +322,8 @@ static void *do_data_compress(void *opaque) + param->block = NULL; + qemu_mutex_unlock(¶m->mutex); + +- do_compress_ram_page(param->file, ¶m->stream, block, offset); ++ do_compress_ram_page(param->file, ¶m->stream, block, offset, ++ param->originbuf); + + qemu_mutex_lock(&comp_done_lock); + param->done = true; +@@ -370,6 +375,7 @@ static void compress_threads_save_cleanup(void) + qemu_mutex_destroy(&comp_param[i].mutex); + qemu_cond_destroy(&comp_param[i].cond); + deflateEnd(&comp_param[i].stream); ++ g_free(comp_param[i].originbuf); + qemu_fclose(comp_param[i].file); + comp_param[i].file = NULL; + } +@@ -394,8 +400,14 @@ static int compress_threads_save_setup(void) + qemu_cond_init(&comp_done_cond); + qemu_mutex_init(&comp_done_lock); + for (i = 0; i < thread_count; i++) { ++ comp_param[i].originbuf = g_try_malloc(TARGET_PAGE_SIZE); ++ if (!comp_param[i].originbuf) { ++ goto exit; ++ } ++ + if (deflateInit(&comp_param[i].stream, + migrate_compress_level()) != Z_OK) { ++ g_free(comp_param[i].originbuf); + goto exit; + } + +@@ -1054,7 +1066,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) + } + + static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, +- ram_addr_t offset) ++ ram_addr_t offset, uint8_t *source_buf) + { + RAMState *rs = ram_state; + int bytes_sent, blen; +@@ -1062,7 +1074,14 @@ static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, + + bytes_sent = save_page_header(rs, f, block, offset | + RAM_SAVE_FLAG_COMPRESS_PAGE); +- blen = qemu_put_compression_data(f, stream, p, TARGET_PAGE_SIZE); ++ ++ /* ++ * copy it to a internal buffer to avoid it being modified by VM ++ * so that we can catch up the error during compression and ++ * decompression ++ */ ++ memcpy(source_buf, p, TARGET_PAGE_SIZE); ++ blen = qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_SIZE); + if (blen < 0) { + bytes_sent = 0; + qemu_file_set_error(migrate_get_current()->to_dst_file, blen); +@@ -2556,7 +2575,7 @@ static void *do_data_decompress(void *opaque) + DecompressParam *param = opaque; + unsigned long pagesize; + uint8_t *des; +- int len; ++ int len, ret; + + qemu_mutex_lock(¶m->mutex); + while (!param->quit) { +@@ -2567,13 +2586,13 @@ static void *do_data_decompress(void *opaque) + qemu_mutex_unlock(¶m->mutex); + + pagesize = TARGET_PAGE_SIZE; +- /* qemu_uncompress_data() will return failed in some case, +- * especially when the page is dirtied when doing the compression, +- * it's not a problem because the dirty page will be retransferred +- * and uncompress() won't break the data in other pages. +- */ +- qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf, +- len); ++ ++ ret = qemu_uncompress_data(¶m->stream, des, pagesize, ++ param->compbuf, len); ++ if (ret < 0) { ++ error_report("decompress data failed"); ++ qemu_file_set_error(decomp_file, ret); ++ } + + qemu_mutex_lock(&decomp_done_lock); + param->done = true; +@@ -2590,12 +2609,12 @@ static void *do_data_decompress(void *opaque) + return NULL; + } + +-static void wait_for_decompress_done(void) ++static int wait_for_decompress_done(void) + { + int idx, thread_count; + + if (!migrate_use_compression()) { +- return; ++ return 0; + } + + thread_count = migrate_decompress_threads(); +@@ -2606,6 +2625,7 @@ static void wait_for_decompress_done(void) + } + } + qemu_mutex_unlock(&decomp_done_lock); ++ return qemu_file_get_error(decomp_file); + } + + static void compress_threads_load_cleanup(void) +@@ -2646,9 +2666,10 @@ static void compress_threads_load_cleanup(void) + g_free(decomp_param); + decompress_threads = NULL; + decomp_param = NULL; ++ decomp_file = NULL; + } + +-static int compress_threads_load_setup(void) ++static int compress_threads_load_setup(QEMUFile *f) + { + int i, thread_count; + +@@ -2661,6 +2682,7 @@ static int compress_threads_load_setup(void) + decomp_param = g_new0(DecompressParam, thread_count); + qemu_mutex_init(&decomp_done_lock); + qemu_cond_init(&decomp_done_cond); ++ decomp_file = f; + for (i = 0; i < thread_count; i++) { + if (inflateInit(&decomp_param[i].stream) != Z_OK) { + goto exit; +@@ -2720,7 +2742,7 @@ static void decompress_data_with_multi_threads(QEMUFile *f, + */ + static int ram_load_setup(QEMUFile *f, void *opaque) + { +- if (compress_threads_load_setup()) { ++ if (compress_threads_load_setup(f)) { + return -1; + } + +@@ -3075,7 +3097,7 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) + } + } + +- wait_for_decompress_done(); ++ ret |= wait_for_decompress_done(); + rcu_read_unlock(); + trace_ram_load_complete(ret, seq_iter); + return ret; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-discard-non-migratable-RAMBlocks.patch b/SOURCES/kvm-migration-discard-non-migratable-RAMBlocks.patch new file mode 100644 index 0000000..130c4ff --- /dev/null +++ b/SOURCES/kvm-migration-discard-non-migratable-RAMBlocks.patch @@ -0,0 +1,381 @@ +From c38aaf5a09a4f06096f9a66ca3a7c22c7e657a4f Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:14 +0000 +Subject: [PATCH 13/22] migration: discard non-migratable RAMBlocks +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-2-git-send-email-plai@redhat.com> +Patchwork-id: 83886 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 01/10] migration: discard non-migratable RAMBlocks +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Cédric Le Goater + +On the POWER9 processor, the XIVE interrupt controller can control +interrupt sources using MMIO to trigger events, to EOI or to turn off +the sources. Priority management and interrupt acknowledgment is also +controlled by MMIO in the presenter sub-engine. + +These MMIO regions are exposed to guests in QEMU with a set of 'ram +device' memory mappings, similarly to VFIO, and the VMAs are populated +dynamically with the appropriate pages using a fault handler. + +But, these regions are an issue for migration. We need to discard the +associated RAMBlocks from the RAM state on the source VM and let the +destination VM rebuild the memory mappings on the new host in the +post_load() operation just before resuming the system. + +To achieve this goal, the following introduces a new RAMBlock flag +RAM_MIGRATABLE which is updated in the vmstate_register_ram() and +vmstate_unregister_ram() routines. This flag is then used by the +migration to identify RAMBlocks to discard on the source. Some checks +are also performed on the destination to make sure nothing invalid was +sent. + +This change impacts the boston, malta and jazz mips boards for which +migration compatibility is broken. + +Signed-off-by: Cédric Le Goater +Reviewed-by: Juan Quintela +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Juan Quintela +(cherry picked from commit b895de502717b83b4e5f089df617cb23530c4d2d) +Signed-off-by: Paul Lai +Signed-off-by: Danilo C. L. de Paula +--- + exec.c | 38 ++++++++++++++++++++++++++++++++++++++ + include/exec/cpu-common.h | 4 ++++ + migration/postcopy-ram.c | 12 ++++++------ + migration/ram.c | 46 ++++++++++++++++++++++++++++++++++------------ + migration/savevm.c | 2 ++ + 5 files changed, 84 insertions(+), 18 deletions(-) + +diff --git a/exec.c b/exec.c +index 22cc7ef..fff49ba 100644 +--- a/exec.c ++++ b/exec.c +@@ -104,6 +104,9 @@ static MemoryRegion io_mem_unassigned; + * (Set during postcopy) + */ + #define RAM_UF_ZEROPAGE (1 << 3) ++ ++/* RAM can be migrated */ ++#define RAM_MIGRATABLE (1 << 4) + #endif + + #ifdef TARGET_PAGE_BITS_VARY +@@ -1811,6 +1814,21 @@ void qemu_ram_set_uf_zeroable(RAMBlock *rb) + rb->flags |= RAM_UF_ZEROPAGE; + } + ++bool qemu_ram_is_migratable(RAMBlock *rb) ++{ ++ return rb->flags & RAM_MIGRATABLE; ++} ++ ++void qemu_ram_set_migratable(RAMBlock *rb) ++{ ++ rb->flags |= RAM_MIGRATABLE; ++} ++ ++void qemu_ram_unset_migratable(RAMBlock *rb) ++{ ++ rb->flags &= ~RAM_MIGRATABLE; ++} ++ + /* Called with iothread lock held. */ + void qemu_ram_set_idstr(RAMBlock *new_block, const char *name, DeviceState *dev) + { +@@ -3754,6 +3772,26 @@ int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque) + return ret; + } + ++int qemu_ram_foreach_migratable_block(RAMBlockIterFunc func, void *opaque) ++{ ++ RAMBlock *block; ++ int ret = 0; ++ ++ rcu_read_lock(); ++ RAMBLOCK_FOREACH(block) { ++ if (!qemu_ram_is_migratable(block)) { ++ continue; ++ } ++ ret = func(block->idstr, block->host, block->offset, ++ block->used_length, opaque); ++ if (ret) { ++ break; ++ } ++ } ++ rcu_read_unlock(); ++ return ret; ++} ++ + /* + * Unmap pages of memory from start to start+length such that + * they a) read as 0, b) Trigger whatever fault mechanism +diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h +index 24d335f..0b58e26 100644 +--- a/include/exec/cpu-common.h ++++ b/include/exec/cpu-common.h +@@ -75,6 +75,9 @@ const char *qemu_ram_get_idstr(RAMBlock *rb); + bool qemu_ram_is_shared(RAMBlock *rb); + bool qemu_ram_is_uf_zeroable(RAMBlock *rb); + void qemu_ram_set_uf_zeroable(RAMBlock *rb); ++bool qemu_ram_is_migratable(RAMBlock *rb); ++void qemu_ram_set_migratable(RAMBlock *rb); ++void qemu_ram_unset_migratable(RAMBlock *rb); + + size_t qemu_ram_pagesize(RAMBlock *block); + size_t qemu_ram_pagesize_largest(void); +@@ -119,6 +122,7 @@ typedef int (RAMBlockIterFunc)(const char *block_name, void *host_addr, + ram_addr_t offset, ram_addr_t length, void *opaque); + + int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque); ++int qemu_ram_foreach_migratable_block(RAMBlockIterFunc func, void *opaque); + int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length); + + #endif +diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c +index b04e903..4b65ff9 100644 +--- a/migration/postcopy-ram.c ++++ b/migration/postcopy-ram.c +@@ -264,7 +264,7 @@ bool postcopy_ram_supported_by_host(MigrationIncomingState *mis) + } + + /* We don't support postcopy with shared RAM yet */ +- if (qemu_ram_foreach_block(test_ramblock_postcopiable, NULL)) { ++ if (qemu_ram_foreach_migratable_block(test_ramblock_postcopiable, NULL)) { + goto out; + } + +@@ -392,7 +392,7 @@ static int cleanup_range(const char *block_name, void *host_addr, + */ + int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_pages) + { +- if (qemu_ram_foreach_block(init_range, NULL)) { ++ if (qemu_ram_foreach_migratable_block(init_range, NULL)) { + return -1; + } + +@@ -428,7 +428,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) + return -1; + } + +- if (qemu_ram_foreach_block(cleanup_range, mis)) { ++ if (qemu_ram_foreach_migratable_block(cleanup_range, mis)) { + return -1; + } + /* Let the fault thread quit */ +@@ -494,7 +494,7 @@ static int nhp_range(const char *block_name, void *host_addr, + */ + int postcopy_ram_prepare_discard(MigrationIncomingState *mis) + { +- if (qemu_ram_foreach_block(nhp_range, mis)) { ++ if (qemu_ram_foreach_migratable_block(nhp_range, mis)) { + return -1; + } + +@@ -505,7 +505,7 @@ int postcopy_ram_prepare_discard(MigrationIncomingState *mis) + + /* + * Mark the given area of RAM as requiring notification to unwritten areas +- * Used as a callback on qemu_ram_foreach_block. ++ * Used as a callback on qemu_ram_foreach_migratable_block. + * host_addr: Base of area to mark + * offset: Offset in the whole ram arena + * length: Length of the section +@@ -807,7 +807,7 @@ int postcopy_ram_enable_notify(MigrationIncomingState *mis) + mis->have_fault_thread = true; + + /* Mark so that we get notified of accesses to unwritten areas */ +- if (qemu_ram_foreach_block(ram_block_enable_notify, mis)) { ++ if (qemu_ram_foreach_migratable_block(ram_block_enable_notify, mis)) { + return -1; + } + +diff --git a/migration/ram.c b/migration/ram.c +index bd563b5..04b5df5 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -153,11 +153,16 @@ out: + return ret; + } + ++/* Should be holding either ram_list.mutex, or the RCU lock. */ ++#define RAMBLOCK_FOREACH_MIGRATABLE(block) \ ++ RAMBLOCK_FOREACH(block) \ ++ if (!qemu_ram_is_migratable(block)) {} else ++ + static void ramblock_recv_map_init(void) + { + RAMBlock *rb; + +- RAMBLOCK_FOREACH(rb) { ++ RAMBLOCK_FOREACH_MIGRATABLE(rb) { + assert(!rb->receivedmap); + rb->receivedmap = bitmap_new(rb->max_length >> qemu_target_page_bits()); + } +@@ -813,6 +818,10 @@ unsigned long migration_bitmap_find_dirty(RAMState *rs, RAMBlock *rb, + unsigned long *bitmap = rb->bmap; + unsigned long next; + ++ if (!qemu_ram_is_migratable(rb)) { ++ return size; ++ } ++ + if (rs->ram_bulk_stage && start > 0) { + next = start + 1; + } else { +@@ -858,7 +867,7 @@ uint64_t ram_pagesize_summary(void) + RAMBlock *block; + uint64_t summary = 0; + +- RAMBLOCK_FOREACH(block) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + summary |= block->page_size; + } + +@@ -882,7 +891,7 @@ static void migration_bitmap_sync(RAMState *rs) + + qemu_mutex_lock(&rs->bitmap_mutex); + rcu_read_lock(); +- RAMBLOCK_FOREACH(block) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + migration_bitmap_sync_range(rs, block, 0, block->used_length); + } + ram_counters.remaining = ram_bytes_remaining(); +@@ -1522,6 +1531,11 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, + size_t pagesize_bits = + qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; + ++ if (!qemu_ram_is_migratable(pss->block)) { ++ error_report("block %s should not be migrated !", pss->block->idstr); ++ return 0; ++ } ++ + do { + /* Check the pages is dirty and if it is send it */ + if (!migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { +@@ -1620,7 +1634,7 @@ uint64_t ram_bytes_total(void) + uint64_t total = 0; + + rcu_read_lock(); +- RAMBLOCK_FOREACH(block) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + total += block->used_length; + } + rcu_read_unlock(); +@@ -1675,7 +1689,7 @@ static void ram_save_cleanup(void *opaque) + */ + memory_global_dirty_log_stop(); + +- QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + g_free(block->bmap); + block->bmap = NULL; + g_free(block->unsentmap); +@@ -1738,7 +1752,7 @@ void ram_postcopy_migrated_memory_release(MigrationState *ms) + { + struct RAMBlock *block; + +- RAMBLOCK_FOREACH(block) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + unsigned long *bitmap = block->bmap; + unsigned long range = block->used_length >> TARGET_PAGE_BITS; + unsigned long run_start = find_next_zero_bit(bitmap, range, 0); +@@ -1816,7 +1830,7 @@ static int postcopy_each_ram_send_discard(MigrationState *ms) + struct RAMBlock *block; + int ret; + +- RAMBLOCK_FOREACH(block) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + PostcopyDiscardState *pds = + postcopy_discard_send_init(ms, block->idstr); + +@@ -2024,7 +2038,7 @@ int ram_postcopy_send_discard_bitmap(MigrationState *ms) + rs->last_sent_block = NULL; + rs->last_page = 0; + +- QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + unsigned long pages = block->used_length >> TARGET_PAGE_BITS; + unsigned long *bitmap = block->bmap; + unsigned long *unsentmap = block->unsentmap; +@@ -2183,7 +2197,7 @@ static void ram_list_init_bitmaps(void) + + /* Skip setting bitmap if there is no RAM */ + if (ram_bytes_total()) { +- QLIST_FOREACH_RCU(block, &ram_list.blocks, next) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + pages = block->max_length >> TARGET_PAGE_BITS; + block->bmap = bitmap_new(pages); + bitmap_set(block->bmap, 0, pages); +@@ -2264,7 +2278,7 @@ static int ram_save_setup(QEMUFile *f, void *opaque) + + qemu_put_be64(f, ram_bytes_total() | RAM_SAVE_FLAG_MEM_SIZE); + +- RAMBLOCK_FOREACH(block) { ++ RAMBLOCK_FOREACH_MIGRATABLE(block) { + qemu_put_byte(f, strlen(block->idstr)); + qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idstr)); + qemu_put_be64(f, block->used_length); +@@ -2508,6 +2522,11 @@ static inline RAMBlock *ram_block_from_stream(QEMUFile *f, int flags) + return NULL; + } + ++ if (!qemu_ram_is_migratable(block)) { ++ error_report("block %s should not be migrated !", id); ++ return NULL; ++ } ++ + return block; + } + +@@ -2750,7 +2769,7 @@ static int ram_load_cleanup(void *opaque) + xbzrle_load_cleanup(); + compress_threads_load_cleanup(); + +- RAMBLOCK_FOREACH(rb) { ++ RAMBLOCK_FOREACH_MIGRATABLE(rb) { + g_free(rb->receivedmap); + rb->receivedmap = NULL; + } +@@ -3012,7 +3031,10 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) + length = qemu_get_be64(f); + + block = qemu_ram_block_by_name(id); +- if (block) { ++ if (block && !qemu_ram_is_migratable(block)) { ++ error_report("block %s should not be migrated !", id); ++ ret = -EINVAL; ++ } else if (block) { + if (length != block->used_length) { + Error *local_err = NULL; + +diff --git a/migration/savevm.c b/migration/savevm.c +index edb3b94..0bb9446 100644 +--- a/migration/savevm.c ++++ b/migration/savevm.c +@@ -2506,11 +2506,13 @@ void vmstate_register_ram(MemoryRegion *mr, DeviceState *dev) + { + qemu_ram_set_idstr(mr->ram_block, + memory_region_name(mr), dev); ++ qemu_ram_set_migratable(mr->ram_block); + } + + void vmstate_unregister_ram(MemoryRegion *mr, DeviceState *dev) + { + qemu_ram_unset_idstr(mr->ram_block); ++ qemu_ram_unset_migratable(mr->ram_block); + } + + void vmstate_register_ram_global(MemoryRegion *mr) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-fix-saving-normal-page-even-if-it-s-been-c.patch b/SOURCES/kvm-migration-fix-saving-normal-page-even-if-it-s-been-c.patch new file mode 100644 index 0000000..8869592 --- /dev/null +++ b/SOURCES/kvm-migration-fix-saving-normal-page-even-if-it-s-been-c.patch @@ -0,0 +1,48 @@ +From c3f7e63561bbede9c3b6278a5fd5e262ce960f4c Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:16 +0100 +Subject: [PATCH 12/21] migration: fix saving normal page even if it's been + compressed + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-13-dgilbert@redhat.com> +Patchwork-id: 81568 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 12/18] migration: fix saving normal page even if it's been compressed +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Fix the bug introduced by da3f56cb2e767016 (migration: remove +ram_save_compressed_page()), It should be 'return' rather than +'res' + +Sorry for this stupid mistake :( + +Signed-off-by: Xiao Guangrong +Message-Id: <20180428081045.8878-1-xiaoguangrong@tencent.com> +Signed-off-by: Juan Quintela +(cherry picked from commit 701b1876c0fc0c583e4aff300ace5d33a1b97ed6) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 466609f..c982201 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1491,7 +1491,7 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, + * CPU resource. + */ + if (block == rs->last_sent_block && save_page_use_compression(rs)) { +- res = compress_page_with_multi_thread(rs, block, offset); ++ return compress_page_with_multi_thread(rs, block, offset); + } + + return ram_save_page(rs, pss, last_stage); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-introduce-control_save_page.patch b/SOURCES/kvm-migration-introduce-control_save_page.patch new file mode 100644 index 0000000..e7ee139 --- /dev/null +++ b/SOURCES/kvm-migration-introduce-control_save_page.patch @@ -0,0 +1,254 @@ +From 6d52c264fa9a7a78ee532f45d24704491ce4c431 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:09 +0100 +Subject: [PATCH 05/21] migration: introduce control_save_page() + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-6-dgilbert@redhat.com> +Patchwork-id: 81581 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 05/18] migration: introduce control_save_page() +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Abstract the common function control_save_page() to cleanup the code, +no logic is changed + +Reviewed-by: Peter Xu +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-6-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 059ff0fb29dd3a56ac2843676915efc279938c6b) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 174 +++++++++++++++++++++++++++++--------------------------- + 1 file changed, 89 insertions(+), 85 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index cd6d98a..8dc98a5 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -975,6 +975,44 @@ static void ram_release_pages(const char *rbname, uint64_t offset, int pages) + ram_discard_range(rbname, offset, pages << TARGET_PAGE_BITS); + } + ++/* ++ * @pages: the number of pages written by the control path, ++ * < 0 - error ++ * > 0 - number of pages written ++ * ++ * Return true if the pages has been saved, otherwise false is returned. ++ */ ++static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t offset, ++ int *pages) ++{ ++ uint64_t bytes_xmit = 0; ++ int ret; ++ ++ *pages = -1; ++ ret = ram_control_save_page(rs->f, block->offset, offset, TARGET_PAGE_SIZE, ++ &bytes_xmit); ++ if (ret == RAM_SAVE_CONTROL_NOT_SUPP) { ++ return false; ++ } ++ ++ if (bytes_xmit) { ++ ram_counters.transferred += bytes_xmit; ++ *pages = 1; ++ } ++ ++ if (ret == RAM_SAVE_CONTROL_DELAYED) { ++ return true; ++ } ++ ++ if (bytes_xmit > 0) { ++ ram_counters.normal++; ++ } else if (bytes_xmit == 0) { ++ ram_counters.duplicate++; ++ } ++ ++ return true; ++} ++ + /** + * ram_save_page: send the given page to the stream + * +@@ -991,56 +1029,36 @@ static void ram_release_pages(const char *rbname, uint64_t offset, int pages) + static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) + { + int pages = -1; +- uint64_t bytes_xmit; +- ram_addr_t current_addr; + uint8_t *p; +- int ret; + bool send_async = true; + RAMBlock *block = pss->block; + ram_addr_t offset = pss->page << TARGET_PAGE_BITS; ++ ram_addr_t current_addr = block->offset + offset; + + p = block->host + offset; + trace_ram_save_page(block->idstr, (uint64_t)offset, p); + +- /* In doubt sent page as normal */ +- bytes_xmit = 0; +- ret = ram_control_save_page(rs->f, block->offset, +- offset, TARGET_PAGE_SIZE, &bytes_xmit); +- if (bytes_xmit) { +- ram_counters.transferred += bytes_xmit; +- pages = 1; ++ if (control_save_page(rs, block, offset, &pages)) { ++ return pages; + } + + XBZRLE_cache_lock(); +- +- current_addr = block->offset + offset; +- +- if (ret != RAM_SAVE_CONTROL_NOT_SUPP) { +- if (ret != RAM_SAVE_CONTROL_DELAYED) { +- if (bytes_xmit > 0) { +- ram_counters.normal++; +- } else if (bytes_xmit == 0) { +- ram_counters.duplicate++; +- } +- } +- } else { +- pages = save_zero_page(rs, block, offset); +- if (pages > 0) { +- /* Must let xbzrle know, otherwise a previous (now 0'd) cached +- * page would be stale ++ pages = save_zero_page(rs, block, offset); ++ if (pages > 0) { ++ /* Must let xbzrle know, otherwise a previous (now 0'd) cached ++ * page would be stale ++ */ ++ xbzrle_cache_zero_page(rs, current_addr); ++ ram_release_pages(block->idstr, offset, pages); ++ } else if (!rs->ram_bulk_stage && ++ !migration_in_postcopy() && migrate_use_xbzrle()) { ++ pages = save_xbzrle_page(rs, &p, current_addr, block, ++ offset, last_stage); ++ if (!last_stage) { ++ /* Can't send this cached data async, since the cache page ++ * might get updated before it gets to the wire + */ +- xbzrle_cache_zero_page(rs, current_addr); +- ram_release_pages(block->idstr, offset, pages); +- } else if (!rs->ram_bulk_stage && +- !migration_in_postcopy() && migrate_use_xbzrle()) { +- pages = save_xbzrle_page(rs, &p, current_addr, block, +- offset, last_stage); +- if (!last_stage) { +- /* Can't send this cached data async, since the cache page +- * might get updated before it gets to the wire +- */ +- send_async = false; +- } ++ send_async = false; + } + } + +@@ -1175,63 +1193,49 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, + bool last_stage) + { + int pages = -1; +- uint64_t bytes_xmit = 0; + uint8_t *p; +- int ret; + RAMBlock *block = pss->block; + ram_addr_t offset = pss->page << TARGET_PAGE_BITS; + + p = block->host + offset; + +- ret = ram_control_save_page(rs->f, block->offset, +- offset, TARGET_PAGE_SIZE, &bytes_xmit); +- if (bytes_xmit) { +- ram_counters.transferred += bytes_xmit; +- pages = 1; ++ if (control_save_page(rs, block, offset, &pages)) { ++ return pages; + } +- if (ret != RAM_SAVE_CONTROL_NOT_SUPP) { +- if (ret != RAM_SAVE_CONTROL_DELAYED) { +- if (bytes_xmit > 0) { +- ram_counters.normal++; +- } else if (bytes_xmit == 0) { +- ram_counters.duplicate++; +- } ++ ++ /* When starting the process of a new block, the first page of ++ * the block should be sent out before other pages in the same ++ * block, and all the pages in last block should have been sent ++ * out, keeping this order is important, because the 'cont' flag ++ * is used to avoid resending the block name. ++ */ ++ if (block != rs->last_sent_block) { ++ flush_compressed_data(rs); ++ pages = save_zero_page(rs, block, offset); ++ if (pages > 0) { ++ ram_release_pages(block->idstr, offset, pages); ++ } else { ++ /* ++ * Make sure the first page is sent out before other pages. ++ * ++ * we post it as normal page as compression will take much ++ * CPU resource. ++ */ ++ ram_counters.transferred += save_page_header(rs, rs->f, block, ++ offset | RAM_SAVE_FLAG_PAGE); ++ qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, ++ migrate_release_ram() & ++ migration_in_postcopy()); ++ ram_counters.transferred += TARGET_PAGE_SIZE; ++ ram_counters.normal++; ++ pages = 1; + } + } else { +- /* When starting the process of a new block, the first page of +- * the block should be sent out before other pages in the same +- * block, and all the pages in last block should have been sent +- * out, keeping this order is important, because the 'cont' flag +- * is used to avoid resending the block name. +- */ +- if (block != rs->last_sent_block) { +- flush_compressed_data(rs); +- pages = save_zero_page(rs, block, offset); +- if (pages > 0) { +- ram_release_pages(block->idstr, offset, pages); +- } else { +- /* +- * Make sure the first page is sent out before other pages. +- * +- * we post it as normal page as compression will take much +- * CPU resource. +- */ +- ram_counters.transferred += save_page_header(rs, rs->f, block, +- offset | RAM_SAVE_FLAG_PAGE); +- qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, +- migrate_release_ram() & +- migration_in_postcopy()); +- ram_counters.transferred += TARGET_PAGE_SIZE; +- ram_counters.normal++; +- pages = 1; +- } ++ pages = save_zero_page(rs, block, offset); ++ if (pages == -1) { ++ pages = compress_page_with_multi_thread(rs, block, offset); + } else { +- pages = save_zero_page(rs, block, offset); +- if (pages == -1) { +- pages = compress_page_with_multi_thread(rs, block, offset); +- } else { +- ram_release_pages(block->idstr, offset, pages); +- } ++ ram_release_pages(block->idstr, offset, pages); + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-introduce-decompress-error-check.patch b/SOURCES/kvm-migration-introduce-decompress-error-check.patch new file mode 100644 index 0000000..16199d5 --- /dev/null +++ b/SOURCES/kvm-migration-introduce-decompress-error-check.patch @@ -0,0 +1,163 @@ +From 2343d566f8dff6f97dfb280fbf409ff20379640c Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 12:55:19 +0000 +Subject: [PATCH 14/21] migration: introduce decompress-error-check + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-16-dgilbert@redhat.com> +Patchwork-id: 81575 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 15/18] migration: introduce decompress-error-check +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +QEMU 3.0 enables strict check for compression & decompression to +make the migration more robust, that depends on the source to fix +the internal design which triggers the unexpected error conditions + +To make it work for migrating old version QEMU to 2.13 QEMU, we +introduce this parameter to disable the error check on the +destination which is the default behavior of the machine type +which is older than 2.13, alternately, the strict check can be +enabled explicitly as followings: + -M pc-q35-2.11 -global migration.decompress-error-check=true + +Signed-off-by: Xiao Guangrong +Reviewed-by: Juan Quintela +Signed-off-by: Juan Quintela +(cherry picked from commit f548222c24342ca74689de7794f9006b43f86a54) + added compat entry to HW_COMPAT_RHEL7_5 +Signed-off-by: Dr. David Alan Gilbert + Context conflict in compat.h; 8.0 hasn't got the cirrus change 7.6 has +--- + hw/arm/virt.c | 4 ++++ + hw/i386/pc_piix.c | 1 + + hw/i386/pc_q35.c | 1 + + include/hw/compat.h | 5 +++++ + migration/migration.c | 4 ++++ + migration/migration.h | 8 ++++++++ + migration/ram.c | 2 +- + 7 files changed, 24 insertions(+), 1 deletion(-) + +diff --git a/hw/arm/virt.c b/hw/arm/virt.c +index a4d0f52..751a93c 100644 +--- a/hw/arm/virt.c ++++ b/hw/arm/virt.c +@@ -1607,6 +1607,9 @@ static void machvirt_machine_init(void) + } + type_init(machvirt_machine_init); + ++#define VIRT_COMPAT_2_12 \ ++ HW_COMPAT_2_12 ++ + static void virt_2_12_instance_init(Object *obj) + { + VirtMachineState *vms = VIRT_MACHINE(obj); +@@ -1669,6 +1672,7 @@ static void virt_2_12_instance_init(Object *obj) + + static void virt_machine_2_12_options(MachineClass *mc) + { ++ SET_MACHINE_COMPAT(mc, VIRT_COMPAT_2_12); + } + DEFINE_VIRT_MACHINE_AS_LATEST(2, 12) + +diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c +index 60441c1..9d9cee0 100644 +--- a/hw/i386/pc_piix.c ++++ b/hw/i386/pc_piix.c +@@ -432,6 +432,7 @@ static void pc_i440fx_2_12_machine_options(MachineClass *m) + pc_i440fx_machine_options(m); + m->alias = "pc"; + m->is_default = 1; ++ SET_MACHINE_COMPAT(m, PC_COMPAT_2_12); + } + + DEFINE_I440FX_MACHINE(v2_12, "pc-i440fx-2.12", NULL, +diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c +index ccdeb11..90f12b1 100644 +--- a/hw/i386/pc_q35.c ++++ b/hw/i386/pc_q35.c +@@ -313,6 +313,7 @@ static void pc_q35_2_12_machine_options(MachineClass *m) + { + pc_q35_machine_options(m); + m->alias = "q35"; ++ SET_MACHINE_COMPAT(m, PC_COMPAT_2_12); + } + + DEFINE_Q35_MACHINE(v2_12, "pc-q35-2.12", NULL, +diff --git a/include/hw/compat.h b/include/hw/compat.h +index c343e8f..8b59c30 100644 +--- a/include/hw/compat.h ++++ b/include/hw/compat.h +@@ -477,6 +477,11 @@ + .driver = "cirrus-vga",\ + .property = "vgamem_mb",\ + .value = "16",\ ++ },{ /* HW_COMPAT_RHEL7_5 */ \ ++ .driver = "migration",\ ++ .property = "decompress-error-check",\ ++ .value = "off",\ + }, + ++ + #endif /* HW_COMPAT_H */ +diff --git a/migration/migration.c b/migration/migration.c +index 43d8a64..b6294f6 100644 +--- a/migration/migration.c ++++ b/migration/migration.c +@@ -2483,6 +2483,8 @@ void migration_global_dump(Monitor *mon) + ms->send_configuration ? "on" : "off"); + monitor_printf(mon, "send-section-footer: %s\n", + ms->send_section_footer ? "on" : "off"); ++ monitor_printf(mon, "decompress-error-check: %s\n", ++ ms->decompress_error_check ? "on" : "off"); + } + + #define DEFINE_PROP_MIG_CAP(name, x) \ +@@ -2496,6 +2498,8 @@ static Property migration_properties[] = { + send_configuration, true), + DEFINE_PROP_BOOL("send-section-footer", MigrationState, + send_section_footer, true), ++ DEFINE_PROP_BOOL("decompress-error-check", MigrationState, ++ decompress_error_check, true), + + /* Migration parameters */ + DEFINE_PROP_UINT8("x-compress-level", MigrationState, +diff --git a/migration/migration.h b/migration/migration.h +index 06833d7..a9c5c7f 100644 +--- a/migration/migration.h ++++ b/migration/migration.h +@@ -182,6 +182,14 @@ struct MigrationState + bool send_configuration; + /* Whether we send section footer during migration */ + bool send_section_footer; ++ ++ /* ++ * Whether we abort the migration if decompression errors are ++ * detected at the destination. It is left at false for qemu ++ * older than 3.0, since only newer qemu sends streams that ++ * do not trigger spurious decompression errors. ++ */ ++ bool decompress_error_check; + }; + + void migrate_set_state(int *state, int old_state, int new_state); +diff --git a/migration/ram.c b/migration/ram.c +index c982201..bd563b5 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -2582,7 +2582,7 @@ static void *do_data_decompress(void *opaque) + + ret = qemu_uncompress_data(¶m->stream, des, pagesize, + param->compbuf, len); +- if (ret < 0) { ++ if (ret < 0 && migrate_get_current()->decompress_error_check) { + error_report("decompress data failed"); + qemu_file_set_error(decomp_file, ret); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-introduce-save_normal_page.patch b/SOURCES/kvm-migration-introduce-save_normal_page.patch new file mode 100644 index 0000000..ff76d94 --- /dev/null +++ b/SOURCES/kvm-migration-introduce-save_normal_page.patch @@ -0,0 +1,108 @@ +From d66f4034a5aa54082d467584d90db79c52bd001c Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:13 +0100 +Subject: [PATCH 09/21] migration: introduce save_normal_page() + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-10-dgilbert@redhat.com> +Patchwork-id: 81566 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 09/18] migration: introduce save_normal_page() +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +It directly sends the page to the stream neither checking zero nor +using xbzrle or compression + +Reviewed-by: Peter Xu +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-10-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 65dacaa04fa7e6104cbcee9251c7845355769a10) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 50 ++++++++++++++++++++++++++++++-------------------- + 1 file changed, 30 insertions(+), 20 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 6e8a7e2..908879f 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1013,6 +1013,34 @@ static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t offset, + return true; + } + ++/* ++ * directly send the page to the stream ++ * ++ * Returns the number of pages written. ++ * ++ * @rs: current RAM state ++ * @block: block that contains the page we want to send ++ * @offset: offset inside the block for the page ++ * @buf: the page to be sent ++ * @async: send to page asyncly ++ */ ++static int save_normal_page(RAMState *rs, RAMBlock *block, ram_addr_t offset, ++ uint8_t *buf, bool async) ++{ ++ ram_counters.transferred += save_page_header(rs, rs->f, block, ++ offset | RAM_SAVE_FLAG_PAGE); ++ if (async) { ++ qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE, ++ migrate_release_ram() & ++ migration_in_postcopy()); ++ } else { ++ qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE); ++ } ++ ram_counters.transferred += TARGET_PAGE_SIZE; ++ ram_counters.normal++; ++ return 1; ++} ++ + /** + * ram_save_page: send the given page to the stream + * +@@ -1053,18 +1081,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) + + /* XBZRLE overflow or normal page */ + if (pages == -1) { +- ram_counters.transferred += +- save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_PAGE); +- if (send_async) { +- qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, +- migrate_release_ram() & +- migration_in_postcopy()); +- } else { +- qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE); +- } +- ram_counters.transferred += TARGET_PAGE_SIZE; +- pages = 1; +- ram_counters.normal++; ++ pages = save_normal_page(rs, block, offset, p, send_async); + } + + XBZRLE_cache_unlock(); +@@ -1195,14 +1212,7 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, + * we post it as normal page as compression will take much + * CPU resource. + */ +- ram_counters.transferred += save_page_header(rs, rs->f, block, +- offset | RAM_SAVE_FLAG_PAGE); +- qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, +- migrate_release_ram() & +- migration_in_postcopy()); +- ram_counters.transferred += TARGET_PAGE_SIZE; +- ram_counters.normal++; +- pages = 1; ++ pages = save_normal_page(rs, block, offset, p, true); + } else { + pages = compress_page_with_multi_thread(rs, block, offset); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-move-calling-control_save_page-to-the-comm.patch b/SOURCES/kvm-migration-move-calling-control_save_page-to-the-comm.patch new file mode 100644 index 0000000..7cfcc2a --- /dev/null +++ b/SOURCES/kvm-migration-move-calling-control_save_page-to-the-comm.patch @@ -0,0 +1,74 @@ +From c378bddd4b750773a7e8e4987806d08248bc239d Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:11 +0100 +Subject: [PATCH 07/21] migration: move calling control_save_page to the common + place + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-8-dgilbert@redhat.com> +Patchwork-id: 81580 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 07/18] migration: move calling control_save_page to the common place +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +The function is called by both ram_save_page and ram_save_target_page, +so move it to the common caller to cleanup the code + +Reviewed-by: Peter Xu +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-8-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit a8ec91f941c5f83123796331c09333d3557eb5fc) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 16 ++++++++-------- + 1 file changed, 8 insertions(+), 8 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 106fcf1..9d6c41c 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1038,10 +1038,6 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) + p = block->host + offset; + trace_ram_save_page(block->idstr, (uint64_t)offset, p); + +- if (control_save_page(rs, block, offset, &pages)) { +- return pages; +- } +- + XBZRLE_cache_lock(); + pages = save_zero_page(rs, block, offset); + if (pages > 0) { +@@ -1199,10 +1195,6 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, + + p = block->host + offset; + +- if (control_save_page(rs, block, offset, &pages)) { +- return pages; +- } +- + /* When starting the process of a new block, the first page of + * the block should be sent out before other pages in the same + * block, and all the pages in last block should have been sent +@@ -1490,6 +1482,14 @@ err: + static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, + bool last_stage) + { ++ RAMBlock *block = pss->block; ++ ram_addr_t offset = pss->page << TARGET_PAGE_BITS; ++ int res; ++ ++ if (control_save_page(rs, block, offset, &res)) { ++ return res; ++ } ++ + /* + * If xbzrle is on, stop using the data compression after first + * round of migration even if compression is enabled. In theory, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-move-calling-save_zero_page-to-the-common-.patch b/SOURCES/kvm-migration-move-calling-save_zero_page-to-the-common-.patch new file mode 100644 index 0000000..a3c727f --- /dev/null +++ b/SOURCES/kvm-migration-move-calling-save_zero_page-to-the-common-.patch @@ -0,0 +1,175 @@ +From ea016aa3636ce101148bb6246d86474fc1cdcbf8 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:12 +0100 +Subject: [PATCH 08/21] migration: move calling save_zero_page to the common + place + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-9-dgilbert@redhat.com> +Patchwork-id: 81582 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 08/18] migration: move calling save_zero_page to the common place +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +save_zero_page() is always our first approach to try, move it to +the common place before calling ram_save_compressed_page +and ram_save_page + +Reviewed-by: Peter Xu +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-9-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit d7400a3409982a52ac451cd3ca9caee9db670ca7) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 105 +++++++++++++++++++++++++++++++------------------------- + 1 file changed, 59 insertions(+), 46 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 9d6c41c..6e8a7e2 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1039,15 +1039,8 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) + trace_ram_save_page(block->idstr, (uint64_t)offset, p); + + XBZRLE_cache_lock(); +- pages = save_zero_page(rs, block, offset); +- if (pages > 0) { +- /* Must let xbzrle know, otherwise a previous (now 0'd) cached +- * page would be stale +- */ +- xbzrle_cache_zero_page(rs, current_addr); +- ram_release_pages(block->idstr, offset, pages); +- } else if (!rs->ram_bulk_stage && +- !migration_in_postcopy() && migrate_use_xbzrle()) { ++ if (!rs->ram_bulk_stage && !migration_in_postcopy() && ++ migrate_use_xbzrle()) { + pages = save_xbzrle_page(rs, &p, current_addr, block, + offset, last_stage); + if (!last_stage) { +@@ -1195,40 +1188,23 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, + + p = block->host + offset; + +- /* When starting the process of a new block, the first page of +- * the block should be sent out before other pages in the same +- * block, and all the pages in last block should have been sent +- * out, keeping this order is important, because the 'cont' flag +- * is used to avoid resending the block name. +- */ + if (block != rs->last_sent_block) { +- flush_compressed_data(rs); +- pages = save_zero_page(rs, block, offset); +- if (pages > 0) { +- ram_release_pages(block->idstr, offset, pages); +- } else { +- /* +- * Make sure the first page is sent out before other pages. +- * +- * we post it as normal page as compression will take much +- * CPU resource. +- */ +- ram_counters.transferred += save_page_header(rs, rs->f, block, +- offset | RAM_SAVE_FLAG_PAGE); +- qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, +- migrate_release_ram() & +- migration_in_postcopy()); +- ram_counters.transferred += TARGET_PAGE_SIZE; +- ram_counters.normal++; +- pages = 1; +- } ++ /* ++ * Make sure the first page is sent out before other pages. ++ * ++ * we post it as normal page as compression will take much ++ * CPU resource. ++ */ ++ ram_counters.transferred += save_page_header(rs, rs->f, block, ++ offset | RAM_SAVE_FLAG_PAGE); ++ qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, ++ migrate_release_ram() & ++ migration_in_postcopy()); ++ ram_counters.transferred += TARGET_PAGE_SIZE; ++ ram_counters.normal++; ++ pages = 1; + } else { +- pages = save_zero_page(rs, block, offset); +- if (pages == -1) { +- pages = compress_page_with_multi_thread(rs, block, offset); +- } else { +- ram_release_pages(block->idstr, offset, pages); +- } ++ pages = compress_page_with_multi_thread(rs, block, offset); + } + + return pages; +@@ -1470,6 +1446,24 @@ err: + return -1; + } + ++static bool save_page_use_compression(RAMState *rs) ++{ ++ if (!migrate_use_compression()) { ++ return false; ++ } ++ ++ /* ++ * If xbzrle is on, stop using the data compression after first ++ * round of migration even if compression is enabled. In theory, ++ * xbzrle can do better than compression. ++ */ ++ if (rs->ram_bulk_stage || !migrate_use_xbzrle()) { ++ return true; ++ } ++ ++ return false; ++} ++ + /** + * ram_save_target_page: save one target page + * +@@ -1491,12 +1485,31 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, + } + + /* +- * If xbzrle is on, stop using the data compression after first +- * round of migration even if compression is enabled. In theory, +- * xbzrle can do better than compression. ++ * When starting the process of a new block, the first page of ++ * the block should be sent out before other pages in the same ++ * block, and all the pages in last block should have been sent ++ * out, keeping this order is important, because the 'cont' flag ++ * is used to avoid resending the block name. + */ +- if (migrate_use_compression() && +- (rs->ram_bulk_stage || !migrate_use_xbzrle())) { ++ if (block != rs->last_sent_block && save_page_use_compression(rs)) { ++ flush_compressed_data(rs); ++ } ++ ++ res = save_zero_page(rs, block, offset); ++ if (res > 0) { ++ /* Must let xbzrle know, otherwise a previous (now 0'd) cached ++ * page would be stale ++ */ ++ if (!save_page_use_compression(rs)) { ++ XBZRLE_cache_lock(); ++ xbzrle_cache_zero_page(rs, block->offset + offset); ++ XBZRLE_cache_unlock(); ++ } ++ ram_release_pages(block->idstr, offset, res); ++ return res; ++ } ++ ++ if (save_page_use_compression(rs)) { + return ram_save_compressed_page(rs, pss, last_stage); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-move-some-code-to-ram_save_host_page.patch b/SOURCES/kvm-migration-move-some-code-to-ram_save_host_page.patch new file mode 100644 index 0000000..1c18794 --- /dev/null +++ b/SOURCES/kvm-migration-move-some-code-to-ram_save_host_page.patch @@ -0,0 +1,108 @@ +From 35790a371aa43b6cc357bc78ee07dd20db16ec4b Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:10 +0100 +Subject: [PATCH 06/21] migration: move some code to ram_save_host_page + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-7-dgilbert@redhat.com> +Patchwork-id: 81570 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 06/18] migration: move some code to ram_save_host_page +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Move some code from ram_save_target_page() to ram_save_host_page() +to make it be more readable for latter patches that dramatically +clean ram_save_target_page() up + +Reviewed-by: Peter Xu +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-7-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 1faa5665c0f1df2eff291454a3a85625a3bc93dd) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 43 +++++++++++++++++++------------------------ + 1 file changed, 19 insertions(+), 24 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 8dc98a5..106fcf1 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1484,38 +1484,23 @@ err: + * Returns the number of pages written + * + * @rs: current RAM state +- * @ms: current migration state + * @pss: data about the page we want to send + * @last_stage: if we are at the completion stage + */ + static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, + bool last_stage) + { +- int res = 0; +- +- /* Check the pages is dirty and if it is send it */ +- if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { +- /* +- * If xbzrle is on, stop using the data compression after first +- * round of migration even if compression is enabled. In theory, +- * xbzrle can do better than compression. +- */ +- if (migrate_use_compression() && +- (rs->ram_bulk_stage || !migrate_use_xbzrle())) { +- res = ram_save_compressed_page(rs, pss, last_stage); +- } else { +- res = ram_save_page(rs, pss, last_stage); +- } +- +- if (res < 0) { +- return res; +- } +- if (pss->block->unsentmap) { +- clear_bit(pss->page, pss->block->unsentmap); +- } ++ /* ++ * If xbzrle is on, stop using the data compression after first ++ * round of migration even if compression is enabled. In theory, ++ * xbzrle can do better than compression. ++ */ ++ if (migrate_use_compression() && ++ (rs->ram_bulk_stage || !migrate_use_xbzrle())) { ++ return ram_save_compressed_page(rs, pss, last_stage); + } + +- return res; ++ return ram_save_page(rs, pss, last_stage); + } + + /** +@@ -1544,12 +1529,22 @@ static int ram_save_host_page(RAMState *rs, PageSearchStatus *pss, + qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS; + + do { ++ /* Check the pages is dirty and if it is send it */ ++ if (!migration_bitmap_clear_dirty(rs, pss->block, pss->page)) { ++ pss->page++; ++ continue; ++ } ++ + tmppages = ram_save_target_page(rs, pss, last_stage); + if (tmppages < 0) { + return tmppages; + } + + pages += tmppages; ++ if (pss->block->unsentmap) { ++ clear_bit(pss->page, pss->block->unsentmap); ++ } ++ + pss->page++; + } while ((pss->page & (pagesize_bits - 1)) && + offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS)); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-not-wait-RDMA_CM_EVENT_DISCONNECTED-event-.patch b/SOURCES/kvm-migration-not-wait-RDMA_CM_EVENT_DISCONNECTED-event-.patch new file mode 100644 index 0000000..ede40c1 --- /dev/null +++ b/SOURCES/kvm-migration-not-wait-RDMA_CM_EVENT_DISCONNECTED-event-.patch @@ -0,0 +1,109 @@ +From 3ee3fef23ff91d6bf974820b4dbe8280c6ad27b2 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Fri, 22 Jun 2018 19:00:04 +0200 +Subject: [PATCH 16/21] migration: not wait RDMA_CM_EVENT_DISCONNECTED event + after rdma_disconnect + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-18-dgilbert@redhat.com> +Patchwork-id: 81572 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 17/18] migration: not wait RDMA_CM_EVENT_DISCONNECTED event after rdma_disconnect +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Lidong Chen + +When cancel migration during RDMA precopy, the source qemu main thread hangs sometime. + +The backtrace is: + (gdb) bt + #0 0x00007f249eabd43d in write () from /lib64/libpthread.so.0 + #1 0x00007f24a1ce98e4 in rdma_get_cm_event (channel=0x4675d10, event=0x7ffe2f643dd0) at src/cma.c:2189 + #2 0x00000000007b6166 in qemu_rdma_cleanup (rdma=0x6784000) at migration/rdma.c:2296 + #3 0x00000000007b7cae in qio_channel_rdma_close (ioc=0x3bfcc30, errp=0x0) at migration/rdma.c:2999 + #4 0x00000000008db60e in qio_channel_close (ioc=0x3bfcc30, errp=0x0) at io/channel.c:273 + #5 0x00000000007a8765 in channel_close (opaque=0x3bfcc30) at migration/qemu-file-channel.c:98 + #6 0x00000000007a71f9 in qemu_fclose (f=0x527c000) at migration/qemu-file.c:334 + #7 0x0000000000795b96 in migrate_fd_cleanup (opaque=0x3b46280) at migration/migration.c:1162 + #8 0x000000000093a71b in aio_bh_call (bh=0x3db7a20) at util/async.c:90 + #9 0x000000000093a7b2 in aio_bh_poll (ctx=0x3b121c0) at util/async.c:118 + #10 0x000000000093f2ad in aio_dispatch (ctx=0x3b121c0) at util/aio-posix.c:436 + #11 0x000000000093ab41 in aio_ctx_dispatch (source=0x3b121c0, callback=0x0, user_data=0x0) + at util/async.c:261 + #12 0x00007f249f73c7aa in g_main_context_dispatch () from /lib64/libglib-2.0.so.0 + #13 0x000000000093dc5e in glib_pollfds_poll () at util/main-loop.c:215 + #14 0x000000000093dd4e in os_host_main_loop_wait (timeout=28000000) at util/main-loop.c:263 + #15 0x000000000093de05 in main_loop_wait (nonblocking=0) at util/main-loop.c:522 + #16 0x00000000005bc6a5 in main_loop () at vl.c:1944 + #17 0x00000000005c39b5 in main (argc=56, argv=0x7ffe2f6443f8, envp=0x3ad0030) at vl.c:4752 + +It does not get the RDMA_CM_EVENT_DISCONNECTED event after rdma_disconnect sometime. + +According to IB Spec once active side send DREQ message, it should wait for DREP message +and only once it arrived it should trigger a DISCONNECT event. DREP message can be dropped +due to network issues. +For that case the spec defines a DREP_timeout state in the CM state machine, if the DREP is +dropped we should get a timeout and a TIMEWAIT_EXIT event will be trigger. +Unfortunately the current kernel CM implementation doesn't include the DREP_timeout state +and in above scenario we will not get DISCONNECT or TIMEWAIT_EXIT events. + +So it should not invoke rdma_get_cm_event which may hang forever, and the event channel +is also destroyed in qemu_rdma_cleanup. + +Signed-off-by: Lidong Chen +Reviewed-by: Juan Quintela +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Juan Quintela +(cherry picked from commit c5e76115ccb4979cec795a8ae38becd07c2fde9f) +--- + migration/rdma.c | 12 ++---------- + migration/trace-events | 1 - + 2 files changed, 2 insertions(+), 11 deletions(-) + +diff --git a/migration/rdma.c b/migration/rdma.c +index 7d233b0..6e29ebf 100644 +--- a/migration/rdma.c ++++ b/migration/rdma.c +@@ -2268,8 +2268,7 @@ static int qemu_rdma_write(QEMUFile *f, RDMAContext *rdma, + + static void qemu_rdma_cleanup(RDMAContext *rdma) + { +- struct rdma_cm_event *cm_event; +- int ret, idx; ++ int idx; + + if (rdma->cm_id && rdma->connected) { + if ((rdma->error_state || +@@ -2283,14 +2282,7 @@ static void qemu_rdma_cleanup(RDMAContext *rdma) + qemu_rdma_post_send_control(rdma, NULL, &head); + } + +- ret = rdma_disconnect(rdma->cm_id); +- if (!ret) { +- trace_qemu_rdma_cleanup_waiting_for_disconnect(); +- ret = rdma_get_cm_event(rdma->channel, &cm_event); +- if (!ret) { +- rdma_ack_cm_event(cm_event); +- } +- } ++ rdma_disconnect(rdma->cm_id); + trace_qemu_rdma_cleanup_disconnect(); + rdma->connected = false; + } +diff --git a/migration/trace-events b/migration/trace-events +index a180d7b..92b3179 100644 +--- a/migration/trace-events ++++ b/migration/trace-events +@@ -123,7 +123,6 @@ qemu_rdma_accept_pin_state(bool pin) "%d" + qemu_rdma_accept_pin_verbsc(void *verbs) "Verbs context after listen: %p" + qemu_rdma_block_for_wrid_miss(const char *wcompstr, int wcomp, const char *gcompstr, uint64_t req) "A Wanted wrid %s (%d) but got %s (%" PRIu64 ")" + qemu_rdma_cleanup_disconnect(void) "" +-qemu_rdma_cleanup_waiting_for_disconnect(void) "" + qemu_rdma_close(void) "" + qemu_rdma_connect_pin_all_requested(void) "" + qemu_rdma_connect_pin_all_outcome(bool pin) "%d" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-postcopy-Clear-have_listen_thread.patch b/SOURCES/kvm-migration-postcopy-Clear-have_listen_thread.patch new file mode 100644 index 0000000..444f5b0 --- /dev/null +++ b/SOURCES/kvm-migration-postcopy-Clear-have_listen_thread.patch @@ -0,0 +1,51 @@ +From 66e37a444b4b4818957dabadcc4580f1877e4ebb Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Mon, 1 Oct 2018 10:54:48 +0100 +Subject: [PATCH 02/28] migration/postcopy: Clear have_listen_thread + +RH-Author: Dr. David Alan Gilbert +Message-id: <20181001105449.41090-2-dgilbert@redhat.com> +Patchwork-id: 82326 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 1/2] migration/postcopy: Clear have_listen_thread +Bugzilla: 1608765 +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Laurent Vivier + +From: "Dr. David Alan Gilbert" + +Clear have_listen_thread when we exit the thread. +The fallout from this was that various things thought there was +an ongoing postcopy after the postcopy had finished. + +The case that failed was postcopy->savevm->loadvm. + +This corresponds to RH bug https://bugzilla.redhat.com/show_bug.cgi?id=1608765 + +Signed-off-by: Dr. David Alan Gilbert +Message-Id: <20180914170430.54271-2-dgilbert@redhat.com> +Reviewed-by: Peter Xu +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 9cf4bb8730c669c40550e635a9e2b8ee4f1664ca) + Manual merge due to context + +Signed-off-by: Danilo C. L. de Paula +--- + migration/savevm.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/migration/savevm.c b/migration/savevm.c +index 6c539d1..6a8d363 100644 +--- a/migration/savevm.c ++++ b/migration/savevm.c +@@ -1620,6 +1620,7 @@ static void *postcopy_ram_listen_thread(void *opaque) + migration_incoming_state_destroy(); + qemu_loadvm_state_cleanup(); + ++ mis->have_listen_thread = false; + return NULL; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-ram-Add-check-and-info-message-to-nvdimm-p.patch b/SOURCES/kvm-migration-ram-Add-check-and-info-message-to-nvdimm-p.patch new file mode 100644 index 0000000..96a4e08 --- /dev/null +++ b/SOURCES/kvm-migration-ram-Add-check-and-info-message-to-nvdimm-p.patch @@ -0,0 +1,56 @@ +From 1f0ccebc1a1ed974fe13841c06742f52589c6064 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:21 +0000 +Subject: [PATCH 20/22] migration/ram: Add check and info message to nvdimm + post copy. + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-9-git-send-email-plai@redhat.com> +Patchwork-id: 83891 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 08/10] migration/ram: Add check and info message to nvdimm post copy. +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +The nvdimm kind memory does not support post copy now. +We disable post copy if we have nvdimm memory and print some +log hint to user. + +Signed-off-by: Junyan He +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 469dd51bc664979f159d47885997d482991394b8) +Signed-off-by: Paul Lai +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 9 +++++++++ + 1 file changed, 9 insertions(+) + +diff --git a/migration/ram.c b/migration/ram.c +index 04b5df5..f850fd0 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -3120,6 +3120,15 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) + + static bool ram_has_postcopy(void *opaque) + { ++ RAMBlock *rb; ++ RAMBLOCK_FOREACH_MIGRATABLE(rb) { ++ if (ramblock_is_pmem(rb)) { ++ info_report("Block: %s, host: %p is a nvdimm memory, postcopy" ++ "is not supported now!", rb->idstr, rb->host); ++ return false; ++ } ++ } ++ + return migrate_postcopy_ram(); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-ram-ensure-write-persistence-on-loading-al.patch b/SOURCES/kvm-migration-ram-ensure-write-persistence-on-loading-al.patch new file mode 100644 index 0000000..604dff9 --- /dev/null +++ b/SOURCES/kvm-migration-ram-ensure-write-persistence-on-loading-al.patch @@ -0,0 +1,81 @@ +From d06445e16ee0fa7d5ee4637aa686c48e88a9cb30 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:22 +0000 +Subject: [PATCH 21/22] migration/ram: ensure write persistence on loading all + data to PMEM. + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-10-git-send-email-plai@redhat.com> +Patchwork-id: 83894 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 09/10] migration/ram: ensure write persistence on loading all data to PMEM. +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Junyan He + +Because we need to make sure the pmem kind memory data is synced +after migration, we choose to call pmem_persist() when the migration +finish. This will make sure the data of pmem is safe and will not +lose if power is off. + +Signed-off-by: Junyan He +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Igor Mammedov +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 56eb90af39abf66c0e80588a9f50c31e7df7320b) +Signed-off-by: Paul Lai +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/pmem.h | 6 ++++++ + migration/ram.c | 8 ++++++++ + 2 files changed, 14 insertions(+) + +diff --git a/include/qemu/pmem.h b/include/qemu/pmem.h +index ebdb070..dfb6d0d 100644 +--- a/include/qemu/pmem.h ++++ b/include/qemu/pmem.h +@@ -25,6 +25,12 @@ pmem_memcpy_persist(void *pmemdest, const void *src, size_t len) + return NULL; + } + ++static inline void ++pmem_persist(const void *addr, size_t len) ++{ ++ g_assert_not_reached(); ++} ++ + #endif /* CONFIG_LIBPMEM */ + + #endif /* !QEMU_PMEM_H */ +diff --git a/migration/ram.c b/migration/ram.c +index f850fd0..aba0f70 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -33,6 +33,7 @@ + #include "qemu/bitops.h" + #include "qemu/bitmap.h" + #include "qemu/main-loop.h" ++#include "qemu/pmem.h" + #include "xbzrle.h" + #include "ram.h" + #include "migration.h" +@@ -2766,6 +2767,13 @@ static int ram_load_setup(QEMUFile *f, void *opaque) + static int ram_load_cleanup(void *opaque) + { + RAMBlock *rb; ++ ++ RAMBLOCK_FOREACH_MIGRATABLE(rb) { ++ if (ramblock_is_pmem(rb)) { ++ pmem_persist(rb->host, rb->used_length); ++ } ++ } ++ + xbzrle_load_cleanup(); + compress_threads_load_cleanup(); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-remove-ram_save_compressed_page.patch b/SOURCES/kvm-migration-remove-ram_save_compressed_page.patch new file mode 100644 index 0000000..b3cca6b --- /dev/null +++ b/SOURCES/kvm-migration-remove-ram_save_compressed_page.patch @@ -0,0 +1,97 @@ +From e3c0812c4d61c956ac5b0641826c4e4fbda55954 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:14 +0100 +Subject: [PATCH 10/21] migration: remove ram_save_compressed_page() + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-11-dgilbert@redhat.com> +Patchwork-id: 81578 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 10/18] migration: remove ram_save_compressed_page() +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Now, we can reuse the path in ram_save_page() to post the page out +as normal, then the only thing remained in ram_save_compressed_page() +is compression that we can move it out to the caller + +Reviewed-by: Peter Xu +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-11-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit da3f56cb2e767016d3f204837a77caf35b463f90) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 45 ++++++++------------------------------------- + 1 file changed, 8 insertions(+), 37 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 908879f..466609f 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1186,41 +1186,6 @@ static int compress_page_with_multi_thread(RAMState *rs, RAMBlock *block, + } + + /** +- * ram_save_compressed_page: compress the given page and send it to the stream +- * +- * Returns the number of pages written. +- * +- * @rs: current RAM state +- * @block: block that contains the page we want to send +- * @offset: offset inside the block for the page +- * @last_stage: if we are at the completion stage +- */ +-static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, +- bool last_stage) +-{ +- int pages = -1; +- uint8_t *p; +- RAMBlock *block = pss->block; +- ram_addr_t offset = pss->page << TARGET_PAGE_BITS; +- +- p = block->host + offset; +- +- if (block != rs->last_sent_block) { +- /* +- * Make sure the first page is sent out before other pages. +- * +- * we post it as normal page as compression will take much +- * CPU resource. +- */ +- pages = save_normal_page(rs, block, offset, p, true); +- } else { +- pages = compress_page_with_multi_thread(rs, block, offset); +- } +- +- return pages; +-} +- +-/** + * find_dirty_block: find the next dirty page and update any state + * associated with the search process. + * +@@ -1519,8 +1484,14 @@ static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss, + return res; + } + +- if (save_page_use_compression(rs)) { +- return ram_save_compressed_page(rs, pss, last_stage); ++ /* ++ * Make sure the first page is sent out before other pages. ++ * ++ * we post it as normal page as compression will take much ++ * CPU resource. ++ */ ++ if (block == rs->last_sent_block && save_page_use_compression(rs)) { ++ res = compress_page_with_multi_thread(rs, block, offset); + } + + return ram_save_page(rs, pss, last_stage); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-stop-compressing-page-in-migration-thread.patch b/SOURCES/kvm-migration-stop-compressing-page-in-migration-thread.patch new file mode 100644 index 0000000..5b05a90 --- /dev/null +++ b/SOURCES/kvm-migration-stop-compressing-page-in-migration-thread.patch @@ -0,0 +1,86 @@ +From 932d0761f1f473638a38614796bd14939cdd6215 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:05 +0100 +Subject: [PATCH 01/21] migration: stop compressing page in migration thread + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-2-dgilbert@redhat.com> +Patchwork-id: 81577 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 01/18] migration: stop compressing page in migration thread +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +As compression is a heavy work, do not do it in migration thread, +instead, we post it out as a normal page + +Reviewed-by: Wei Wang +Reviewed-by: Peter Xu +Reviewed-by: Dr. David Alan Gilbert +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-2-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 263a289ae61c8344a417a95b0142650fdff3af56) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 32 ++++++++++++++++---------------- + 1 file changed, 16 insertions(+), 16 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 00c06b5..f27038a 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -1138,7 +1138,7 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, + int pages = -1; + uint64_t bytes_xmit = 0; + uint8_t *p; +- int ret, blen; ++ int ret; + RAMBlock *block = pss->block; + ram_addr_t offset = pss->page << TARGET_PAGE_BITS; + +@@ -1168,23 +1168,23 @@ static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss, + if (block != rs->last_sent_block) { + flush_compressed_data(rs); + pages = save_zero_page(rs, block, offset); +- if (pages == -1) { +- /* Make sure the first page is sent out before other pages */ +- bytes_xmit = save_page_header(rs, rs->f, block, offset | +- RAM_SAVE_FLAG_COMPRESS_PAGE); +- blen = qemu_put_compression_data(rs->f, p, TARGET_PAGE_SIZE, +- migrate_compress_level()); +- if (blen > 0) { +- ram_counters.transferred += bytes_xmit + blen; +- ram_counters.normal++; +- pages = 1; +- } else { +- qemu_file_set_error(rs->f, blen); +- error_report("compressed data failed!"); +- } +- } + if (pages > 0) { + ram_release_pages(block->idstr, offset, pages); ++ } else { ++ /* ++ * Make sure the first page is sent out before other pages. ++ * ++ * we post it as normal page as compression will take much ++ * CPU resource. ++ */ ++ ram_counters.transferred += save_page_header(rs, rs->f, block, ++ offset | RAM_SAVE_FLAG_PAGE); ++ qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE, ++ migrate_release_ram() & ++ migration_in_postcopy()); ++ ram_counters.transferred += TARGET_PAGE_SIZE; ++ ram_counters.normal++; ++ pages = 1; + } + } else { + pages = save_zero_page(rs, block, offset); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-stop-compression-to-allocate-and-free-memo.patch b/SOURCES/kvm-migration-stop-compression-to-allocate-and-free-memo.patch new file mode 100644 index 0000000..e00fbbf --- /dev/null +++ b/SOURCES/kvm-migration-stop-compression-to-allocate-and-free-memo.patch @@ -0,0 +1,265 @@ +From d9c3b12460d40900dbc2c0f4b6b03886488178d6 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:06 +0100 +Subject: [PATCH 02/21] migration: stop compression to allocate and free memory + frequently + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-3-dgilbert@redhat.com> +Patchwork-id: 81579 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 02/18] migration: stop compression to allocate and free memory frequently +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Current code uses compress2() to compress memory which manages memory +internally, that causes huge memory is allocated and freed very +frequently + +More worse, frequently returning memory to kernel will flush TLBs +and trigger invalidation callbacks on mmu-notification which +interacts with KVM MMU, that dramatically reduce the performance +of VM + +So, we maintain the memory by ourselves and reuse it for each +compression + +Reviewed-by: Peter Xu +Reviewed-by: Jiang Biao +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-3-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit dcaf446ebda5d87e05eb41cdbafb7ae4a7cc4a62) +Signed-off-by: Danilo C. L. de Paula +--- + migration/qemu-file.c | 39 ++++++++++++++++++++++++++++++++------- + migration/qemu-file.h | 6 ++++-- + migration/ram.c | 41 ++++++++++++++++++++++++++++++++--------- + 3 files changed, 68 insertions(+), 18 deletions(-) + +diff --git a/migration/qemu-file.c b/migration/qemu-file.c +index bb63c77..bafe3a0 100644 +--- a/migration/qemu-file.c ++++ b/migration/qemu-file.c +@@ -658,8 +658,32 @@ uint64_t qemu_get_be64(QEMUFile *f) + return v; + } + +-/* Compress size bytes of data start at p with specific compression +- * level and store the compressed data to the buffer of f. ++/* return the size after compression, or negative value on error */ ++static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len, ++ const uint8_t *source, size_t source_len) ++{ ++ int err; ++ ++ err = deflateReset(stream); ++ if (err != Z_OK) { ++ return -1; ++ } ++ ++ stream->avail_in = source_len; ++ stream->next_in = (uint8_t *)source; ++ stream->avail_out = dest_len; ++ stream->next_out = dest; ++ ++ err = deflate(stream, Z_FINISH); ++ if (err != Z_STREAM_END) { ++ return -1; ++ } ++ ++ return stream->next_out - dest; ++} ++ ++/* Compress size bytes of data start at p and store the compressed ++ * data to the buffer of f. + * + * When f is not writable, return -1 if f has no space to save the + * compressed data. +@@ -667,9 +691,8 @@ uint64_t qemu_get_be64(QEMUFile *f) + * do fflush first, if f still has no space to save the compressed + * data, return -1. + */ +- +-ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size, +- int level) ++ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, ++ const uint8_t *p, size_t size) + { + ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t); + +@@ -683,8 +706,10 @@ ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size, + return -1; + } + } +- if (compress2(f->buf + f->buf_index + sizeof(int32_t), (uLongf *)&blen, +- (Bytef *)p, size, level) != Z_OK) { ++ ++ blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t), ++ blen, p, size); ++ if (blen < 0) { + error_report("Compress Failed!"); + return 0; + } +diff --git a/migration/qemu-file.h b/migration/qemu-file.h +index f4f356a..2ccfcfb 100644 +--- a/migration/qemu-file.h ++++ b/migration/qemu-file.h +@@ -25,6 +25,8 @@ + #ifndef MIGRATION_QEMU_FILE_H + #define MIGRATION_QEMU_FILE_H + ++#include ++ + /* Read a chunk of data from a file at the given position. The pos argument + * can be ignored if the file is only be used for streaming. The number of + * bytes actually read should be returned. +@@ -132,8 +134,8 @@ bool qemu_file_is_writable(QEMUFile *f); + + size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t offset); + size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size); +-ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size, +- int level); ++ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream, ++ const uint8_t *p, size_t size); + int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src); + + /* +diff --git a/migration/ram.c b/migration/ram.c +index f27038a..7d3b1da 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -269,6 +269,7 @@ struct CompressParam { + QemuCond cond; + RAMBlock *block; + ram_addr_t offset; ++ z_stream stream; + }; + typedef struct CompressParam CompressParam; + +@@ -299,7 +300,7 @@ static QemuThread *decompress_threads; + static QemuMutex decomp_done_lock; + static QemuCond decomp_done_cond; + +-static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, ++static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, + ram_addr_t offset); + + static void *do_data_compress(void *opaque) +@@ -316,7 +317,7 @@ static void *do_data_compress(void *opaque) + param->block = NULL; + qemu_mutex_unlock(¶m->mutex); + +- do_compress_ram_page(param->file, block, offset); ++ do_compress_ram_page(param->file, ¶m->stream, block, offset); + + qemu_mutex_lock(&comp_done_lock); + param->done = true; +@@ -357,10 +358,19 @@ static void compress_threads_save_cleanup(void) + terminate_compression_threads(); + thread_count = migrate_compress_threads(); + for (i = 0; i < thread_count; i++) { ++ /* ++ * we use it as a indicator which shows if the thread is ++ * properly init'd or not ++ */ ++ if (!comp_param[i].file) { ++ break; ++ } + qemu_thread_join(compress_threads + i); +- qemu_fclose(comp_param[i].file); + qemu_mutex_destroy(&comp_param[i].mutex); + qemu_cond_destroy(&comp_param[i].cond); ++ deflateEnd(&comp_param[i].stream); ++ qemu_fclose(comp_param[i].file); ++ comp_param[i].file = NULL; + } + qemu_mutex_destroy(&comp_done_lock); + qemu_cond_destroy(&comp_done_cond); +@@ -370,12 +380,12 @@ static void compress_threads_save_cleanup(void) + comp_param = NULL; + } + +-static void compress_threads_save_setup(void) ++static int compress_threads_save_setup(void) + { + int i, thread_count; + + if (!migrate_use_compression()) { +- return; ++ return 0; + } + thread_count = migrate_compress_threads(); + compress_threads = g_new0(QemuThread, thread_count); +@@ -383,6 +393,11 @@ static void compress_threads_save_setup(void) + qemu_cond_init(&comp_done_cond); + qemu_mutex_init(&comp_done_lock); + for (i = 0; i < thread_count; i++) { ++ if (deflateInit(&comp_param[i].stream, ++ migrate_compress_level()) != Z_OK) { ++ goto exit; ++ } ++ + /* comp_param[i].file is just used as a dummy buffer to save data, + * set its ops to empty. + */ +@@ -395,6 +410,11 @@ static void compress_threads_save_setup(void) + do_data_compress, comp_param + i, + QEMU_THREAD_JOINABLE); + } ++ return 0; ++ ++exit: ++ compress_threads_save_cleanup(); ++ return -1; + } + + /* Multiple fd's */ +@@ -1032,7 +1052,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage) + return pages; + } + +-static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, ++static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block, + ram_addr_t offset) + { + RAMState *rs = ram_state; +@@ -1041,8 +1061,7 @@ static int do_compress_ram_page(QEMUFile *f, RAMBlock *block, + + bytes_sent = save_page_header(rs, f, block, offset | + RAM_SAVE_FLAG_COMPRESS_PAGE); +- blen = qemu_put_compression_data(f, p, TARGET_PAGE_SIZE, +- migrate_compress_level()); ++ blen = qemu_put_compression_data(f, stream, p, TARGET_PAGE_SIZE); + if (blen < 0) { + bytes_sent = 0; + qemu_file_set_error(migrate_get_current()->to_dst_file, blen); +@@ -2215,9 +2234,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque) + RAMState **rsp = opaque; + RAMBlock *block; + ++ if (compress_threads_save_setup()) { ++ return -1; ++ } ++ + /* migration has already setup the bitmap, reuse it. */ + if (!migration_in_colo_state()) { + if (ram_init_all(rsp) != 0) { ++ compress_threads_save_cleanup(); + return -1; + } + } +@@ -2237,7 +2261,6 @@ static int ram_save_setup(QEMUFile *f, void *opaque) + } + + rcu_read_unlock(); +- compress_threads_save_setup(); + + ram_control_before_iterate(f, RAM_CONTROL_SETUP); + ram_control_after_iterate(f, RAM_CONTROL_SETUP); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-stop-decompression-to-allocate-and-free-me.patch b/SOURCES/kvm-migration-stop-decompression-to-allocate-and-free-me.patch new file mode 100644 index 0000000..0f491bf --- /dev/null +++ b/SOURCES/kvm-migration-stop-decompression-to-allocate-and-free-me.patch @@ -0,0 +1,217 @@ +From 2caad7e48038f5303651bde85b559b3ad19dd262 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:07 +0100 +Subject: [PATCH 03/21] migration: stop decompression to allocate and free + memory frequently + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-4-dgilbert@redhat.com> +Patchwork-id: 81571 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 03/18] migration: stop decompression to allocate and free memory frequently +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Xiao Guangrong + +Current code uses uncompress() to decompress memory which manages +memory internally, that causes huge memory is allocated and freed +very frequently, more worse, frequently returning memory to kernel +will flush TLBs + +So, we maintain the memory by ourselves and reuse it for each +decompression + +Reviewed-by: Peter Xu +Reviewed-by: Jiang Biao +Signed-off-by: Xiao Guangrong +Message-Id: <20180330075128.26919-4-xiaoguangrong@tencent.com> +Signed-off-by: Dr. David Alan Gilbert +(cherry picked from commit 797ca154b4c68dbd8e93382f714388ab311f672d) +Signed-off-by: Danilo C. L. de Paula +--- + migration/ram.c | 112 +++++++++++++++++++++++++++++++++++++++++--------------- + 1 file changed, 82 insertions(+), 30 deletions(-) + +diff --git a/migration/ram.c b/migration/ram.c +index 7d3b1da..be89cd8 100644 +--- a/migration/ram.c ++++ b/migration/ram.c +@@ -281,6 +281,7 @@ struct DecompressParam { + void *des; + uint8_t *compbuf; + int len; ++ z_stream stream; + }; + typedef struct DecompressParam DecompressParam; + +@@ -2525,6 +2526,31 @@ void ram_handle_compressed(void *host, uint8_t ch, uint64_t size) + } + } + ++/* return the size after decompression, or negative value on error */ ++static int ++qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len, ++ const uint8_t *source, size_t source_len) ++{ ++ int err; ++ ++ err = inflateReset(stream); ++ if (err != Z_OK) { ++ return -1; ++ } ++ ++ stream->avail_in = source_len; ++ stream->next_in = (uint8_t *)source; ++ stream->avail_out = dest_len; ++ stream->next_out = dest; ++ ++ err = inflate(stream, Z_NO_FLUSH); ++ if (err != Z_STREAM_END) { ++ return -1; ++ } ++ ++ return stream->total_out; ++} ++ + static void *do_data_decompress(void *opaque) + { + DecompressParam *param = opaque; +@@ -2541,13 +2567,13 @@ static void *do_data_decompress(void *opaque) + qemu_mutex_unlock(¶m->mutex); + + pagesize = TARGET_PAGE_SIZE; +- /* uncompress() will return failed in some case, especially +- * when the page is dirted when doing the compression, it's +- * not a problem because the dirty page will be retransferred ++ /* qemu_uncompress_data() will return failed in some case, ++ * especially when the page is dirtied when doing the compression, ++ * it's not a problem because the dirty page will be retransferred + * and uncompress() won't break the data in other pages. + */ +- uncompress((Bytef *)des, &pagesize, +- (const Bytef *)param->compbuf, len); ++ qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf, ++ len); + + qemu_mutex_lock(&decomp_done_lock); + param->done = true; +@@ -2582,30 +2608,6 @@ static void wait_for_decompress_done(void) + qemu_mutex_unlock(&decomp_done_lock); + } + +-static void compress_threads_load_setup(void) +-{ +- int i, thread_count; +- +- if (!migrate_use_compression()) { +- return; +- } +- thread_count = migrate_decompress_threads(); +- decompress_threads = g_new0(QemuThread, thread_count); +- decomp_param = g_new0(DecompressParam, thread_count); +- qemu_mutex_init(&decomp_done_lock); +- qemu_cond_init(&decomp_done_cond); +- for (i = 0; i < thread_count; i++) { +- qemu_mutex_init(&decomp_param[i].mutex); +- qemu_cond_init(&decomp_param[i].cond); +- decomp_param[i].compbuf = g_malloc0(compressBound(TARGET_PAGE_SIZE)); +- decomp_param[i].done = true; +- decomp_param[i].quit = false; +- qemu_thread_create(decompress_threads + i, "decompress", +- do_data_decompress, decomp_param + i, +- QEMU_THREAD_JOINABLE); +- } +-} +- + static void compress_threads_load_cleanup(void) + { + int i, thread_count; +@@ -2615,16 +2617,30 @@ static void compress_threads_load_cleanup(void) + } + thread_count = migrate_decompress_threads(); + for (i = 0; i < thread_count; i++) { ++ /* ++ * we use it as a indicator which shows if the thread is ++ * properly init'd or not ++ */ ++ if (!decomp_param[i].compbuf) { ++ break; ++ } ++ + qemu_mutex_lock(&decomp_param[i].mutex); + decomp_param[i].quit = true; + qemu_cond_signal(&decomp_param[i].cond); + qemu_mutex_unlock(&decomp_param[i].mutex); + } + for (i = 0; i < thread_count; i++) { ++ if (!decomp_param[i].compbuf) { ++ break; ++ } ++ + qemu_thread_join(decompress_threads + i); + qemu_mutex_destroy(&decomp_param[i].mutex); + qemu_cond_destroy(&decomp_param[i].cond); ++ inflateEnd(&decomp_param[i].stream); + g_free(decomp_param[i].compbuf); ++ decomp_param[i].compbuf = NULL; + } + g_free(decompress_threads); + g_free(decomp_param); +@@ -2632,6 +2648,39 @@ static void compress_threads_load_cleanup(void) + decomp_param = NULL; + } + ++static int compress_threads_load_setup(void) ++{ ++ int i, thread_count; ++ ++ if (!migrate_use_compression()) { ++ return 0; ++ } ++ ++ thread_count = migrate_decompress_threads(); ++ decompress_threads = g_new0(QemuThread, thread_count); ++ decomp_param = g_new0(DecompressParam, thread_count); ++ qemu_mutex_init(&decomp_done_lock); ++ qemu_cond_init(&decomp_done_cond); ++ for (i = 0; i < thread_count; i++) { ++ if (inflateInit(&decomp_param[i].stream) != Z_OK) { ++ goto exit; ++ } ++ ++ decomp_param[i].compbuf = g_malloc0(compressBound(TARGET_PAGE_SIZE)); ++ qemu_mutex_init(&decomp_param[i].mutex); ++ qemu_cond_init(&decomp_param[i].cond); ++ decomp_param[i].done = true; ++ decomp_param[i].quit = false; ++ qemu_thread_create(decompress_threads + i, "decompress", ++ do_data_decompress, decomp_param + i, ++ QEMU_THREAD_JOINABLE); ++ } ++ return 0; ++exit: ++ compress_threads_load_cleanup(); ++ return -1; ++} ++ + static void decompress_data_with_multi_threads(QEMUFile *f, + void *host, int len) + { +@@ -2671,8 +2720,11 @@ static void decompress_data_with_multi_threads(QEMUFile *f, + */ + static int ram_load_setup(QEMUFile *f, void *opaque) + { ++ if (compress_threads_load_setup()) { ++ return -1; ++ } ++ + xbzrle_load_setup(); +- compress_threads_load_setup(); + ramblock_recv_map_init(); + return 0; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-migration-update-index-field-when-delete-or-qsort-RD.patch b/SOURCES/kvm-migration-update-index-field-when-delete-or-qsort-RD.patch new file mode 100644 index 0000000..3c3e3a8 --- /dev/null +++ b/SOURCES/kvm-migration-update-index-field-when-delete-or-qsort-RD.patch @@ -0,0 +1,66 @@ +From 10aadee745539521945b4d6482717011eeb15fc2 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Wed, 1 Aug 2018 13:55:17 +0100 +Subject: [PATCH 13/21] migration: update index field when delete or qsort + RDMALocalBlock + +RH-Author: Dr. David Alan Gilbert +Message-id: <20180801135522.11658-14-dgilbert@redhat.com> +Patchwork-id: 81574 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 13/18] migration: update index field when delete or qsort RDMALocalBlock +Bugzilla: 1594384 +RH-Acked-by: Peter Xu +RH-Acked-by: John Snow +RH-Acked-by: Juan Quintela + +From: Lidong Chen + +rdma_delete_block function deletes RDMALocalBlock base on index field, +but not update the index field. So when next time invoke rdma_delete_block, +it will not work correctly. + +If start and cancel migration repeatedly, some RDMALocalBlock not invoke +ibv_dereg_mr to decrease kernel mm_struct vmpin. When vmpin is large than +max locked memory limitation, ibv_reg_mr will failed, and migration can not +start successfully again. + +Signed-off-by: Lidong Chen +Reviewed-by: Dr. David Alan Gilbert +Message-Id: <1525618499-1560-1-git-send-email-lidongchen@tencent.com> +Signed-off-by: Juan Quintela + +Signed-off-by: Lidong Chen +(cherry picked from commit 71cd73061c014d04bc6b54936e675347ebc8d964) +Signed-off-by: Danilo C. L. de Paula +--- + migration/rdma.c | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/migration/rdma.c b/migration/rdma.c +index da474fc..7d233b0 100644 +--- a/migration/rdma.c ++++ b/migration/rdma.c +@@ -708,6 +708,9 @@ static int rdma_delete_block(RDMAContext *rdma, RDMALocalBlock *block) + memcpy(local->block + block->index, old + (block->index + 1), + sizeof(RDMALocalBlock) * + (local->nb_blocks - (block->index + 1))); ++ for (x = block->index; x < local->nb_blocks - 1; x++) { ++ local->block[x].index--; ++ } + } + } else { + assert(block == local->block); +@@ -3246,6 +3249,10 @@ static int qemu_rdma_registration_handle(QEMUFile *f, void *opaque) + qsort(rdma->local_ram_blocks.block, + rdma->local_ram_blocks.nb_blocks, + sizeof(RDMALocalBlock), dest_ram_sort_func); ++ for (i = 0; i < local->nb_blocks; i++) { ++ local->block[i].index = i; ++ } ++ + if (rdma->pin_all) { + ret = qemu_rdma_reg_whole_ram_blocks(rdma); + if (ret) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-mirror-Fail-gracefully-for-source-target.patch b/SOURCES/kvm-mirror-Fail-gracefully-for-source-target.patch new file mode 100644 index 0000000..69b7a1c --- /dev/null +++ b/SOURCES/kvm-mirror-Fail-gracefully-for-source-target.patch @@ -0,0 +1,87 @@ +From 7ec94ee16262eb0da7fb7788347a65162845b1c2 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 13:19:57 +0100 +Subject: [PATCH 3/5] mirror: Fail gracefully for source == target + +RH-Author: Kevin Wolf +Message-id: <20181010131957.23198-2-kwolf@redhat.com> +Patchwork-id: 82564 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/1] mirror: Fail gracefully for source == target +Bugzilla: 1637963 +RH-Acked-by: John Snow +RH-Acked-by: Fam Zheng +RH-Acked-by: Stefan Hajnoczi + +blockdev-mirror with the same node for source and target segfaults +today: A node is in its own backing chain, so mirror_start_job() decides +that this is an active commit. When adding the intermediate nodes with +block_job_add_bdrv(), it starts the iteration through the subchain with +the backing file of source, though, so it never reaches target and +instead runs into NULL at the base. + +While we could fix that by starting with source itself, there is no +point in allowing mirroring a node into itself and I wouldn't be +surprised if this caused more problems later. + +So just check for this scenario and error out. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 86fae10c64d642256cf019e6829929fa0d259c7a) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/mirror.c | 5 +++++ + tests/qemu-iotests/041 | 6 ++++++ + tests/qemu-iotests/041.out | 4 ++-- + 3 files changed, 13 insertions(+), 2 deletions(-) + +diff --git a/block/mirror.c b/block/mirror.c +index 163b1d4..313e6e9 100644 +--- a/block/mirror.c ++++ b/block/mirror.c +@@ -1149,6 +1149,11 @@ static void mirror_start_job(const char *job_id, BlockDriverState *bs, + buf_size = DEFAULT_MIRROR_BUF_SIZE; + } + ++ if (bs == target) { ++ error_setg(errp, "Can't mirror node into itself"); ++ return; ++ } ++ + /* In the case of active commit, add dummy driver to provide consistent + * reads on the top, while disabling it in the intermediate nodes, and make + * the backing chain writable. */ +diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 +index c20ac7d..9336ab6 100755 +--- a/tests/qemu-iotests/041 ++++ b/tests/qemu-iotests/041 +@@ -234,6 +234,12 @@ class TestSingleBlockdev(TestSingleDrive): + result = self.vm.qmp("blockdev-add", **args) + self.assert_qmp(result, 'return', {}) + ++ def test_mirror_to_self(self): ++ result = self.vm.qmp(self.qmp_cmd, job_id='job0', ++ device=self.qmp_target, sync='full', ++ target=self.qmp_target) ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ + test_large_cluster = None + test_image_not_found = None + test_small_buffer2 = None +diff --git a/tests/qemu-iotests/041.out b/tests/qemu-iotests/041.out +index c28b392..e071d0b 100644 +--- a/tests/qemu-iotests/041.out ++++ b/tests/qemu-iotests/041.out +@@ -1,5 +1,5 @@ +-..................................................................................... ++........................................................................................ + ---------------------------------------------------------------------- +-Ran 85 tests ++Ran 88 tests + + OK +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-Support-auto-read-only-option.patch b/SOURCES/kvm-nbd-Support-auto-read-only-option.patch new file mode 100644 index 0000000..b72ffc6 --- /dev/null +++ b/SOURCES/kvm-nbd-Support-auto-read-only-option.patch @@ -0,0 +1,51 @@ +From 83f682c20418e556f2ef280b0cddfd2df41f8d9f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:35 +0000 +Subject: [PATCH 05/14] nbd: Support auto-read-only option + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-6-kwolf@redhat.com> +Patchwork-id: 83953 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 05/12] nbd: Support auto-read-only option +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +If read-only=off, but auto-read-only=on is given, open a read-write NBD +connection if the server provides a read-write export, but instead of +erroring out for read-only exports, just degrade to read-only. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 6c2e581d4d7751f035e9bac0384703879c8a1538) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/nbd-client.c | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +diff --git a/block/nbd-client.c b/block/nbd-client.c +index 98637c0..80d3625 100644 +--- a/block/nbd-client.c ++++ b/block/nbd-client.c +@@ -988,11 +988,11 @@ int nbd_client_init(BlockDriverState *bs, + logout("Failed to negotiate with the NBD server\n"); + return ret; + } +- if (client->info.flags & NBD_FLAG_READ_ONLY && +- !bdrv_is_read_only(bs)) { +- error_setg(errp, +- "request for write access conflicts with read-only export"); +- return -EACCES; ++ if (client->info.flags & NBD_FLAG_READ_ONLY) { ++ ret = bdrv_apply_auto_read_only(bs, "NBD export is read-only", errp); ++ if (ret < 0) { ++ return ret; ++ } + } + if (client->info.flags & NBD_FLAG_SEND_FUA) { + bs->supported_write_flags = BDRV_REQ_FUA; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-client-Add-x-dirty-bitmap-to-query-bitmap-from-s.patch b/SOURCES/kvm-nbd-client-Add-x-dirty-bitmap-to-query-bitmap-from-s.patch new file mode 100644 index 0000000..16c4a3f --- /dev/null +++ b/SOURCES/kvm-nbd-client-Add-x-dirty-bitmap-to-query-bitmap-from-s.patch @@ -0,0 +1,174 @@ +From 09e6d08bc3cb4c80b508d2e72f861fdb1b5612ea Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:10 +0200 +Subject: [PATCH 252/268] nbd/client: Add x-dirty-bitmap to query bitmap from + server + +RH-Author: John Snow +Message-id: <20180718225511.14878-35-jsnow@redhat.com> +Patchwork-id: 81414 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 34/35] nbd/client: Add x-dirty-bitmap to query bitmap from server +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Eric Blake + +In order to test that the NBD server is properly advertising +dirty bitmaps, we need a bare minimum client that can request +and read the context. Since feature freeze for 3.0 is imminent, +this is the smallest workable patch, which replaces the qemu +block status report with the results of the NBD server's dirty +bitmap (making it very easy to use 'qemu-img map --output=json' +to learn where the dirty portions are). Note that the NBD +protocol defines a dirty section with the same bit but opposite +sense that normal "base:allocation" uses to report an allocated +section; so in qemu-img map output, "data":true corresponds to +clean, "data":false corresponds to dirty. + +A more complete solution that allows dirty bitmaps to be queried +at the same time as normal block status will be required before +this addition can lose the x- prefix. Until then, the fact that +this replaces normal status with dirty status means actions +like 'qemu-img convert' will likely misbehave due to treating +dirty regions of the file as if they are unallocated. + +The next patch adds an iotest to exercise this new code. + +Signed-off-by: Eric Blake +Message-Id: <20180702191458.28741-2-eblake@redhat.com> +(cherry picked from commit 216ee3657e14013505abe7853cecb632199fb13e) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/nbd-client.c | 3 +++ + block/nbd-client.h | 1 + + block/nbd.c | 10 ++++++++-- + include/block/nbd.h | 1 + + nbd/client.c | 4 ++-- + qapi/block-core.json | 7 ++++++- + 6 files changed, 21 insertions(+), 5 deletions(-) + +diff --git a/block/nbd-client.c b/block/nbd-client.c +index e7caf49..98637c0 100644 +--- a/block/nbd-client.c ++++ b/block/nbd-client.c +@@ -966,6 +966,7 @@ int nbd_client_init(BlockDriverState *bs, + const char *export, + QCryptoTLSCreds *tlscreds, + const char *hostname, ++ const char *x_dirty_bitmap, + Error **errp) + { + NBDClientSession *client = nbd_get_client_session(bs); +@@ -978,9 +979,11 @@ int nbd_client_init(BlockDriverState *bs, + client->info.request_sizes = true; + client->info.structured_reply = true; + client->info.base_allocation = true; ++ client->info.x_dirty_bitmap = g_strdup(x_dirty_bitmap); + ret = nbd_receive_negotiate(QIO_CHANNEL(sioc), export, + tlscreds, hostname, + &client->ioc, &client->info, errp); ++ g_free(client->info.x_dirty_bitmap); + if (ret < 0) { + logout("Failed to negotiate with the NBD server\n"); + return ret; +diff --git a/block/nbd-client.h b/block/nbd-client.h +index 0ece76e..cfc9055 100644 +--- a/block/nbd-client.h ++++ b/block/nbd-client.h +@@ -45,6 +45,7 @@ int nbd_client_init(BlockDriverState *bs, + const char *export_name, + QCryptoTLSCreds *tlscreds, + const char *hostname, ++ const char *x_dirty_bitmap, + Error **errp); + void nbd_client_close(BlockDriverState *bs); + +diff --git a/block/nbd.c b/block/nbd.c +index 10912c3..f29c10f 100644 +--- a/block/nbd.c ++++ b/block/nbd.c +@@ -378,6 +378,12 @@ static QemuOptsList nbd_runtime_opts = { + .type = QEMU_OPT_STRING, + .help = "ID of the TLS credentials to use", + }, ++ { ++ .name = "x-dirty-bitmap", ++ .type = QEMU_OPT_STRING, ++ .help = "experimental: expose named dirty bitmap in place of " ++ "block status", ++ }, + { /* end of list */ } + }, + }; +@@ -438,8 +444,8 @@ static int nbd_open(BlockDriverState *bs, QDict *options, int flags, + } + + /* NBD handshake */ +- ret = nbd_client_init(bs, sioc, s->export, +- tlscreds, hostname, errp); ++ ret = nbd_client_init(bs, sioc, s->export, tlscreds, hostname, ++ qemu_opt_get(opts, "x-dirty-bitmap"), errp); + error: + if (sioc) { + object_unref(OBJECT(sioc)); +diff --git a/include/block/nbd.h b/include/block/nbd.h +index daaeae6..4638c83 100644 +--- a/include/block/nbd.h ++++ b/include/block/nbd.h +@@ -259,6 +259,7 @@ static inline bool nbd_reply_type_is_error(int type) + struct NBDExportInfo { + /* Set by client before nbd_receive_negotiate() */ + bool request_sizes; ++ char *x_dirty_bitmap; + + /* In-out fields, set by client before nbd_receive_negotiate() and + * updated by server results during nbd_receive_negotiate() */ +diff --git a/nbd/client.c b/nbd/client.c +index b9e175d..25603f2 100644 +--- a/nbd/client.c ++++ b/nbd/client.c +@@ -1,5 +1,5 @@ + /* +- * Copyright (C) 2016-2017 Red Hat, Inc. ++ * Copyright (C) 2016-2018 Red Hat, Inc. + * Copyright (C) 2005 Anthony Liguori + * + * Network Block Device Client Side +@@ -825,7 +825,7 @@ int nbd_receive_negotiate(QIOChannel *ioc, const char *name, + + if (info->structured_reply && base_allocation) { + result = nbd_negotiate_simple_meta_context( +- ioc, name, "base:allocation", ++ ioc, name, info->x_dirty_bitmap ?: "base:allocation", + &info->meta_base_allocation_id, errp); + if (result < 0) { + goto fail; +diff --git a/qapi/block-core.json b/qapi/block-core.json +index b2de7af..a6399af 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -3457,12 +3457,17 @@ + # + # @tls-creds: TLS credentials ID + # ++# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of ++# traditional "base:allocation" block status (see ++# NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0) ++# + # Since: 2.9 + ## + { 'struct': 'BlockdevOptionsNbd', + 'data': { 'server': 'SocketAddress', + '*export': 'str', +- '*tls-creds': 'str' } } ++ '*tls-creds': 'str', ++ '*x-dirty-bitmap': 'str' } } + + ## + # @BlockdevOptionsRaw: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-fix-NBD_FLAG_SEND_CACHE-value.patch b/SOURCES/kvm-nbd-fix-NBD_FLAG_SEND_CACHE-value.patch new file mode 100644 index 0000000..5a7166e --- /dev/null +++ b/SOURCES/kvm-nbd-fix-NBD_FLAG_SEND_CACHE-value.patch @@ -0,0 +1,96 @@ +From bbcd98d89650ff2394cef9d446ed8b595b4703c7 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 18:19:24 +0100 +Subject: [PATCH 03/49] nbd: fix NBD_FLAG_SEND_CACHE value + +RH-Author: John Snow +Message-id: <20181010181924.30470-3-jsnow@redhat.com> +Patchwork-id: 82578 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/2] nbd: fix NBD_FLAG_SEND_CACHE value +Bugzilla: 1636142 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula +RH-Acked-by: Thomas Huth + +From: "Denis V. Lunev" + +Commit bc37b06a5 added NBD_CMD_CACHE support, but used the wrong value +for NBD_FLAG_SEND_CACHE flag for negotiation. That commit picked bit 8, +which had already been assigned by the NBD specification to mean +NBD_FLAG_CAN_MULTI_CONN, and which was already implemented in the +Linux kernel as a part of stable userspace-kernel API since 4.10: + +"bit 8, NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates +entirely without cache, or that the cache it uses is shared among all +connections to the given device. In particular, if this flag is +present, then the effects of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA +MUST be visible across all connections when the server sends its reply +to that command to the client. In the absense of this flag, clients +SHOULD NOT multiplex their commands over more than one connection to +the export. +... +bit 10, NBD_FLAG_SEND_CACHE: documents that the server understands +NBD_CMD_CACHE; however, note that server implementations exist +which support the command without advertising this bit, and +conversely that this bit does not guarantee that the command will +succeed or have an impact." + +Consequences: +- a client trying to use NBD_CMD_CACHE per the NBD spec will not +see the feature as available from a qemu 3.0 server (not fatal, +clients already have to be prepared for caching to not exist) +- a client accidentally coded to the qemu 3.0 bit value instead +of following the spec may interpret NBD_CMD_CACHE as being available +when it is not (probably not fatal, the spec says the server should +gracefully fail unknown commands, and that clients of NBD_CMD_CACHE +should be prepared for failure even when the feature is advertised); +such clients are unlikely (perhaps only in unreleased Virtuozzo code), +and will disappear over time +- a client prepared to use multiple connections based on +NBD_FLAG_CAN_MULTI_CONN may cause data corruption when it assumes +that caching is consistent when in reality qemu 3.0 did not have +a consistent cache. Partially mitigated by using read-only +connections (where nothing needs to be flushed, so caching is +indeed consistent) or when using qemu-nbd with the default -e 1 +(at most one client at a time); visible only when using -e 2 or +more for a writable export. + +Thus the commit fixes negotiation flag in QEMU according to the +specification. + +Signed-off-by: Denis V. Lunev +CC: Vladimir Sementsov-Ogievskiy +CC: Valery Vdovin +CC: Eric Blake +CC: Paolo Bonzini +CC: qemu-stable@nongnu.org +Message-Id: <20181004100313.4253-1-den@openvz.org> +Reviewed-by: Eric Blake +[eblake: enhance commit message, add defines for unimplemented flags] +Signed-off-by: Eric Blake +(cherry picked from commit df91328adab8490367776d2b21b35d790a606120) +Signed-off-by: John Snow + +Signed-off-by: Danilo C. L. de Paula +--- + include/block/nbd.h | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/include/block/nbd.h b/include/block/nbd.h +index 4638c83..a53b0cf 100644 +--- a/include/block/nbd.h ++++ b/include/block/nbd.h +@@ -135,7 +135,9 @@ typedef struct NBDExtent { + #define NBD_FLAG_SEND_TRIM (1 << 5) /* Send TRIM (discard) */ + #define NBD_FLAG_SEND_WRITE_ZEROES (1 << 6) /* Send WRITE_ZEROES */ + #define NBD_FLAG_SEND_DF (1 << 7) /* Send DF (Do not Fragment) */ +-#define NBD_FLAG_SEND_CACHE (1 << 8) /* Send CACHE (prefetch) */ ++#define NBD_FLAG_CAN_MULTI_CONN (1 << 8) /* Multi-client cache consistent */ ++#define NBD_FLAG_SEND_RESIZE (1 << 9) /* Send resize */ ++#define NBD_FLAG_SEND_CACHE (1 << 10) /* Send CACHE (prefetch) */ + + /* New-style handshake (global) flags, sent from server to client, and + control what will happen during handshake phase. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-forbid-use-of-frozen-bitmaps.patch b/SOURCES/kvm-nbd-forbid-use-of-frozen-bitmaps.patch new file mode 100644 index 0000000..9846c34 --- /dev/null +++ b/SOURCES/kvm-nbd-forbid-use-of-frozen-bitmaps.patch @@ -0,0 +1,46 @@ +From 61a642fa86014af236c1ea5ae84a9bbffcf67e6f Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:21 +0000 +Subject: [PATCH 27/35] nbd: forbid use of frozen bitmaps + +RH-Author: John Snow +Message-id: <20181120181828.15132-18-jsnow@redhat.com> +Patchwork-id: 83071 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 17/24] nbd: forbid use of frozen bitmaps +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +Whether it's "locked" or "frozen", it's in use and should +not be allowed for the purposes of this operation. + +Signed-off-by: John Snow +Reviewed-by: Vladimir Sementsov-Ogievskiy +Message-id: 20181002230218.13949-7-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit d9782022bda7f8eccaf961044e9efe980dc90c04) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + nbd/server.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/nbd/server.c b/nbd/server.c +index 1ce3f44..e094300 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -2435,8 +2435,8 @@ void nbd_export_bitmap(NBDExport *exp, const char *bitmap, + return; + } + +- if (bdrv_dirty_bitmap_qmp_locked(bm)) { +- error_setg(errp, "Bitmap '%s' is locked", bitmap); ++ if (bdrv_dirty_bitmap_user_locked(bm)) { ++ error_setg(errp, "Bitmap '%s' is in use", bitmap); + return; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-Fix-dirty-bitmap-logic-regression.patch b/SOURCES/kvm-nbd-server-Fix-dirty-bitmap-logic-regression.patch new file mode 100644 index 0000000..31732e5 --- /dev/null +++ b/SOURCES/kvm-nbd-server-Fix-dirty-bitmap-logic-regression.patch @@ -0,0 +1,54 @@ +From 62c3dde0dcd04ce52e761beb66a3ec4aacd007e6 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:08 +0200 +Subject: [PATCH 250/268] nbd/server: Fix dirty bitmap logic regression + +RH-Author: John Snow +Message-id: <20180718225511.14878-33-jsnow@redhat.com> +Patchwork-id: 81427 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 32/35] nbd/server: Fix dirty bitmap logic regression +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Eric Blake + +In my hurry to fix a build failure, I introduced a logic bug. +The assertion conditional is backwards, meaning that qemu will +now abort instead of reporting dirty bitmap status. + +The bug can only be tickled by an NBD client using an exported +dirty bitmap (which is still an experimental QMP command), so +it's not the end of the world for supported usage (and neither +'make check' nor qemu-iotests fails); but it also shows that we +really want qemu-io support for reading dirty bitmaps if only +so that I can add iotests coverage to prevent future +brown-bag-of-shame events like this one. + +Fixes: 45eb6fb6 +Signed-off-by: Eric Blake +Message-Id: <20180622153509.375130-1-eblake@redhat.com> +(cherry picked from commit 7606c99a0421be7e9d984766fe239f7791a2fd9c) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/nbd/server.c b/nbd/server.c +index 50ac8bf..e52b76b 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -1977,7 +1977,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap *bitmap, uint64_t offset, + + bdrv_dirty_bitmap_unlock(bitmap); + +- assert(offset > end); ++ assert(offset < end); + *length = end - offset; + return i; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-Reject-0-length-block-status-request.patch b/SOURCES/kvm-nbd-server-Reject-0-length-block-status-request.patch new file mode 100644 index 0000000..e22d050 --- /dev/null +++ b/SOURCES/kvm-nbd-server-Reject-0-length-block-status-request.patch @@ -0,0 +1,51 @@ +From 7f8edd3a33860bb52428edb0f8f84c87ec479543 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:59 +0200 +Subject: [PATCH 241/268] nbd/server: Reject 0-length block status request + +RH-Author: John Snow +Message-id: <20180718225511.14878-24-jsnow@redhat.com> +Patchwork-id: 81423 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 23/35] nbd/server: Reject 0-length block status request +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Eric Blake + +The NBD spec says that behavior is unspecified if the client +requests 0 length for block status; but since the structured +reply is documenting as returning a non-zero length, it's +easier to just diagnose this with an EINVAL error than to +figure out what to return. + +CC: qemu-stable@nongnu.org +Signed-off-by: Eric Blake +Message-Id: <20180621124937.166549-1-eblake@redhat.com> +Reviewed-by: Vladimir Sementsov-Ogievskiy +(cherry picked from commit d8b20291cba6aa9bb295885a34f2b5f05d59d1b2) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/nbd/server.c b/nbd/server.c +index 9e1f227..493a926 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -2007,6 +2007,10 @@ static coroutine_fn int nbd_handle_request(NBDClient *client, + "discard failed", errp); + + case NBD_CMD_BLOCK_STATUS: ++ if (!request->len) { ++ return nbd_send_generic_reply(client, request->handle, -EINVAL, ++ "need non-zero length", errp); ++ } + if (client->export_meta.valid && client->export_meta.base_allocation) { + return nbd_co_send_block_status(client, request->handle, + blk_bs(exp->blk), request->from, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-Silence-gcc-false-positive.patch b/SOURCES/kvm-nbd-server-Silence-gcc-false-positive.patch new file mode 100644 index 0000000..30a858b --- /dev/null +++ b/SOURCES/kvm-nbd-server-Silence-gcc-false-positive.patch @@ -0,0 +1,56 @@ +From c24b1c565318bada6133e9489d9fcf8f589da56f Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:07 +0200 +Subject: [PATCH 249/268] nbd/server: Silence gcc false positive + +RH-Author: John Snow +Message-id: <20180718225511.14878-32-jsnow@redhat.com> +Patchwork-id: 81425 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 31/35] nbd/server: Silence gcc false positive +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Eric Blake + +The code has a while() loop that always initialized 'end', and +the loop always executes at least once (as evidenced by the assert() +just prior to the loop). But some versions of gcc still complain +that 'end' is used uninitialized, so silence them. + +Signed-off-by: Eric Blake +Reviewed-by: Peter Maydell +Message-id: 20180622125814.345274-1-eblake@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit 45eb6fb6cea28cdc937764aac6585751047bb294) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/nbd/server.c b/nbd/server.c +index 2746046..50ac8bf 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -1937,7 +1937,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap *bitmap, uint64_t offset, + unsigned int nb_extents, + bool dont_fragment) + { +- uint64_t begin = offset, end; ++ uint64_t begin = offset, end = offset; + uint64_t overall_end = offset + *length; + unsigned int i = 0; + BdrvDirtyBitmapIter *it; +@@ -1977,6 +1977,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap *bitmap, uint64_t offset, + + bdrv_dirty_bitmap_unlock(bitmap); + ++ assert(offset > end); + *length = end - offset; + return i; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-add-nbd_meta_empty_or_pattern-helper.patch b/SOURCES/kvm-nbd-server-add-nbd_meta_empty_or_pattern-helper.patch new file mode 100644 index 0000000..55c28c4 --- /dev/null +++ b/SOURCES/kvm-nbd-server-add-nbd_meta_empty_or_pattern-helper.patch @@ -0,0 +1,163 @@ +From de65ad93b1b0a38c8a3f26eb1bc4090eb19db00f Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:02 +0200 +Subject: [PATCH 244/268] nbd/server: add nbd_meta_empty_or_pattern helper + +RH-Author: John Snow +Message-id: <20180718225511.14878-27-jsnow@redhat.com> +Patchwork-id: 81398 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 26/35] nbd/server: add nbd_meta_empty_or_pattern helper +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Add nbd_meta_pattern() and nbd_meta_empty_or_pattern() helpers for +metadata query parsing. nbd_meta_pattern() will be reused for the +"qemu" namespace in following patches. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180609151758.17343-4-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: comment tweaks] +Signed-off-by: Eric Blake +(cherry picked from commit b0769d8f8df0b51881f1f15c9e29722cf6191a43) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 89 ++++++++++++++++++++++++++++++++++++++++++------------------ + 1 file changed, 62 insertions(+), 27 deletions(-) + +diff --git a/nbd/server.c b/nbd/server.c +index 26cc41a..9171cd4 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -733,53 +733,87 @@ static int nbd_negotiate_send_meta_context(NBDClient *client, + return qio_channel_writev_all(client->ioc, iov, 2, errp) < 0 ? -EIO : 0; + } + +-/* nbd_meta_base_query +- * +- * Handle queries to 'base' namespace. For now, only the base:allocation +- * context is available. 'len' is the amount of text remaining to be read from +- * the current name, after the 'base:' portion has been stripped. ++/* Read strlen(@pattern) bytes, and set @match to true if they match @pattern. ++ * @match is never set to false. + * + * Return -errno on I/O error, 0 if option was completely handled by + * sending a reply about inconsistent lengths, or 1 on success. + * +- * Note: return code = 1 doesn't mean that we've parsed the "base:allocation" +- * namespace. It only means that there are no errors. ++ * Note: return code = 1 doesn't mean that we've read exactly @pattern. ++ * It only means that there are no errors. + */ +-static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, +- uint32_t len, Error **errp) ++static int nbd_meta_pattern(NBDClient *client, const char *pattern, bool *match, ++ Error **errp) + { + int ret; +- char query[sizeof("allocation") - 1]; +- size_t alen = strlen("allocation"); +- +- if (len == 0) { +- if (client->opt == NBD_OPT_LIST_META_CONTEXT) { +- meta->base_allocation = true; +- } +- trace_nbd_negotiate_meta_query_parse("base:"); +- return 1; +- } ++ char *query; ++ size_t len = strlen(pattern); + +- if (len != alen) { +- trace_nbd_negotiate_meta_query_skip("not base:allocation"); +- return nbd_opt_skip(client, len, errp); +- } ++ assert(len); + ++ query = g_malloc(len); + ret = nbd_opt_read(client, query, len, errp); + if (ret <= 0) { ++ g_free(query); + return ret; + } + +- if (strncmp(query, "allocation", alen) == 0) { +- trace_nbd_negotiate_meta_query_parse("base:allocation"); +- meta->base_allocation = true; ++ if (strncmp(query, pattern, len) == 0) { ++ trace_nbd_negotiate_meta_query_parse(pattern); ++ *match = true; + } else { +- trace_nbd_negotiate_meta_query_skip("not base:allocation"); ++ trace_nbd_negotiate_meta_query_skip("pattern not matched"); + } ++ g_free(query); + + return 1; + } + ++/* ++ * Read @len bytes, and set @match to true if they match @pattern, or if @len ++ * is 0 and the client is performing _LIST_. @match is never set to false. ++ * ++ * Return -errno on I/O error, 0 if option was completely handled by ++ * sending a reply about inconsistent lengths, or 1 on success. ++ * ++ * Note: return code = 1 doesn't mean that we've read exactly @pattern. ++ * It only means that there are no errors. ++ */ ++static int nbd_meta_empty_or_pattern(NBDClient *client, const char *pattern, ++ uint32_t len, bool *match, Error **errp) ++{ ++ if (len == 0) { ++ if (client->opt == NBD_OPT_LIST_META_CONTEXT) { ++ *match = true; ++ } ++ trace_nbd_negotiate_meta_query_parse("empty"); ++ return 1; ++ } ++ ++ if (len != strlen(pattern)) { ++ trace_nbd_negotiate_meta_query_skip("different lengths"); ++ return nbd_opt_skip(client, len, errp); ++ } ++ ++ return nbd_meta_pattern(client, pattern, match, errp); ++} ++ ++/* nbd_meta_base_query ++ * ++ * Handle queries to 'base' namespace. For now, only the base:allocation ++ * context is available. 'len' is the amount of text remaining to be read from ++ * the current name, after the 'base:' portion has been stripped. ++ * ++ * Return -errno on I/O error, 0 if option was completely handled by ++ * sending a reply about inconsistent lengths, or 1 on success. ++ */ ++static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, ++ uint32_t len, Error **errp) ++{ ++ return nbd_meta_empty_or_pattern(client, "allocation", len, ++ &meta->base_allocation, errp); ++} ++ + /* nbd_negotiate_meta_query + * + * Parse namespace name and call corresponding function to parse body of the +@@ -823,6 +857,7 @@ static int nbd_negotiate_meta_query(NBDClient *client, + return nbd_opt_skip(client, len, errp); + } + ++ trace_nbd_negotiate_meta_query_parse("base:"); + return nbd_meta_base_query(client, meta, len, errp); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-fix-NBD_CMD_CACHE.patch b/SOURCES/kvm-nbd-server-fix-NBD_CMD_CACHE.patch new file mode 100644 index 0000000..f80e870 --- /dev/null +++ b/SOURCES/kvm-nbd-server-fix-NBD_CMD_CACHE.patch @@ -0,0 +1,52 @@ +From 290b8b6ab5f9a1f747be698e628c646c7c76b972 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 10 Oct 2018 18:19:23 +0100 +Subject: [PATCH 02/49] nbd/server: fix NBD_CMD_CACHE + +RH-Author: John Snow +Message-id: <20181010181924.30470-2-jsnow@redhat.com> +Patchwork-id: 82576 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/2] nbd/server: fix NBD_CMD_CACHE +Bugzilla: 1636142 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Danilo de Paula +RH-Acked-by: Thomas Huth + +From: Vladimir Sementsov-Ogievskiy + +We should not go to structured-read branch on CACHE command, fix that. + +Bug introduced in bc37b06a5cde24 "nbd/server: introduce NBD_CMD_CACHE" +with the whole feature and affects 3.0.0 release. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +CC: qemu-stable@nongnu.org +Message-Id: <20181003144738.70670-1-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: commit message typo fix] +Signed-off-by: Eric Blake +(cherry picked from commit 2f454defc23e1be78f2a96bad2877ce7829f61b4) +Signed-off-by: John Snow + +Signed-off-by: Danilo C. L. de Paula +--- + nbd/server.c | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/nbd/server.c b/nbd/server.c +index ea5fe0e..1ce3f44 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -2135,7 +2135,8 @@ static coroutine_fn int nbd_do_cmd_read(NBDClient *client, NBDRequest *request, + } + + if (client->structured_reply && !(request->flags & NBD_CMD_FLAG_DF) && +- request->len) { ++ request->len && request->type != NBD_CMD_CACHE) ++ { + return nbd_co_send_sparse_read(client, request->handle, request->from, + data, request->len, errp); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-fix-nbd_co_send_block_status.patch b/SOURCES/kvm-nbd-server-fix-nbd_co_send_block_status.patch new file mode 100644 index 0000000..2e86125 --- /dev/null +++ b/SOURCES/kvm-nbd-server-fix-nbd_co_send_block_status.patch @@ -0,0 +1,58 @@ +From e52756f20cebe52713bd13672f1217d637e039f7 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:09 +0200 +Subject: [PATCH 251/268] nbd/server: fix nbd_co_send_block_status + +RH-Author: John Snow +Message-id: <20180718225511.14878-34-jsnow@redhat.com> +Patchwork-id: 81418 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 33/35] nbd/server: fix nbd_co_send_block_status +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Call nbd_co_send_extents() with correct length parameter +(extent.length may be smaller than original length). + +Also, switch length parameter type to uint32_t, to correspond with +request->len and similar nbd_co_send_bitmap(). + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180704112302.471456-2-vsementsov@virtuozzo.com> +Signed-off-by: Eric Blake +(cherry picked from commit 0c0eaed14721f8a9db334deb35316411c512059a) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 5 +++-- + 1 file changed, 3 insertions(+), 2 deletions(-) + +diff --git a/nbd/server.c b/nbd/server.c +index e52b76b..ea5fe0e 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -1910,7 +1910,7 @@ static int nbd_co_send_extents(NBDClient *client, uint64_t handle, + /* Get block status from the exported device and send it to the client */ + static int nbd_co_send_block_status(NBDClient *client, uint64_t handle, + BlockDriverState *bs, uint64_t offset, +- uint64_t length, bool last, ++ uint32_t length, bool last, + uint32_t context_id, Error **errp) + { + int ret; +@@ -1922,7 +1922,8 @@ static int nbd_co_send_block_status(NBDClient *client, uint64_t handle, + client, handle, -ret, "can't get block status", errp); + } + +- return nbd_co_send_extents(client, handle, &extent, 1, length, last, ++ return nbd_co_send_extents(client, handle, &extent, 1, ++ be32_to_cpu(extent.length), last, + context_id, errp); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-fix-trace.patch b/SOURCES/kvm-nbd-server-fix-trace.patch new file mode 100644 index 0000000..ff3ebd5 --- /dev/null +++ b/SOURCES/kvm-nbd-server-fix-trace.patch @@ -0,0 +1,73 @@ +From b84814bde02585f990a32f2ebfbdda1de1fce5a8 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:00 +0200 +Subject: [PATCH 242/268] nbd/server: fix trace + +RH-Author: John Snow +Message-id: <20180718225511.14878-25-jsnow@redhat.com> +Patchwork-id: 81401 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 24/35] nbd/server: fix trace +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Return code = 1 doesn't mean that we parsed base:allocation. Use +correct traces in both -parsed and -skipped cases. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180609151758.17343-2-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: comment tweaks] +Signed-off-by: Eric Blake +(cherry picked from commit dbb8b396bb46388cee92e9094c563297d04c43ed) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 14 ++++++++++---- + 1 file changed, 10 insertions(+), 4 deletions(-) + +diff --git a/nbd/server.c b/nbd/server.c +index 493a926..942c016 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -736,12 +736,16 @@ static int nbd_negotiate_send_meta_context(NBDClient *client, + + /* nbd_meta_base_query + * +- * Handle query to 'base' namespace. For now, only base:allocation context is +- * available in it. 'len' is the amount of text remaining to be read from ++ * Handle queries to 'base' namespace. For now, only the base:allocation ++ * context is available. 'len' is the amount of text remaining to be read from + * the current name, after the 'base:' portion has been stripped. + * + * Return -errno on I/O error, 0 if option was completely handled by +- * sending a reply about inconsistent lengths, or 1 on success. */ ++ * sending a reply about inconsistent lengths, or 1 on success. ++ * ++ * Note: return code = 1 doesn't mean that we've parsed the "base:allocation" ++ * namespace. It only means that there are no errors. ++ */ + static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, + uint32_t len, Error **errp) + { +@@ -768,10 +772,12 @@ static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, + } + + if (strncmp(query, "allocation", alen) == 0) { ++ trace_nbd_negotiate_meta_query_parse("base:allocation"); + meta->base_allocation = true; ++ } else { ++ trace_nbd_negotiate_meta_query_skip("not base:allocation"); + } + +- trace_nbd_negotiate_meta_query_parse("base:allocation"); + return 1; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-implement-dirty-bitmap-export.patch b/SOURCES/kvm-nbd-server-implement-dirty-bitmap-export.patch new file mode 100644 index 0000000..3ce8ed3 --- /dev/null +++ b/SOURCES/kvm-nbd-server-implement-dirty-bitmap-export.patch @@ -0,0 +1,471 @@ +From e058bf5e47af745997f99aa21c003e40f50bddcc Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:03 +0200 +Subject: [PATCH 245/268] nbd/server: implement dirty bitmap export + +RH-Author: John Snow +Message-id: <20180718225511.14878-28-jsnow@redhat.com> +Patchwork-id: 81412 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 27/35] nbd/server: implement dirty bitmap export +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Handle a new NBD meta namespace: "qemu", and corresponding queries: +"qemu:dirty-bitmap:". + +With the new metadata context negotiated, BLOCK_STATUS query will reply +with dirty-bitmap data, converted to extents. The new public function +nbd_export_bitmap selects which bitmap to export. For now, only one bitmap +may be exported. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180609151758.17343-5-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: wording tweaks, minor cleanups, additional tracing] +Signed-off-by: Eric Blake +(cherry picked from commit 3d068aff16d6dbf066328977c5152847a62f2a0a) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + include/block/nbd.h | 8 +- + nbd/server.c | 278 +++++++++++++++++++++++++++++++++++++++++++++++----- + nbd/trace-events | 1 + + 3 files changed, 262 insertions(+), 25 deletions(-) + +diff --git a/include/block/nbd.h b/include/block/nbd.h +index fcdcd54..8bb9606 100644 +--- a/include/block/nbd.h ++++ b/include/block/nbd.h +@@ -229,11 +229,13 @@ enum { + #define NBD_REPLY_TYPE_ERROR NBD_REPLY_ERR(1) + #define NBD_REPLY_TYPE_ERROR_OFFSET NBD_REPLY_ERR(2) + +-/* Flags for extents (NBDExtent.flags) of NBD_REPLY_TYPE_BLOCK_STATUS, +- * for base:allocation meta context */ ++/* Extent flags for base:allocation in NBD_REPLY_TYPE_BLOCK_STATUS */ + #define NBD_STATE_HOLE (1 << 0) + #define NBD_STATE_ZERO (1 << 1) + ++/* Extent flags for qemu:dirty-bitmap in NBD_REPLY_TYPE_BLOCK_STATUS */ ++#define NBD_STATE_DIRTY (1 << 0) ++ + static inline bool nbd_reply_type_is_error(int type) + { + return type & (1 << 15); +@@ -315,6 +317,8 @@ void nbd_client_put(NBDClient *client); + void nbd_server_start(SocketAddress *addr, const char *tls_creds, + Error **errp); + ++void nbd_export_bitmap(NBDExport *exp, const char *bitmap, ++ const char *bitmap_export_name, Error **errp); + + /* nbd_read + * Reads @size bytes from @ioc. Returns 0 on success. +diff --git a/nbd/server.c b/nbd/server.c +index 9171cd4..2c2d62c 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -23,6 +23,13 @@ + #include "nbd-internal.h" + + #define NBD_META_ID_BASE_ALLOCATION 0 ++#define NBD_META_ID_DIRTY_BITMAP 1 ++ ++/* NBD_MAX_BITMAP_EXTENTS: 1 mb of extents data. An empirical ++ * constant. If an increase is needed, note that the NBD protocol ++ * recommends no larger than 32 mb, so that the client won't consider ++ * the reply as a denial of service attack. */ ++#define NBD_MAX_BITMAP_EXTENTS (0x100000 / 8) + + static int system_errno_to_nbd_errno(int err) + { +@@ -80,6 +87,9 @@ struct NBDExport { + + BlockBackend *eject_notifier_blk; + Notifier eject_notifier; ++ ++ BdrvDirtyBitmap *export_bitmap; ++ char *export_bitmap_context; + }; + + static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports); +@@ -92,6 +102,7 @@ typedef struct NBDExportMetaContexts { + bool valid; /* means that negotiation of the option finished without + errors */ + bool base_allocation; /* export base:allocation context (block status) */ ++ bool bitmap; /* export qemu:dirty-bitmap: */ + } NBDExportMetaContexts; + + struct NBDClient { +@@ -814,6 +825,56 @@ static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, + &meta->base_allocation, errp); + } + ++/* nbd_meta_bitmap_query ++ * ++ * Handle query to 'qemu:' namespace. ++ * @len is the amount of text remaining to be read from the current name, after ++ * the 'qemu:' portion has been stripped. ++ * ++ * Return -errno on I/O error, 0 if option was completely handled by ++ * sending a reply about inconsistent lengths, or 1 on success. */ ++static int nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, ++ uint32_t len, Error **errp) ++{ ++ bool dirty_bitmap = false; ++ size_t dirty_bitmap_len = strlen("dirty-bitmap:"); ++ int ret; ++ ++ if (!meta->exp->export_bitmap) { ++ trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported"); ++ return nbd_opt_skip(client, len, errp); ++ } ++ ++ if (len == 0) { ++ if (client->opt == NBD_OPT_LIST_META_CONTEXT) { ++ meta->bitmap = true; ++ } ++ trace_nbd_negotiate_meta_query_parse("empty"); ++ return 1; ++ } ++ ++ if (len < dirty_bitmap_len) { ++ trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:"); ++ return nbd_opt_skip(client, len, errp); ++ } ++ ++ len -= dirty_bitmap_len; ++ ret = nbd_meta_pattern(client, "dirty-bitmap:", &dirty_bitmap, errp); ++ if (ret <= 0) { ++ return ret; ++ } ++ if (!dirty_bitmap) { ++ trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:"); ++ return nbd_opt_skip(client, len, errp); ++ } ++ ++ trace_nbd_negotiate_meta_query_parse("dirty-bitmap:"); ++ ++ return nbd_meta_empty_or_pattern( ++ client, meta->exp->export_bitmap_context + ++ strlen("qemu:dirty_bitmap:"), len, &meta->bitmap, errp); ++} ++ + /* nbd_negotiate_meta_query + * + * Parse namespace name and call corresponding function to parse body of the +@@ -829,9 +890,14 @@ static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, + static int nbd_negotiate_meta_query(NBDClient *client, + NBDExportMetaContexts *meta, Error **errp) + { ++ /* ++ * Both 'qemu' and 'base' namespaces have length = 5 including a ++ * colon. If another length namespace is later introduced, this ++ * should certainly be refactored. ++ */ + int ret; +- char query[sizeof("base:") - 1]; +- size_t baselen = strlen("base:"); ++ size_t ns_len = 5; ++ char ns[5]; + uint32_t len; + + ret = nbd_opt_read(client, &len, sizeof(len), errp); +@@ -840,25 +906,27 @@ static int nbd_negotiate_meta_query(NBDClient *client, + } + cpu_to_be32s(&len); + +- /* The only supported namespace for now is 'base'. So query should start +- * with 'base:'. Otherwise, we can ignore it and skip the remainder. */ +- if (len < baselen) { ++ if (len < ns_len) { + trace_nbd_negotiate_meta_query_skip("length too short"); + return nbd_opt_skip(client, len, errp); + } + +- len -= baselen; +- ret = nbd_opt_read(client, query, baselen, errp); ++ len -= ns_len; ++ ret = nbd_opt_read(client, ns, ns_len, errp); + if (ret <= 0) { + return ret; + } +- if (strncmp(query, "base:", baselen) != 0) { +- trace_nbd_negotiate_meta_query_skip("not for base: namespace"); +- return nbd_opt_skip(client, len, errp); ++ ++ if (!strncmp(ns, "base:", ns_len)) { ++ trace_nbd_negotiate_meta_query_parse("base:"); ++ return nbd_meta_base_query(client, meta, len, errp); ++ } else if (!strncmp(ns, "qemu:", ns_len)) { ++ trace_nbd_negotiate_meta_query_parse("qemu:"); ++ return nbd_meta_qemu_query(client, meta, len, errp); + } + +- trace_nbd_negotiate_meta_query_parse("base:"); +- return nbd_meta_base_query(client, meta, len, errp); ++ trace_nbd_negotiate_meta_query_skip("unknown namespace"); ++ return nbd_opt_skip(client, len, errp); + } + + /* nbd_negotiate_meta_queries +@@ -928,6 +996,16 @@ static int nbd_negotiate_meta_queries(NBDClient *client, + } + } + ++ if (meta->bitmap) { ++ ret = nbd_negotiate_send_meta_context(client, ++ meta->exp->export_bitmap_context, ++ NBD_META_ID_DIRTY_BITMAP, ++ errp); ++ if (ret < 0) { ++ return ret; ++ } ++ } ++ + ret = nbd_negotiate_send_rep(client, NBD_REP_ACK, errp); + if (ret == 0) { + meta->valid = true; +@@ -1556,6 +1634,11 @@ void nbd_export_put(NBDExport *exp) + exp->blk = NULL; + } + ++ if (exp->export_bitmap) { ++ bdrv_dirty_bitmap_set_qmp_locked(exp->export_bitmap, false); ++ g_free(exp->export_bitmap_context); ++ } ++ + g_free(exp); + } + } +@@ -1797,9 +1880,15 @@ static int blockstatus_to_extent_be(BlockDriverState *bs, uint64_t offset, + } + + /* nbd_co_send_extents +- * @extents should be in big-endian */ ++ * ++ * @length is only for tracing purposes (and may be smaller or larger ++ * than the client's original request). @last controls whether ++ * NBD_REPLY_FLAG_DONE is sent. @extents should already be in ++ * big-endian format. ++ */ + static int nbd_co_send_extents(NBDClient *client, uint64_t handle, +- NBDExtent *extents, unsigned nb_extents, ++ NBDExtent *extents, unsigned int nb_extents, ++ uint64_t length, bool last, + uint32_t context_id, Error **errp) + { + NBDStructuredMeta chunk; +@@ -1809,7 +1898,9 @@ static int nbd_co_send_extents(NBDClient *client, uint64_t handle, + {.iov_base = extents, .iov_len = nb_extents * sizeof(extents[0])} + }; + +- set_be_chunk(&chunk.h, NBD_REPLY_FLAG_DONE, NBD_REPLY_TYPE_BLOCK_STATUS, ++ trace_nbd_co_send_extents(handle, nb_extents, context_id, length, last); ++ set_be_chunk(&chunk.h, last ? NBD_REPLY_FLAG_DONE : 0, ++ NBD_REPLY_TYPE_BLOCK_STATUS, + handle, sizeof(chunk) - sizeof(chunk.h) + iov[1].iov_len); + stl_be_p(&chunk.context_id, context_id); + +@@ -1819,8 +1910,8 @@ static int nbd_co_send_extents(NBDClient *client, uint64_t handle, + /* Get block status from the exported device and send it to the client */ + static int nbd_co_send_block_status(NBDClient *client, uint64_t handle, + BlockDriverState *bs, uint64_t offset, +- uint64_t length, uint32_t context_id, +- Error **errp) ++ uint64_t length, bool last, ++ uint32_t context_id, Error **errp) + { + int ret; + NBDExtent extent; +@@ -1831,7 +1922,84 @@ static int nbd_co_send_block_status(NBDClient *client, uint64_t handle, + client, handle, -ret, "can't get block status", errp); + } + +- return nbd_co_send_extents(client, handle, &extent, 1, context_id, errp); ++ return nbd_co_send_extents(client, handle, &extent, 1, length, last, ++ context_id, errp); ++} ++ ++/* ++ * Populate @extents from a dirty bitmap. Unless @dont_fragment, the ++ * final extent may exceed the original @length. Store in @length the ++ * byte length encoded (which may be smaller or larger than the ++ * original), and return the number of extents used. ++ */ ++static unsigned int bitmap_to_extents(BdrvDirtyBitmap *bitmap, uint64_t offset, ++ uint64_t *length, NBDExtent *extents, ++ unsigned int nb_extents, ++ bool dont_fragment) ++{ ++ uint64_t begin = offset, end; ++ uint64_t overall_end = offset + *length; ++ unsigned int i = 0; ++ BdrvDirtyBitmapIter *it; ++ bool dirty; ++ ++ bdrv_dirty_bitmap_lock(bitmap); ++ ++ it = bdrv_dirty_iter_new(bitmap); ++ dirty = bdrv_get_dirty_locked(NULL, bitmap, offset); ++ ++ assert(begin < overall_end && nb_extents); ++ while (begin < overall_end && i < nb_extents) { ++ if (dirty) { ++ end = bdrv_dirty_bitmap_next_zero(bitmap, begin); ++ } else { ++ bdrv_set_dirty_iter(it, begin); ++ end = bdrv_dirty_iter_next(it); ++ } ++ if (end == -1 || end - begin > UINT32_MAX) { ++ /* Cap to an aligned value < 4G beyond begin. */ ++ end = MIN(bdrv_dirty_bitmap_size(bitmap), ++ begin + UINT32_MAX + 1 - ++ bdrv_dirty_bitmap_granularity(bitmap)); ++ } ++ if (dont_fragment && end > overall_end) { ++ end = overall_end; ++ } ++ ++ extents[i].length = cpu_to_be32(end - begin); ++ extents[i].flags = cpu_to_be32(dirty ? NBD_STATE_DIRTY : 0); ++ i++; ++ begin = end; ++ dirty = !dirty; ++ } ++ ++ bdrv_dirty_iter_free(it); ++ ++ bdrv_dirty_bitmap_unlock(bitmap); ++ ++ *length = end - offset; ++ return i; ++} ++ ++static int nbd_co_send_bitmap(NBDClient *client, uint64_t handle, ++ BdrvDirtyBitmap *bitmap, uint64_t offset, ++ uint32_t length, bool dont_fragment, bool last, ++ uint32_t context_id, Error **errp) ++{ ++ int ret; ++ unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BITMAP_EXTENTS; ++ NBDExtent *extents = g_new(NBDExtent, nb_extents); ++ uint64_t final_length = length; ++ ++ nb_extents = bitmap_to_extents(bitmap, offset, &final_length, extents, ++ nb_extents, dont_fragment); ++ ++ ret = nbd_co_send_extents(client, handle, extents, nb_extents, ++ final_length, last, context_id, errp); ++ ++ g_free(extents); ++ ++ return ret; + } + + /* nbd_co_receive_request +@@ -2051,11 +2219,34 @@ static coroutine_fn int nbd_handle_request(NBDClient *client, + return nbd_send_generic_reply(client, request->handle, -EINVAL, + "need non-zero length", errp); + } +- if (client->export_meta.valid && client->export_meta.base_allocation) { +- return nbd_co_send_block_status(client, request->handle, +- blk_bs(exp->blk), request->from, +- request->len, +- NBD_META_ID_BASE_ALLOCATION, errp); ++ if (client->export_meta.valid && ++ (client->export_meta.base_allocation || ++ client->export_meta.bitmap)) ++ { ++ if (client->export_meta.base_allocation) { ++ ret = nbd_co_send_block_status(client, request->handle, ++ blk_bs(exp->blk), request->from, ++ request->len, ++ !client->export_meta.bitmap, ++ NBD_META_ID_BASE_ALLOCATION, ++ errp); ++ if (ret < 0) { ++ return ret; ++ } ++ } ++ ++ if (client->export_meta.bitmap) { ++ ret = nbd_co_send_bitmap(client, request->handle, ++ client->exp->export_bitmap, ++ request->from, request->len, ++ request->flags & NBD_CMD_FLAG_REQ_ONE, ++ true, NBD_META_ID_DIRTY_BITMAP, errp); ++ if (ret < 0) { ++ return ret; ++ } ++ } ++ ++ return ret; + } else { + return nbd_send_generic_reply(client, request->handle, -EINVAL, + "CMD_BLOCK_STATUS not negotiated", +@@ -2207,3 +2398,44 @@ void nbd_client_new(NBDExport *exp, + co = qemu_coroutine_create(nbd_co_client_start, client); + qemu_coroutine_enter(co); + } ++ ++void nbd_export_bitmap(NBDExport *exp, const char *bitmap, ++ const char *bitmap_export_name, Error **errp) ++{ ++ BdrvDirtyBitmap *bm = NULL; ++ BlockDriverState *bs = blk_bs(exp->blk); ++ ++ if (exp->export_bitmap) { ++ error_setg(errp, "Export bitmap is already set"); ++ return; ++ } ++ ++ while (true) { ++ bm = bdrv_find_dirty_bitmap(bs, bitmap); ++ if (bm != NULL || bs->backing == NULL) { ++ break; ++ } ++ ++ bs = bs->backing->bs; ++ } ++ ++ if (bm == NULL) { ++ error_setg(errp, "Bitmap '%s' is not found", bitmap); ++ return; ++ } ++ ++ if (bdrv_dirty_bitmap_enabled(bm)) { ++ error_setg(errp, "Bitmap '%s' is enabled", bitmap); ++ return; ++ } ++ ++ if (bdrv_dirty_bitmap_qmp_locked(bm)) { ++ error_setg(errp, "Bitmap '%s' is locked", bitmap); ++ return; ++ } ++ ++ bdrv_dirty_bitmap_set_qmp_locked(bm, true); ++ exp->export_bitmap = bm; ++ exp->export_bitmap_context = ++ g_strdup_printf("qemu:dirty-bitmap:%s", bitmap_export_name); ++} +diff --git a/nbd/trace-events b/nbd/trace-events +index dee081e..5e1d4af 100644 +--- a/nbd/trace-events ++++ b/nbd/trace-events +@@ -64,6 +64,7 @@ nbd_co_send_simple_reply(uint64_t handle, uint32_t error, const char *errname, i + nbd_co_send_structured_done(uint64_t handle) "Send structured reply done: handle = %" PRIu64 + nbd_co_send_structured_read(uint64_t handle, uint64_t offset, void *data, size_t size) "Send structured read data reply: handle = %" PRIu64 ", offset = %" PRIu64 ", data = %p, len = %zu" + nbd_co_send_structured_read_hole(uint64_t handle, uint64_t offset, size_t size) "Send structured read hole reply: handle = %" PRIu64 ", offset = %" PRIu64 ", len = %zu" ++nbd_co_send_extents(uint64_t handle, unsigned int extents, uint32_t id, uint64_t length, int last) "Send block status reply: handle = %" PRIu64 ", extents = %u, context = %d (extents cover %" PRIu64 " bytes, last chunk = %d)" + nbd_co_send_structured_error(uint64_t handle, int err, const char *errname, const char *msg) "Send structured error reply: handle = %" PRIu64 ", error = %d (%s), msg = '%s'" + nbd_co_receive_request_decode_type(uint64_t handle, uint16_t type, const char *name) "Decoding type: handle = %" PRIu64 ", type = %" PRIu16 " (%s)" + nbd_co_receive_request_payload_received(uint64_t handle, uint32_t len) "Payload received: handle = %" PRIu64 ", len = %" PRIu32 +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-introduce-NBD_CMD_CACHE.patch b/SOURCES/kvm-nbd-server-introduce-NBD_CMD_CACHE.patch new file mode 100644 index 0000000..77db4d5 --- /dev/null +++ b/SOURCES/kvm-nbd-server-introduce-NBD_CMD_CACHE.patch @@ -0,0 +1,121 @@ +From e0d145328022f5c0897a6d157400a9d071acc767 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:06 +0200 +Subject: [PATCH 248/268] nbd/server: introduce NBD_CMD_CACHE + +RH-Author: John Snow +Message-id: <20180718225511.14878-31-jsnow@redhat.com> +Patchwork-id: 81419 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 30/35] nbd/server: introduce NBD_CMD_CACHE +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Handle nbd CACHE command. Just do read, without sending read data back. +Cache mechanism should be done by exported node driver chain. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180413143156.11409-1-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: fix two missing case labels in switch statements] +Signed-off-by: Eric Blake +(cherry picked from commit bc37b06a5cde24fb24c2a2cc44dd86756034ba9d) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + include/block/nbd.h | 3 ++- + nbd/common.c | 2 ++ + nbd/server.c | 11 +++++++---- + 3 files changed, 11 insertions(+), 5 deletions(-) + +diff --git a/include/block/nbd.h b/include/block/nbd.h +index 8bb9606..daaeae6 100644 +--- a/include/block/nbd.h ++++ b/include/block/nbd.h +@@ -135,6 +135,7 @@ typedef struct NBDExtent { + #define NBD_FLAG_SEND_TRIM (1 << 5) /* Send TRIM (discard) */ + #define NBD_FLAG_SEND_WRITE_ZEROES (1 << 6) /* Send WRITE_ZEROES */ + #define NBD_FLAG_SEND_DF (1 << 7) /* Send DF (Do not Fragment) */ ++#define NBD_FLAG_SEND_CACHE (1 << 8) /* Send CACHE (prefetch) */ + + /* New-style handshake (global) flags, sent from server to client, and + control what will happen during handshake phase. */ +@@ -195,7 +196,7 @@ enum { + NBD_CMD_DISC = 2, + NBD_CMD_FLUSH = 3, + NBD_CMD_TRIM = 4, +- /* 5 reserved for failed experiment NBD_CMD_CACHE */ ++ NBD_CMD_CACHE = 5, + NBD_CMD_WRITE_ZEROES = 6, + NBD_CMD_BLOCK_STATUS = 7, + }; +diff --git a/nbd/common.c b/nbd/common.c +index 8c95c1d..41f5ed8 100644 +--- a/nbd/common.c ++++ b/nbd/common.c +@@ -148,6 +148,8 @@ const char *nbd_cmd_lookup(uint16_t cmd) + return "flush"; + case NBD_CMD_TRIM: + return "trim"; ++ case NBD_CMD_CACHE: ++ return "cache"; + case NBD_CMD_WRITE_ZEROES: + return "write zeroes"; + case NBD_CMD_BLOCK_STATUS: +diff --git a/nbd/server.c b/nbd/server.c +index 2c2d62c..2746046 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -1252,7 +1252,7 @@ static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp) + int ret; + const uint16_t myflags = (NBD_FLAG_HAS_FLAGS | NBD_FLAG_SEND_TRIM | + NBD_FLAG_SEND_FLUSH | NBD_FLAG_SEND_FUA | +- NBD_FLAG_SEND_WRITE_ZEROES); ++ NBD_FLAG_SEND_WRITE_ZEROES | NBD_FLAG_SEND_CACHE); + bool oldStyle; + + /* Old style negotiation header, no room for options +@@ -2034,7 +2034,9 @@ static int nbd_co_receive_request(NBDRequestData *req, NBDRequest *request, + return -EIO; + } + +- if (request->type == NBD_CMD_READ || request->type == NBD_CMD_WRITE) { ++ if (request->type == NBD_CMD_READ || request->type == NBD_CMD_WRITE || ++ request->type == NBD_CMD_CACHE) ++ { + if (request->len > NBD_MAX_BUFFER_SIZE) { + error_setg(errp, "len (%" PRIu32" ) is larger than max len (%u)", + request->len, NBD_MAX_BUFFER_SIZE); +@@ -2119,7 +2121,7 @@ static coroutine_fn int nbd_do_cmd_read(NBDClient *client, NBDRequest *request, + int ret; + NBDExport *exp = client->exp; + +- assert(request->type == NBD_CMD_READ); ++ assert(request->type == NBD_CMD_READ || request->type == NBD_CMD_CACHE); + + /* XXX: NBD Protocol only documents use of FUA with WRITE */ + if (request->flags & NBD_CMD_FLAG_FUA) { +@@ -2138,7 +2140,7 @@ static coroutine_fn int nbd_do_cmd_read(NBDClient *client, NBDRequest *request, + + ret = blk_pread(exp->blk, request->from + exp->dev_offset, data, + request->len); +- if (ret < 0) { ++ if (ret < 0 || request->type == NBD_CMD_CACHE) { + return nbd_send_generic_reply(client, request->handle, ret, + "reading from file failed", errp); + } +@@ -2171,6 +2173,7 @@ static coroutine_fn int nbd_handle_request(NBDClient *client, + + switch (request->type) { + case NBD_CMD_READ: ++ case NBD_CMD_CACHE: + return nbd_do_cmd_read(client, request, data, errp); + + case NBD_CMD_WRITE: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nbd-server-refactor-NBDExportMetaContexts.patch b/SOURCES/kvm-nbd-server-refactor-NBDExportMetaContexts.patch new file mode 100644 index 0000000..43691a8 --- /dev/null +++ b/SOURCES/kvm-nbd-server-refactor-NBDExportMetaContexts.patch @@ -0,0 +1,118 @@ +From cc9208123465166b5217d41f9a20e4b652463475 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:01 +0200 +Subject: [PATCH 243/268] nbd/server: refactor NBDExportMetaContexts + +RH-Author: John Snow +Message-id: <20180718225511.14878-26-jsnow@redhat.com> +Patchwork-id: 81413 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 25/35] nbd/server: refactor NBDExportMetaContexts +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Use NBDExport pointer instead of just export name: there is no need to +store a duplicated name in the struct; moreover, NBDExport will be used +further. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180609151758.17343-3-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: commit message grammar tweak] +Signed-off-by: Eric Blake +(cherry picked from commit af736e546717d832168dd332a328bfcf74a0ab3d) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + nbd/server.c | 23 +++++++++++------------ + 1 file changed, 11 insertions(+), 12 deletions(-) + +diff --git a/nbd/server.c b/nbd/server.c +index 942c016..26cc41a 100644 +--- a/nbd/server.c ++++ b/nbd/server.c +@@ -88,7 +88,7 @@ static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports); + * as selected by NBD_OPT_SET_META_CONTEXT. Also used for + * NBD_OPT_LIST_META_CONTEXT. */ + typedef struct NBDExportMetaContexts { +- char export_name[NBD_MAX_NAME_SIZE + 1]; ++ NBDExport *exp; + bool valid; /* means that negotiation of the option finished without + errors */ + bool base_allocation; /* export base:allocation context (block status) */ +@@ -399,10 +399,9 @@ static int nbd_negotiate_handle_list(NBDClient *client, Error **errp) + return nbd_negotiate_send_rep(client, NBD_REP_ACK, errp); + } + +-static void nbd_check_meta_export_name(NBDClient *client) ++static void nbd_check_meta_export(NBDClient *client) + { +- client->export_meta.valid &= !strcmp(client->exp->name, +- client->export_meta.export_name); ++ client->export_meta.valid &= client->exp == client->export_meta.exp; + } + + /* Send a reply to NBD_OPT_EXPORT_NAME. +@@ -456,7 +455,7 @@ static int nbd_negotiate_handle_export_name(NBDClient *client, + + QTAILQ_INSERT_TAIL(&client->exp->clients, client, next); + nbd_export_get(client->exp); +- nbd_check_meta_export_name(client); ++ nbd_check_meta_export(client); + + return 0; + } +@@ -650,7 +649,7 @@ static int nbd_negotiate_handle_info(NBDClient *client, uint16_t myflags, + client->exp = exp; + QTAILQ_INSERT_TAIL(&client->exp->clients, client, next); + nbd_export_get(client->exp); +- nbd_check_meta_export_name(client); ++ nbd_check_meta_export(client); + rc = 1; + } + return rc; +@@ -835,7 +834,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client, + NBDExportMetaContexts *meta, Error **errp) + { + int ret; +- NBDExport *exp; ++ char export_name[NBD_MAX_NAME_SIZE + 1]; + NBDExportMetaContexts local_meta; + uint32_t nb_queries; + int i; +@@ -854,15 +853,15 @@ static int nbd_negotiate_meta_queries(NBDClient *client, + + memset(meta, 0, sizeof(*meta)); + +- ret = nbd_opt_read_name(client, meta->export_name, NULL, errp); ++ ret = nbd_opt_read_name(client, export_name, NULL, errp); + if (ret <= 0) { + return ret; + } + +- exp = nbd_export_find(meta->export_name); +- if (exp == NULL) { ++ meta->exp = nbd_export_find(export_name); ++ if (meta->exp == NULL) { + return nbd_opt_drop(client, NBD_REP_ERR_UNKNOWN, errp, +- "export '%s' not present", meta->export_name); ++ "export '%s' not present", export_name); + } + + ret = nbd_opt_read(client, &nb_queries, sizeof(nb_queries), errp); +@@ -871,7 +870,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client, + } + cpu_to_be32s(&nb_queries); + trace_nbd_negotiate_meta_context(nbd_opt_lookup(client->opt), +- meta->export_name, nb_queries); ++ export_name, nb_queries); + + if (client->opt == NBD_OPT_LIST_META_CONTEXT && !nb_queries) { + /* enable all known contexts */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ne2000-fix-possible-out-of-bound-access-in-ne2000_re.patch b/SOURCES/kvm-ne2000-fix-possible-out-of-bound-access-in-ne2000_re.patch new file mode 100644 index 0000000..daf5148 --- /dev/null +++ b/SOURCES/kvm-ne2000-fix-possible-out-of-bound-access-in-ne2000_re.patch @@ -0,0 +1,63 @@ +From 22ab26a5dcda3ff217d543a4eff8dbad9160ce1c Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:58:56 +0000 +Subject: [PATCH 01/11] ne2000: fix possible out of bound access in + ne2000_receive +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-2-jasowang@redhat.com> +Patchwork-id: 83976 +O-Subject: [RHEL8 qemu-kvm PATCH 1/9] ne2000: fix possible out of bound access in ne2000_receive +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +In ne2000_receive(), we try to assign size_ to size which converts +from size_t to integer. This will cause troubles when size_ is greater +INT_MAX, this will lead a negative value in size and it can then pass +the check of size < MIN_BUF_SIZE which may lead out of bound access of +for both buf and buf1. + +Fixing by converting the type of size to size_t. + +CC: qemu-stable@nongnu.org +Reported-by: Daniel Shapira +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Jason Wang +(cherry picked from commit fdc89e90fac40c5ca2686733df17b6423fb8d8fb) +Signed-off-by: Danilo C. L. de Paula +--- + hw/net/ne2000.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/hw/net/ne2000.c b/hw/net/ne2000.c +index 3a9fc89..19fac06 100644 +--- a/hw/net/ne2000.c ++++ b/hw/net/ne2000.c +@@ -173,7 +173,7 @@ static int ne2000_buffer_full(NE2000State *s) + ssize_t ne2000_receive(NetClientState *nc, const uint8_t *buf, size_t size_) + { + NE2000State *s = qemu_get_nic_opaque(nc); +- int size = size_; ++ size_t size = size_; + uint8_t *p; + unsigned int total_len, next, avail, len, index, mcast_idx; + uint8_t buf1[60]; +@@ -181,7 +181,7 @@ ssize_t ne2000_receive(NetClientState *nc, const uint8_t *buf, size_t size_) + { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; + + #if defined(DEBUG_NE2000) +- printf("NE2000: received len=%d\n", size); ++ printf("NE2000: received len=%zu\n", size); + #endif + + if (s->cmd & E8390_STOP || ne2000_buffer_full(s)) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-net-drop-too-large-packet-early.patch b/SOURCES/kvm-net-drop-too-large-packet-early.patch new file mode 100644 index 0000000..fabf035 --- /dev/null +++ b/SOURCES/kvm-net-drop-too-large-packet-early.patch @@ -0,0 +1,151 @@ +From 3f33dce627f419cbc77a9f6406b3353077697740 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:59:00 +0000 +Subject: [PATCH 05/11] net: drop too large packet early +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-6-jasowang@redhat.com> +Patchwork-id: 83979 +O-Subject: [RHEL8 qemu-kvm PATCH 5/9] net: drop too large packet early +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +We try to detect and drop too large packet (>INT_MAX) in 1592a9947036 +("net: ignore packet size greater than INT_MAX") during packet +delivering. Unfortunately, this is not sufficient as we may hit +another integer overflow when trying to queue such large packet in +qemu_net_queue_append_iov(): + +- size of the allocation may overflow on 32bit +- packet->size is integer which may overflow even on 64bit + +Fixing this by moving the check to qemu_sendv_packet_async() which is +the entrance of all networking codes and reduce the limit to +NET_BUFSIZE to be more conservative. This works since: + +- For the callers that call qemu_sendv_packet_async() directly, they + only care about if zero is returned to determine whether to prevent + the source from producing more packets. A callback will be triggered + if peer can accept more then source could be enabled. This is + usually used by high speed networking implementation like virtio-net + or netmap. +- For the callers that call qemu_sendv_packet() that calls + qemu_sendv_packet_async() indirectly, they often ignore the return + value. In this case qemu will just the drop packets if peer can't + receive. + +Qemu will copy the packet if it was queued. So it was safe for both +kinds of the callers to assume the packet was sent. + +Since we move the check from qemu_deliver_packet_iov() to +qemu_sendv_packet_async(), it would be safer to make +qemu_deliver_packet_iov() static to prevent any external user in the +future. + +This is a revised patch of CVE-2018-17963. + +Cc: qemu-stable@nongnu.org +Cc: Li Qiang +Fixes: 1592a9947036 ("net: ignore packet size greater than INT_MAX") +Reported-by: Li Qiang +Reviewed-by: Li Qiang +Signed-off-by: Jason Wang +Reviewed-by: Thomas Huth +Message-id: 20181204035347.6148-2-jasowang@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit 25c01bd19d0e4b66f357618aeefda1ef7a41e21a) +Signed-off-by: Danilo C. L. de Paula +--- + include/net/net.h | 6 ------ + net/net.c | 28 +++++++++++++++++----------- + 2 files changed, 17 insertions(+), 17 deletions(-) + +diff --git a/include/net/net.h b/include/net/net.h +index 1f7341e..df4df25 100644 +--- a/include/net/net.h ++++ b/include/net/net.h +@@ -170,12 +170,6 @@ void qemu_check_nic_model(NICInfo *nd, const char *model); + int qemu_find_nic_model(NICInfo *nd, const char * const *models, + const char *default_model); + +-ssize_t qemu_deliver_packet_iov(NetClientState *sender, +- unsigned flags, +- const struct iovec *iov, +- int iovcnt, +- void *opaque); +- + void print_net_client(Monitor *mon, NetClientState *nc); + void hmp_info_network(Monitor *mon, const QDict *qdict); + void net_socket_rs_init(SocketReadState *rs, +diff --git a/net/net.c b/net/net.c +index c991243..6e5c335 100644 +--- a/net/net.c ++++ b/net/net.c +@@ -231,6 +231,11 @@ static void qemu_net_client_destructor(NetClientState *nc) + { + g_free(nc); + } ++static ssize_t qemu_deliver_packet_iov(NetClientState *sender, ++ unsigned flags, ++ const struct iovec *iov, ++ int iovcnt, ++ void *opaque); + + static void qemu_net_client_setup(NetClientState *nc, + NetClientInfo *info, +@@ -705,22 +710,18 @@ static ssize_t nc_sendv_compat(NetClientState *nc, const struct iovec *iov, + return ret; + } + +-ssize_t qemu_deliver_packet_iov(NetClientState *sender, +- unsigned flags, +- const struct iovec *iov, +- int iovcnt, +- void *opaque) ++static ssize_t qemu_deliver_packet_iov(NetClientState *sender, ++ unsigned flags, ++ const struct iovec *iov, ++ int iovcnt, ++ void *opaque) + { + NetClientState *nc = opaque; +- size_t size = iov_size(iov, iovcnt); + int ret; + +- if (size > INT_MAX) { +- return size; +- } + + if (nc->link_down) { +- return size; ++ return iov_size(iov, iovcnt); + } + + if (nc->receive_disabled) { +@@ -745,10 +746,15 @@ ssize_t qemu_sendv_packet_async(NetClientState *sender, + NetPacketSent *sent_cb) + { + NetQueue *queue; ++ size_t size = iov_size(iov, iovcnt); + int ret; + ++ if (size > NET_BUFSIZE) { ++ return size; ++ } ++ + if (sender->link_down || !sender->peer) { +- return iov_size(iov, iovcnt); ++ return size; + } + + /* Let filters handle the packet first */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-net-hub-suppress-warnings-of-no-host-network-for-qte.patch b/SOURCES/kvm-net-hub-suppress-warnings-of-no-host-network-for-qte.patch new file mode 100644 index 0000000..d2e54a0 --- /dev/null +++ b/SOURCES/kvm-net-hub-suppress-warnings-of-no-host-network-for-qte.patch @@ -0,0 +1,63 @@ +From daf0de96b084099693fd56314d57747b389c7413 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:59:01 +0000 +Subject: [PATCH 06/11] net: hub: suppress warnings of no host network for + qtest +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-7-jasowang@redhat.com> +Patchwork-id: 83982 +O-Subject: [RHEL8 qemu-kvm PATCH 6/9] net: hub: suppress warnings of no host network for qtest +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +Notes: Conflict since we're lacking + 442da403ead80525761898ab0d8036a9cd3c6829 ("net: Get rid of + 'vlan' terminology and use 'hub' instead in the source files") + +If we want to qtest through hub, it would be much more simpler and +safer to configure the hub without host network. So silent this +warnings for qtest. + +Signed-off-by: Jason Wang +Reviewed-by: Thomas Huth +Message-id: 20181204035347.6148-3-jasowang@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit 56512e1dc1c6a00d37da09baa35981908fb9b5c7) +Signed-off-by: Jason Wang +Signed-off-by: Danilo C. L. de Paula +--- + net/hub.c | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/net/hub.c b/net/hub.c +index 5e84a9a..70a6de2 100644 +--- a/net/hub.c ++++ b/net/hub.c +@@ -20,6 +20,7 @@ + #include "hub.h" + #include "qemu/iov.h" + #include "qemu/error-report.h" ++#include "sysemu/qtest.h" + + /* + * A hub broadcasts incoming packets to all its ports except the source port. +@@ -347,7 +348,7 @@ void net_hub_check_clients(void) + if (has_host_dev && !has_nic) { + warn_report("vlan %d with no nics", hub->id); + } +- if (has_nic && !has_host_dev) { ++ if (has_nic && !has_host_dev && !qtest_enabled()) { + warn_report("vlan %d is not connected to host network", hub->id); + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-net-ignore-packet-size-greater-than-INT_MAX.patch b/SOURCES/kvm-net-ignore-packet-size-greater-than-INT_MAX.patch new file mode 100644 index 0000000..7457bbe --- /dev/null +++ b/SOURCES/kvm-net-ignore-packet-size-greater-than-INT_MAX.patch @@ -0,0 +1,57 @@ +From 725b20685e3faf11198560a6d9035c394ef77594 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:58:59 +0000 +Subject: [PATCH 04/11] net: ignore packet size greater than INT_MAX +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-5-jasowang@redhat.com> +Patchwork-id: 83978 +O-Subject: [RHEL8 qemu-kvm PATCH 4/9] net: ignore packet size greater than INT_MAX +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +There should not be a reason for passing a packet size greater than +INT_MAX. It's usually a hint of bug somewhere, so ignore packet size +greater than INT_MAX in qemu_deliver_packet_iov() + +CC: qemu-stable@nongnu.org +Reported-by: Daniel Shapira +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Jason Wang +(cherry picked from commit 1592a9947036d60dde5404204a5d45975133caf5) +Signed-off-by: Danilo C. L. de Paula +--- + net/net.c | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/net/net.c b/net/net.c +index 29f8398..c991243 100644 +--- a/net/net.c ++++ b/net/net.c +@@ -712,10 +712,15 @@ ssize_t qemu_deliver_packet_iov(NetClientState *sender, + void *opaque) + { + NetClientState *nc = opaque; ++ size_t size = iov_size(iov, iovcnt); + int ret; + ++ if (size > INT_MAX) { ++ return size; ++ } ++ + if (nc->link_down) { +- return iov_size(iov, iovcnt); ++ return size; + } + + if (nc->receive_disabled) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nfs-Fix-error-path-in-nfs_options_qdict_to_qapi.patch b/SOURCES/kvm-nfs-Fix-error-path-in-nfs_options_qdict_to_qapi.patch new file mode 100644 index 0000000..3249e43 --- /dev/null +++ b/SOURCES/kvm-nfs-Fix-error-path-in-nfs_options_qdict_to_qapi.patch @@ -0,0 +1,43 @@ +From 331d680a36525295a59ecbf73690985a8dccd3d5 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:47 +0200 +Subject: [PATCH 079/268] nfs: Fix error path in nfs_options_qdict_to_qapi() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-5-kwolf@redhat.com> +Patchwork-id: 81057 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 04/73] nfs: Fix error path in nfs_options_qdict_to_qapi() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Don't throw away local_err, but propagate it to errp. + +Signed-off-by: Kevin Wolf +Message-id: 20180516161034.27440-1-kwolf@redhat.com +Reviewed-by: Eric Blake +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit 54b7af4369a37afbd82573d0dcfb27febdb6dd24) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/nfs.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/block/nfs.c b/block/nfs.c +index 1e12958..3059ef2 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -568,6 +568,7 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, + visit_free(v); + + if (local_err) { ++ error_propagate(errp, local_err); + return NULL; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-nfs-Remove-processed-options-from-QDict.patch b/SOURCES/kvm-nfs-Remove-processed-options-from-QDict.patch new file mode 100644 index 0000000..ac3873c --- /dev/null +++ b/SOURCES/kvm-nfs-Remove-processed-options-from-QDict.patch @@ -0,0 +1,64 @@ +From b600b0cb2f849e3f7761a9470b1ce05a1ca2548f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:48 +0200 +Subject: [PATCH 080/268] nfs: Remove processed options from QDict + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-6-kwolf@redhat.com> +Patchwork-id: 81058 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 05/73] nfs: Remove processed options from QDict +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Commit c22a03454 QAPIfied option parsing in the NFS block driver, but +forgot to remove all the options we processed. Therefore, we get an +error in bdrv_open_inherit(), which thinks the remaining options are +invalid. Trying to open an NFS image will result in an error like this: + + Block protocol 'nfs' doesn't support the option 'server.host' + +Remove all options from the QDict to make the NFS driver work again. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Kevin Wolf +Message-id: 20180516160816.26259-1-kwolf@redhat.com +Reviewed-by: Eric Blake +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit c82be42cc803b36fd7aed5dceec68312c7056fd5) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/nfs.c | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/block/nfs.c b/block/nfs.c +index 3059ef2..743ca04 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -557,6 +557,7 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, + { + BlockdevOptionsNfs *opts = NULL; + Visitor *v; ++ const QDictEntry *e; + Error *local_err = NULL; + + v = qobject_input_visitor_new_flat_confused(options, errp); +@@ -572,6 +573,12 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, + return NULL; + } + ++ /* Remove the processed options from the QDict (the visitor processes ++ * _all_ options in the QDict) */ ++ while ((e = qdict_first(options))) { ++ qdict_del(options, e->key); ++ } ++ + return opts; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-numa-clarify-error-message-when-node-index-is-out-of.patch b/SOURCES/kvm-numa-clarify-error-message-when-node-index-is-out-of.patch new file mode 100644 index 0000000..14fd65a --- /dev/null +++ b/SOURCES/kvm-numa-clarify-error-message-when-node-index-is-out-of.patch @@ -0,0 +1,57 @@ +From 14578f9dce566ccbedaba80885043cc907bf740b Mon Sep 17 00:00:00 2001 +From: Igor Mammedov +Date: Mon, 2 Jul 2018 13:57:09 +0200 +Subject: [PATCH 181/268] numa: clarify error message when node index is out of + range in -numa dist, ... + +RH-Author: Igor Mammedov +Message-id: <1530539829-260581-1-git-send-email-imammedo@redhat.com> +Patchwork-id: 81184 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH] numa: clarify error message when node index is out of range in -numa dist, ... +Bugzilla: 1578381 +RH-Acked-by: Andrew Jones +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Pankaj Gupta + +When using following CLI: + -numa dist,src=128,dst=1,val=20 +user gets a rather confusing error message: + "Invalid node 128, max possible could be 128" + +Where 128 is number of nodes that QEMU supports (MAX_NODES), +while src/dst is an index up to that limit, so it should be +MAX_NODES - 1 in error message. +Make error message to explicitly state valid range for node +index to be more clear. + +Signed-off-by: Igor Mammedov +Message-Id: <1526483174-169008-1-git-send-email-imammedo@redhat.com> +Reviewed-by: Eric Blake +Signed-off-by: Eduardo Habkost +(cherry picked from commit 74f38e96b321ef8df2bf7fa1bd4f673ef06aca5b) +Signed-off-by: Igor Mammedov +Signed-off-by: Miroslav Rezanina +--- + numa.c | 5 ++--- + 1 file changed, 2 insertions(+), 3 deletions(-) + +diff --git a/numa.c b/numa.c +index 1116c90..a767a9d 100644 +--- a/numa.c ++++ b/numa.c +@@ -140,9 +140,8 @@ static void parse_numa_distance(NumaDistOptions *dist, Error **errp) + uint8_t val = dist->val; + + if (src >= MAX_NODES || dst >= MAX_NODES) { +- error_setg(errp, +- "Invalid node %d, max possible could be %d", +- MAX(src, dst), MAX_NODES); ++ error_setg(errp, "Parameter '%s' expects an integer between 0 and %d", ++ src >= MAX_NODES ? "src" : "dst", MAX_NODES - 1); + return; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-object-fix-OBJ_PROP_LINK_UNREF_ON_RELEASE-ambivalenc.patch b/SOURCES/kvm-object-fix-OBJ_PROP_LINK_UNREF_ON_RELEASE-ambivalenc.patch new file mode 100644 index 0000000..238191c --- /dev/null +++ b/SOURCES/kvm-object-fix-OBJ_PROP_LINK_UNREF_ON_RELEASE-ambivalenc.patch @@ -0,0 +1,335 @@ +From 206abaa45b1c845ef665f2639a8008b04a218165 Mon Sep 17 00:00:00 2001 +From: Serhii Popovych +Date: Mon, 9 Jul 2018 11:31:16 +0200 +Subject: [PATCH 200/268] object: fix OBJ_PROP_LINK_UNREF_ON_RELEASE + ambivalence +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Serhii Popovych +Message-id: <1531135878-18813-2-git-send-email-spopovyc@redhat.com> +Patchwork-id: 81265 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 1/3] object: fix OBJ_PROP_LINK_UNREF_ON_RELEASE ambivalence +Bugzilla: 1556678 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Marc-André Lureau +RH-Acked-by: David Gibson + +From: Marc-André Lureau + +A link property can be set during creation, with +object_property_add_link() and later with object_property_set_link(). + +add_link() doesn't add a reference to the target object, while +set_link() does. + +Furthemore, OBJ_PROP_LINK_UNREF_ON_RELEASE flags, set during add_link, +says whether a reference must be released when the property is destroyed. +This can lead to leaks if the property was later set_link(), as the +added reference is never released. + +Instead, rename OBJ_PROP_LINK_UNREF_ON_RELEASE to OBJ_PROP_LINK_STRONG +and use that has an indication on how the link handle reference +management in set_link(). + +Signed-off-by: Marc-André Lureau +Message-id: 20180531195119.22021-3-marcandre.lureau@redhat.com +Signed-off-by: Gerd Hoffmann +(cherry picked from commit 265b578c584b1a86c7028790deaa2f4392dd0a65) +Signed-off-by: Serhii Popovych +Signed-off-by: Miroslav Rezanina + +Conflicts: + hw/dma/xlnx-zdma.c + +This file is missing in 2.12.0. +--- + hw/core/qdev-properties.c | 2 +- + hw/core/qdev.c | 2 +- + hw/display/xlnx_dp.c | 2 +- + hw/dma/xilinx_axidma.c | 4 ++-- + hw/i386/pc.c | 2 +- + hw/i386/pc_piix.c | 2 +- + hw/i386/pc_q35.c | 2 +- + hw/ipmi/ipmi.c | 2 +- + hw/net/xilinx_axienet.c | 4 ++-- + hw/ssi/xilinx_spips.c | 2 +- + include/qom/object.h | 12 +++++++++--- + net/can/can_host.c | 2 +- + net/colo-compare.c | 2 +- + qom/object.c | 8 +++++--- + target/arm/cpu.c | 4 ++-- + ui/console.c | 2 +- + 16 files changed, 31 insertions(+), 23 deletions(-) + +diff --git a/hw/core/qdev-properties.c b/hw/core/qdev-properties.c +index 5bbc2d9..f3a83a3 100644 +--- a/hw/core/qdev-properties.c ++++ b/hw/core/qdev-properties.c +@@ -1309,7 +1309,7 @@ static void create_link_property(Object *obj, Property *prop, Error **errp) + object_property_add_link(obj, prop->name, prop->link_type, + child, + qdev_prop_allow_set_link_before_realize, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + errp); + } + +diff --git a/hw/core/qdev.c b/hw/core/qdev.c +index f6f9247..ce7c316 100644 +--- a/hw/core/qdev.c ++++ b/hw/core/qdev.c +@@ -435,7 +435,7 @@ void qdev_init_gpio_out_named(DeviceState *dev, qemu_irq *pins, + object_property_add_link(OBJECT(dev), propname, TYPE_IRQ, + (Object **)&pins[i], + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + g_free(propname); + } +diff --git a/hw/display/xlnx_dp.c b/hw/display/xlnx_dp.c +index 6715b9c..b737e1d 100644 +--- a/hw/display/xlnx_dp.c ++++ b/hw/display/xlnx_dp.c +@@ -1221,7 +1221,7 @@ static void xlnx_dp_init(Object *obj) + object_property_add_link(obj, "dpdma", TYPE_XLNX_DPDMA, + (Object **) &s->dpdma, + xlnx_dp_set_dpdma, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + + /* +diff --git a/hw/dma/xilinx_axidma.c b/hw/dma/xilinx_axidma.c +index 9b48103..401a328 100644 +--- a/hw/dma/xilinx_axidma.c ++++ b/hw/dma/xilinx_axidma.c +@@ -525,12 +525,12 @@ static void xilinx_axidma_realize(DeviceState *dev, Error **errp) + object_property_add_link(OBJECT(ds), "dma", TYPE_XILINX_AXI_DMA, + (Object **)&ds->dma, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &local_err); + object_property_add_link(OBJECT(cs), "dma", TYPE_XILINX_AXI_DMA, + (Object **)&cs->dma, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &local_err); + if (local_err) { + goto xilinx_axidma_realize_fail; +diff --git a/hw/i386/pc.c b/hw/i386/pc.c +index 6f686c7..9034f02 100644 +--- a/hw/i386/pc.c ++++ b/hw/i386/pc.c +@@ -484,7 +484,7 @@ void pc_cmos_init(PCMachineState *pcms, + TYPE_ISA_DEVICE, + (Object **)&pcms->rtc, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, &error_abort); ++ OBJ_PROP_LINK_STRONG, &error_abort); + object_property_set_link(OBJECT(pcms), OBJECT(s), + "rtc_state", &error_abort); + +diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c +index 229d551..60441c1 100644 +--- a/hw/i386/pc_piix.c ++++ b/hw/i386/pc_piix.c +@@ -290,7 +290,7 @@ static void pc_init1(MachineState *machine, + TYPE_HOTPLUG_HANDLER, + (Object **)&pcms->acpi_dev, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, &error_abort); ++ OBJ_PROP_LINK_STRONG, &error_abort); + object_property_set_link(OBJECT(machine), OBJECT(piix4_pm), + PC_MACHINE_ACPI_DEVICE_PROP, &error_abort); + } +diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c +index e1fd23e..ccdeb11 100644 +--- a/hw/i386/pc_q35.c ++++ b/hw/i386/pc_q35.c +@@ -194,7 +194,7 @@ static void pc_q35_init(MachineState *machine) + TYPE_HOTPLUG_HANDLER, + (Object **)&pcms->acpi_dev, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, &error_abort); ++ OBJ_PROP_LINK_STRONG, &error_abort); + object_property_set_link(OBJECT(machine), OBJECT(lpc), + PC_MACHINE_ACPI_DEVICE_PROP, &error_abort); + +diff --git a/hw/ipmi/ipmi.c b/hw/ipmi/ipmi.c +index 9be281f..63c0317 100644 +--- a/hw/ipmi/ipmi.c ++++ b/hw/ipmi/ipmi.c +@@ -104,7 +104,7 @@ void ipmi_bmc_find_and_link(Object *obj, Object **bmc) + { + object_property_add_link(obj, "bmc", TYPE_IPMI_BMC, bmc, + isa_ipmi_bmc_check, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + } + +diff --git a/hw/net/xilinx_axienet.c b/hw/net/xilinx_axienet.c +index d4c2c89..cc880a3 100644 +--- a/hw/net/xilinx_axienet.c ++++ b/hw/net/xilinx_axienet.c +@@ -951,12 +951,12 @@ static void xilinx_enet_realize(DeviceState *dev, Error **errp) + object_property_add_link(OBJECT(ds), "enet", "xlnx.axi-ethernet", + (Object **) &ds->enet, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &local_err); + object_property_add_link(OBJECT(cs), "enet", "xlnx.axi-ethernet", + (Object **) &cs->enet, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &local_err); + if (local_err) { + goto xilinx_enet_realize_fail; +diff --git a/hw/ssi/xilinx_spips.c b/hw/ssi/xilinx_spips.c +index 426f971..068878c 100644 +--- a/hw/ssi/xilinx_spips.c ++++ b/hw/ssi/xilinx_spips.c +@@ -1345,7 +1345,7 @@ static void xlnx_zynqmp_qspips_init(Object *obj) + object_property_add_link(obj, "stream-connected-dma", TYPE_STREAM_SLAVE, + (Object **)&rq->dma, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + NULL); + } + +diff --git a/include/qom/object.h b/include/qom/object.h +index 96ce81b..fc4555d 100644 +--- a/include/qom/object.h ++++ b/include/qom/object.h +@@ -1103,6 +1103,11 @@ char *object_property_get_str(Object *obj, const char *name, + * @errp: returns an error if this function fails + * + * Writes an object's canonical path to a property. ++ * ++ * If the link property was created with ++ * OBJ_PROP_LINK_STRONG bit, the old target object is ++ * unreferenced, and a reference is added to the new target object. ++ * + */ + void object_property_set_link(Object *obj, Object *value, + const char *name, Error **errp); +@@ -1393,7 +1398,7 @@ void object_property_add_child(Object *obj, const char *name, + + typedef enum { + /* Unref the link pointer when the property is deleted */ +- OBJ_PROP_LINK_UNREF_ON_RELEASE = 0x1, ++ OBJ_PROP_LINK_STRONG = 0x1, + } ObjectPropertyLinkFlags; + + /** +@@ -1431,8 +1436,9 @@ void object_property_allow_set_link(const Object *, const char *, + * link property. The reference count for *@child is + * managed by the property from after the function returns till the + * property is deleted with object_property_del(). If the +- * @flags OBJ_PROP_LINK_UNREF_ON_RELEASE bit is set, +- * the reference count is decremented when the property is deleted. ++ * @flags OBJ_PROP_LINK_STRONG bit is set, ++ * the reference count is decremented when the property is deleted or ++ * modified. + */ + void object_property_add_link(Object *obj, const char *name, + const char *type, Object **child, +diff --git a/net/can/can_host.c b/net/can/can_host.c +index c3d2652..c79347a 100644 +--- a/net/can/can_host.c ++++ b/net/can/can_host.c +@@ -77,7 +77,7 @@ static void can_host_instance_init(Object *obj) + object_property_add_link(obj, "canbus", TYPE_CAN_BUS, + (Object **)&ch->bus, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + } + +diff --git a/net/colo-compare.c b/net/colo-compare.c +index 23b2d2c..63469b1 100644 +--- a/net/colo-compare.c ++++ b/net/colo-compare.c +@@ -980,7 +980,7 @@ static void colo_compare_init(Object *obj) + object_property_add_link(obj, "iothread", TYPE_IOTHREAD, + (Object **)&s->iothread, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, NULL); ++ OBJ_PROP_LINK_STRONG, NULL); + + s->vnet_hdr = false; + object_property_add_bool(obj, "vnet_hdr_support", compare_get_vnet_hdr, +diff --git a/qom/object.c b/qom/object.c +index 76a89af..c4f1d36 100644 +--- a/qom/object.c ++++ b/qom/object.c +@@ -1564,9 +1564,11 @@ static void object_set_link_property(Object *obj, Visitor *v, + return; + } + +- object_ref(new_target); + *child = new_target; +- object_unref(old_target); ++ if (prop->flags == OBJ_PROP_LINK_STRONG) { ++ object_ref(new_target); ++ object_unref(old_target); ++ } + } + + static Object *object_resolve_link_property(Object *parent, void *opaque, const gchar *part) +@@ -1581,7 +1583,7 @@ static void object_release_link_property(Object *obj, const char *name, + { + LinkProperty *prop = opaque; + +- if ((prop->flags & OBJ_PROP_LINK_UNREF_ON_RELEASE) && *prop->child) { ++ if ((prop->flags & OBJ_PROP_LINK_STRONG) && *prop->child) { + object_unref(*prop->child); + } + g_free(prop); +diff --git a/target/arm/cpu.c b/target/arm/cpu.c +index 4255e9c..9d030e0 100644 +--- a/target/arm/cpu.c ++++ b/target/arm/cpu.c +@@ -672,7 +672,7 @@ static void arm_cpu_post_init(Object *obj) + TYPE_MEMORY_REGION, + (Object **)&cpu->secure_memory, + qdev_prop_allow_set_link_before_realize, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + #endif + } +@@ -700,7 +700,7 @@ static void arm_cpu_post_init(Object *obj) + if (arm_feature(&cpu->env, ARM_FEATURE_M_SECURITY)) { + object_property_add_link(obj, "idau", TYPE_IDAU_INTERFACE, &cpu->idau, + qdev_prop_allow_set_link_before_realize, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + qdev_property_add_static(DEVICE(obj), &arm_cpu_initsvtor_property, + &error_abort); +diff --git a/ui/console.c b/ui/console.c +index 3fb2f4e..594ec63 100644 +--- a/ui/console.c ++++ b/ui/console.c +@@ -1280,7 +1280,7 @@ static QemuConsole *new_console(DisplayState *ds, console_type_t console_type, + object_property_add_link(obj, "device", TYPE_DEVICE, + (Object **)&s->device, + object_property_allow_set_link, +- OBJ_PROP_LINK_UNREF_ON_RELEASE, ++ OBJ_PROP_LINK_STRONG, + &error_abort); + object_property_add_uint32_ptr(obj, "head", + &s->head, &error_abort); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-osdep-add-wait.h-compat-macros.patch b/SOURCES/kvm-osdep-add-wait.h-compat-macros.patch new file mode 100644 index 0000000..0ddbf61 --- /dev/null +++ b/SOURCES/kvm-osdep-add-wait.h-compat-macros.patch @@ -0,0 +1,59 @@ +From cf155192becebab6187749a4df4b8e33e5fc1e68 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:39 +0200 +Subject: [PATCH 160/268] osdep: add wait.h compat macros + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-5-git-send-email-plai@redhat.com> +Patchwork-id: 80938 +O-Subject: [RHEL7.6 PATCH BZ 1526645 04/10] osdep: add wait.h compat macros +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: "Michael S. Tsirkin" + +Man page for WCOREDUMP says: + + WCOREDUMP(wstatus) returns true if the child produced a core dump. + This macro should be employed only if WIFSIGNALED returned true. + + This macro is not specified in POSIX.1-2001 and is not + available on some UNIX implementations (e.g., AIX, SunOS). Therefore, + enclose its use inside #ifdef WCOREDUMP ... #endif. + +Let's do exactly this. + +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 28012e190e6897cfc2a98364240909d08e90ffc0) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + include/qemu/osdep.h | 10 ++++++++++ + 1 file changed, 10 insertions(+) + +diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h +index 5910682..9ed6242 100644 +--- a/include/qemu/osdep.h ++++ b/include/qemu/osdep.h +@@ -108,6 +108,16 @@ extern int daemon(int, int); + #include "qemu/typedefs.h" + + /* ++ * According to waitpid man page: ++ * WCOREDUMP ++ * This macro is not specified in POSIX.1-2001 and is not ++ * available on some UNIX implementations (e.g., AIX, SunOS). ++ * Therefore, enclose its use inside #ifdef WCOREDUMP ... #endif. ++ */ ++#ifndef WCOREDUMP ++#define WCOREDUMP(status) 0 ++#endif ++/* + * We have a lot of unaudited code that may fail in strange ways, or + * even be a security risk during migration, if you disable assertions + * at compile-time. You may comment out these safety checks if you +-- +1.8.3.1 + diff --git a/SOURCES/kvm-osdep-powerpc64-align-memory-to-allow-2MB-radix-THP-.patch b/SOURCES/kvm-osdep-powerpc64-align-memory-to-allow-2MB-radix-THP-.patch new file mode 100644 index 0000000..4855e33 --- /dev/null +++ b/SOURCES/kvm-osdep-powerpc64-align-memory-to-allow-2MB-radix-THP-.patch @@ -0,0 +1,51 @@ +From e54816f13676b09c4bd8899b466f8aafb9cf3d0e Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Wed, 25 Jul 2018 07:35:49 +0100 +Subject: [PATCH 02/14] osdep: powerpc64 align memory to allow 2MB radix THP + page tables + +RH-Author: David Gibson +Message-id: <20180725073549.9857-1-dgibson@redhat.com> +Patchwork-id: 81499 +O-Subject: [RHEL-8.0 qemu-kvm PATCH] osdep: powerpc64 align memory to allow 2MB radix THP page tables +Bugzilla: 1601317 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych + +From: Nicholas Piggin + +This allows KVM with the Book3S radix MMU mode to take advantage of +THP and install larger pages in the partition scope page tables (the +host translation). + +Signed-off-by: Nicholas Piggin +Signed-off-by: David Gibson +(cherry picked from commit 0c1272cc7c72dfe0ef66be8f283cf67c74b58586) + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1601317 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=17346107 + +Signed-off-by: David Gibson +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/osdep.h | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h +index 4165806..5910682 100644 +--- a/include/qemu/osdep.h ++++ b/include/qemu/osdep.h +@@ -357,7 +357,8 @@ void qemu_anon_ram_free(void *ptr, size_t size); + #endif + + #if defined(__linux__) && \ +- (defined(__x86_64__) || defined(__arm__) || defined(__aarch64__)) ++ (defined(__x86_64__) || defined(__arm__) || defined(__aarch64__) \ ++ || defined(__powerpc64__)) + /* Use 2 MiB alignment so transparent hugepages can be used by KVM. + Valgrind does not support alignments larger than 1 MiB, + therefore we need special code which handles running on Valgrind. */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pc-dimm-turn-alignment-assert-into-check.patch b/SOURCES/kvm-pc-dimm-turn-alignment-assert-into-check.patch new file mode 100644 index 0000000..4b3cf98 --- /dev/null +++ b/SOURCES/kvm-pc-dimm-turn-alignment-assert-into-check.patch @@ -0,0 +1,78 @@ +From c39a8362df8c09b9c66bb9c5295dc26373244fed Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Mon, 26 Nov 2018 09:57:34 +0000 +Subject: [PATCH 35/35] pc-dimm: turn alignment assert into check + +RH-Author: David Hildenbrand +Message-id: <20181126095734.30666-1-david@redhat.com> +Patchwork-id: 83163 +O-Subject: [RHEL-8.0 qemu-kvm PATCH] pc-dimm: turn alignment assert into check +Bugzilla: 1630116 +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Igor Mammedov + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1630116 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=19276337 +Upstream: 4d8938a05db15dea2c86c4ab9c5f872f160d2188 +Branch: rhel8/master-2.12.0 + +The start of the address space indicates which maximum alignment is +supported by our machine (e.g. ppc, x86 1GB). This is helpful to +catch fragmenting guest physical memory in strange fashions. + +Right now we can crash QEMU by e.g. (there might be easier examples) + +qemu-system-x86_64 -m 256M,maxmem=20G,slots=2 \ + -object memory-backend-file,id=mem0,size=8192M,mem-path=/dev/zero,align=8192M \ + -device pc-dimm,id=dimm1,memdev=mem0 + +Backport conflicts: + hw/mem/memory-device.c: The memory device refactoring is part of 3.0, + so fix it in previous pc-dimm code. +Note: The upstream patch missed a "x" (0% .. vs. 0x% ..), which was + fixed in 7c63ba2055a0 ("memory-device: fix alignment error message"), + however as this is not a clean cherry pick, I'm fixing it right + away (like in the RHEL7.6 backport). + +Signed-off-by: David Hildenbrand +Message-Id: <20180607154705.6316-2-david@redhat.com> +Reviewed-by: Michael S. Tsirkin +Reviewed-by: Igor Mammedov +Signed-off-by: Paolo Bonzini +(cherry picked from commit 4d8938a05db15dea2c86c4ab9c5f872f160d2188) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + hw/mem/pc-dimm.c | 9 +++++++-- + 1 file changed, 7 insertions(+), 2 deletions(-) + +diff --git a/hw/mem/pc-dimm.c b/hw/mem/pc-dimm.c +index 51350d9..a9d7c51 100644 +--- a/hw/mem/pc-dimm.c ++++ b/hw/mem/pc-dimm.c +@@ -298,14 +298,19 @@ uint64_t pc_dimm_get_free_addr(uint64_t address_space_start, + uint64_t new_addr, ret = 0; + uint64_t address_space_end = address_space_start + address_space_size; + +- g_assert(QEMU_ALIGN_UP(address_space_start, align) == address_space_start); +- + if (!address_space_size) { + error_setg(errp, "memory hotplug is not enabled, " + "please add maxmem option"); + goto out; + } + ++ /* address_space_start indicates the maximum alignment we expect */ ++ if (QEMU_ALIGN_UP(address_space_start, align) != address_space_start) { ++ error_setg(errp, "the alignment (0x%" PRIx64 ") is not supported", ++ align); ++ goto out; ++ } ++ + if (hint && QEMU_ALIGN_UP(*hint, align) != *hint) { + error_setg(errp, "address must be aligned to 0x%" PRIx64 " bytes", + align); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pc-x-migrate-smi-count-to-PC_RHEL_COMPAT.patch b/SOURCES/kvm-pc-x-migrate-smi-count-to-PC_RHEL_COMPAT.patch new file mode 100644 index 0000000..5acc32c --- /dev/null +++ b/SOURCES/kvm-pc-x-migrate-smi-count-to-PC_RHEL_COMPAT.patch @@ -0,0 +1,53 @@ +From 0fc66274cc8e22929a39e17b8d2280b222f1ae63 Mon Sep 17 00:00:00 2001 +From: "Dr. David Alan Gilbert" +Date: Fri, 14 Dec 2018 18:20:56 +0000 +Subject: [PATCH 4/5] pc: x-migrate-smi-count to PC_RHEL_COMPAT + +RH-Author: Dr. David Alan Gilbert +Message-id: <20181214182056.20233-3-dgilbert@redhat.com> +Patchwork-id: 83521 +O-Subject: [RHEL8 qemu-kvm PATCH 2/2] pc: x-migrate-smi-count to PC_RHEL_COMPAT +Bugzilla: 1659565 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Paolo Bonzini + +From: "Dr. David Alan Gilbert" + +MSR_SMI_COUNT started being migrated in QEMU 2.12 and in the 2.12 +release this broke back migration to earlier versions; however +that didn't cause a problem on RHEL because it also relied on newer +kernel features that RHEL 7.* doesn't have. + +QEMU 3.0 got a fix (in PC_COMPAT_2_11) to fix the 2.12->earlier +breakage, but given the kernel dependency, it makes more sense +for us to tie it to 8.* machine types and keep the feature off for +all 7.* machine types. + +In the 2.12 world we're doing this in PC_RHEL_COMPAT. + +Signed-off-by: Dr. David Alan Gilbert +Signed-off-by: Danilo C. L. de Paula +--- + include/hw/i386/pc.h | 5 +++++ + 1 file changed, 5 insertions(+) + +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index dd473ca..244d7b5 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -976,6 +976,11 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .driver = "vfio-pci",\ + .property = "x-no-geforce-quirks",\ + .value = "on",\ ++ },\ ++ { /* PC_RHEL_COMPAT from PC_COMPAT_2_11 bz 1659565 */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "x-migrate-smi-count",\ ++ .value = "off",\ + }, + + /* Similar to PC_COMPAT_2_11 + PC_COMPAT_2_10, but: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pcnet-fix-possible-buffer-overflow.patch b/SOURCES/kvm-pcnet-fix-possible-buffer-overflow.patch new file mode 100644 index 0000000..f11ed2b --- /dev/null +++ b/SOURCES/kvm-pcnet-fix-possible-buffer-overflow.patch @@ -0,0 +1,61 @@ +From 7da8c6f20d4838285af004884666c18f16fb331e Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:58:58 +0000 +Subject: [PATCH 03/11] pcnet: fix possible buffer overflow +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-4-jasowang@redhat.com> +Patchwork-id: 83977 +O-Subject: [RHEL8 qemu-kvm PATCH 3/9] pcnet: fix possible buffer overflow +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +In pcnet_receive(), we try to assign size_ to size which converts from +size_t to integer. This will cause troubles when size_ is greater +INT_MAX, this will lead a negative value in size and it can then pass +the check of size < MIN_BUF_SIZE which may lead out of bound access +for both buf and buf1. + +Fixing by converting the type of size to size_t. + +CC: qemu-stable@nongnu.org +Reported-by: Daniel Shapira +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Jason Wang +(cherry picked from commit b1d80d12c5f7ff081bb80ab4f4241d4248691192) +Signed-off-by: Danilo C. L. de Paula +--- + hw/net/pcnet.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/hw/net/pcnet.c b/hw/net/pcnet.c +index 0c44554..d9ba04b 100644 +--- a/hw/net/pcnet.c ++++ b/hw/net/pcnet.c +@@ -988,14 +988,14 @@ ssize_t pcnet_receive(NetClientState *nc, const uint8_t *buf, size_t size_) + uint8_t buf1[60]; + int remaining; + int crc_err = 0; +- int size = size_; ++ size_t size = size_; + + if (CSR_DRX(s) || CSR_STOP(s) || CSR_SPND(s) || !size || + (CSR_LOOP(s) && !s->looptest)) { + return -1; + } + #ifdef PCNET_DEBUG +- printf("pcnet_receive size=%d\n", size); ++ printf("pcnet_receive size=%zu\n", size); + #endif + + /* if too small buffer, then expand it */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-postcopy-Synchronize-usage-of-the-balloon-inhibitor.patch b/SOURCES/kvm-postcopy-Synchronize-usage-of-the-balloon-inhibitor.patch new file mode 100644 index 0000000..16380d7 --- /dev/null +++ b/SOURCES/kvm-postcopy-Synchronize-usage-of-the-balloon-inhibitor.patch @@ -0,0 +1,81 @@ +From 1e8f268e649136a29814db86735b3c20193bd133 Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:02:33 +0000 +Subject: [PATCH 16/16] postcopy: Synchronize usage of the balloon inhibitor + +RH-Author: Alex Williamson +Message-id: <154387455350.27651.11210229318068647811.stgit@gimli.home> +Patchwork-id: 83241 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 7/7] postcopy: Synchronize usage of the balloon inhibitor +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +While the qemu_balloon_inhibit() interface appears rather general purpose, +postcopy uses it in a last-caller-wins approach with no guarantee of balanced +inhibits and de-inhibits. Wrap postcopy's usage of the inhibitor to give it +one vote overall, using the same last-caller-wins approach as previously +implemented at the balloon level. + +Fixes: 01ccbec7bdf6 ("balloon: Allow multiple inhibit users") +Reported-by: Christian Borntraeger +Tested-by: Christian Borntraeger +Reviewed-by: Cornelia Huck +Reviewed-by: Juan Quintela +Signed-off-by: Alex Williamson +(cherry picked from commit 154304cd6e99e4222ed762976f9d9aca33c094d3) +Signed-off-by: Danilo C. L. de Paula +--- + migration/postcopy-ram.c | 18 ++++++++++++++++-- + 1 file changed, 16 insertions(+), 2 deletions(-) + +diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c +index 4a0b33b..b04e903 100644 +--- a/migration/postcopy-ram.c ++++ b/migration/postcopy-ram.c +@@ -400,6 +400,20 @@ int postcopy_ram_incoming_init(MigrationIncomingState *mis, size_t ram_pages) + } + + /* ++ * Manage a single vote to the QEMU balloon inhibitor for all postcopy usage, ++ * last caller wins. ++ */ ++static void postcopy_balloon_inhibit(bool state) ++{ ++ static bool cur_state = false; ++ ++ if (state != cur_state) { ++ qemu_balloon_inhibit(state); ++ cur_state = state; ++ } ++} ++ ++/* + * At the end of a migration where postcopy_ram_incoming_init was called. + */ + int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) +@@ -429,7 +443,7 @@ int postcopy_ram_incoming_cleanup(MigrationIncomingState *mis) + mis->have_fault_thread = false; + } + +- qemu_balloon_inhibit(false); ++ postcopy_balloon_inhibit(false); + + if (enable_mlock) { + if (os_mlock() < 0) { +@@ -801,7 +815,7 @@ int postcopy_ram_enable_notify(MigrationIncomingState *mis) + * Ballooning can mark pages as absent while we're postcopying + * that would cause false userfaults. + */ +- qemu_balloon_inhibit(true); ++ postcopy_balloon_inhibit(true); + + trace_postcopy_ram_enable_notify(); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ppc-spapr_caps-Add-SPAPR_CAP_NESTED_KVM_HV.patch b/SOURCES/kvm-ppc-spapr_caps-Add-SPAPR_CAP_NESTED_KVM_HV.patch new file mode 100644 index 0000000..4ea1804 --- /dev/null +++ b/SOURCES/kvm-ppc-spapr_caps-Add-SPAPR_CAP_NESTED_KVM_HV.patch @@ -0,0 +1,225 @@ +From 3c8c0e5bae7be14972cd38c470227d7037892a2c Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Mon, 12 Nov 2018 01:28:35 +0000 +Subject: [PATCH 04/16] ppc/spapr_caps: Add SPAPR_CAP_NESTED_KVM_HV + +RH-Author: David Gibson +Message-id: <20181112012835.21863-5-dgibson@redhat.com> +Patchwork-id: 82980 +O-Subject: [RHEL-8 qemu-kvm PATCH 4/4] ppc/spapr_caps: Add SPAPR_CAP_NESTED_KVM_HV +Bugzilla: 1639069 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych +RH-Acked-by: Thomas Huth + +From: Suraj Jitindar Singh + +Add the spapr cap SPAPR_CAP_NESTED_KVM_HV to be used to control the +availability of nested kvm-hv to the level 1 (L1) guest. + +Assuming a hypervisor with support enabled an L1 guest can be allowed to +use the kvm-hv module (and thus run it's own kvm-hv guests) by setting: +-machine pseries,cap-nested-hv=true +or disabled with: +-machine pseries,cap-nested-hv=false + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: David Gibson +(cherry picked from commit b9a477b725788e47bf653eab36e64f232d259f2a) +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + hw/ppc/spapr.c + hw/ppc/spapr_caps.c + include/hw/ppc/spapr.h + +We need some tweaks to renumber the nested KVM cap from upstream, +because we don't have the maximum page size capability downstream and +the code doesn't like sparse cap numbers. This is safe, because the +cap number does not appear in the migration stream, so it's strictly +local (the cap name is used for migration). + +We also work around a contextual conflict. + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1639069 + +Signed-off-by: David Gibson +--- + hw/ppc/spapr.c | 2 ++ + hw/ppc/spapr_caps.c | 32 ++++++++++++++++++++++++++++++++ + include/hw/ppc/spapr.h | 5 ++++- + target/ppc/kvm.c | 12 ++++++++++++ + target/ppc/kvm_ppc.h | 12 ++++++++++++ + 5 files changed, 62 insertions(+), 1 deletion(-) + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index a61dafd..ea72782 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -1816,6 +1816,7 @@ static const VMStateDescription vmstate_spapr = { + &vmstate_spapr_cap_cfpc, + &vmstate_spapr_cap_sbbc, + &vmstate_spapr_cap_ibs, ++ &vmstate_spapr_cap_nested_kvm_hv, + NULL + } + }; +@@ -3922,6 +3923,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) + smc->default_caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_BROKEN; + smc->default_caps.caps[SPAPR_CAP_SBBC] = SPAPR_CAP_BROKEN; + smc->default_caps.caps[SPAPR_CAP_IBS] = SPAPR_CAP_BROKEN; ++ smc->default_caps.caps[SPAPR_CAP_NESTED_KVM_HV] = SPAPR_CAP_OFF; + spapr_caps_add_properties(smc, &error_abort); + smc->has_power9_support = true; + } +diff --git a/hw/ppc/spapr_caps.c b/hw/ppc/spapr_caps.c +index 00e43a9..86a7947 100644 +--- a/hw/ppc/spapr_caps.c ++++ b/hw/ppc/spapr_caps.c +@@ -265,6 +265,28 @@ static void cap_safe_indirect_branch_apply(sPAPRMachineState *spapr, + + #define VALUE_DESC_TRISTATE " (broken, workaround, fixed)" + ++static void cap_nested_kvm_hv_apply(sPAPRMachineState *spapr, ++ uint8_t val, Error **errp) ++{ ++ if (!val) { ++ /* capability disabled by default */ ++ return; ++ } ++ ++ if (tcg_enabled()) { ++ error_setg(errp, ++ "No Nested KVM-HV support in tcg, try cap-nested-hv=off"); ++ } else if (kvm_enabled()) { ++ if (!kvmppc_has_cap_nested_kvm_hv()) { ++ error_setg(errp, ++"KVM implementation does not support Nested KVM-HV, try cap-nested-hv=off"); ++ } else if (kvmppc_set_cap_nested_kvm_hv(val) < 0) { ++ error_setg(errp, ++"Error enabling cap-nested-hv with KVM, try cap-nested-hv=off"); ++ } ++ } ++} ++ + sPAPRCapabilityInfo capability_table[SPAPR_CAP_NUM] = { + [SPAPR_CAP_HTM] = { + .name = "htm", +@@ -324,6 +346,15 @@ sPAPRCapabilityInfo capability_table[SPAPR_CAP_NUM] = { + .possible = &cap_ibs_possible, + .apply = cap_safe_indirect_branch_apply, + }, ++ [SPAPR_CAP_NESTED_KVM_HV] = { ++ .name = "nested-hv", ++ .description = "Allow Nested KVM-HV", ++ .index = SPAPR_CAP_NESTED_KVM_HV, ++ .get = spapr_cap_get_bool, ++ .set = spapr_cap_set_bool, ++ .type = "bool", ++ .apply = cap_nested_kvm_hv_apply, ++ }, + }; + + static sPAPRCapabilities default_caps_with_cpu(sPAPRMachineState *spapr, +@@ -439,6 +470,7 @@ SPAPR_CAP_MIG_STATE(dfp, SPAPR_CAP_DFP); + SPAPR_CAP_MIG_STATE(cfpc, SPAPR_CAP_CFPC); + SPAPR_CAP_MIG_STATE(sbbc, SPAPR_CAP_SBBC); + SPAPR_CAP_MIG_STATE(ibs, SPAPR_CAP_IBS); ++SPAPR_CAP_MIG_STATE(nested_kvm_hv, SPAPR_CAP_NESTED_KVM_HV); + + void spapr_caps_reset(sPAPRMachineState *spapr) + { +diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h +index 5118af6..beb42bc 100644 +--- a/include/hw/ppc/spapr.h ++++ b/include/hw/ppc/spapr.h +@@ -66,8 +66,10 @@ typedef enum { + #define SPAPR_CAP_SBBC 0x04 + /* Indirect Branch Serialisation */ + #define SPAPR_CAP_IBS 0x05 ++/* Nested KVM-HV */ ++#define SPAPR_CAP_NESTED_KVM_HV 0x06 + /* Num Caps */ +-#define SPAPR_CAP_NUM (SPAPR_CAP_IBS + 1) ++#define SPAPR_CAP_NUM (SPAPR_CAP_NESTED_KVM_HV + 1) + + /* + * Capability Values +@@ -794,6 +796,7 @@ extern const VMStateDescription vmstate_spapr_cap_dfp; + extern const VMStateDescription vmstate_spapr_cap_cfpc; + extern const VMStateDescription vmstate_spapr_cap_sbbc; + extern const VMStateDescription vmstate_spapr_cap_ibs; ++extern const VMStateDescription vmstate_spapr_cap_nested_kvm_hv; + + static inline uint8_t spapr_get_cap(sPAPRMachineState *spapr, int cap) + { +diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c +index 192c40d..058dcbe 100644 +--- a/target/ppc/kvm.c ++++ b/target/ppc/kvm.c +@@ -92,6 +92,7 @@ static int cap_ppc_pvr_compat; + static int cap_ppc_safe_cache; + static int cap_ppc_safe_bounds_check; + static int cap_ppc_safe_indirect_branch; ++static int cap_ppc_nested_kvm_hv; + + static uint32_t debug_inst_opcode; + +@@ -152,6 +153,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) + cap_mmu_hash_v3 = kvm_vm_check_extension(s, KVM_CAP_PPC_MMU_HASH_V3); + cap_resize_hpt = kvm_vm_check_extension(s, KVM_CAP_SPAPR_RESIZE_HPT); + kvmppc_get_cpu_characteristics(s); ++ cap_ppc_nested_kvm_hv = kvm_vm_check_extension(s, KVM_CAP_PPC_NESTED_HV); + /* + * Note: setting it to false because there is not such capability + * in KVM at this moment. +@@ -2552,6 +2554,16 @@ int kvmppc_get_cap_safe_indirect_branch(void) + return cap_ppc_safe_indirect_branch; + } + ++bool kvmppc_has_cap_nested_kvm_hv(void) ++{ ++ return !!cap_ppc_nested_kvm_hv; ++} ++ ++int kvmppc_set_cap_nested_kvm_hv(int enable) ++{ ++ return kvm_vm_enable_cap(kvm_state, KVM_CAP_PPC_NESTED_HV, 0, enable); ++} ++ + bool kvmppc_has_cap_spapr_vfio(void) + { + return cap_spapr_vfio; +diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h +index 4d2789e..dc86eff 100644 +--- a/target/ppc/kvm_ppc.h ++++ b/target/ppc/kvm_ppc.h +@@ -63,6 +63,8 @@ bool kvmppc_has_cap_mmu_hash_v3(void); + int kvmppc_get_cap_safe_cache(void); + int kvmppc_get_cap_safe_bounds_check(void); + int kvmppc_get_cap_safe_indirect_branch(void); ++bool kvmppc_has_cap_nested_kvm_hv(void); ++int kvmppc_set_cap_nested_kvm_hv(int enable); + int kvmppc_enable_hwrng(void); + int kvmppc_put_books_sregs(PowerPCCPU *cpu); + PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void); +@@ -314,6 +316,16 @@ static inline int kvmppc_get_cap_safe_indirect_branch(void) + return 0; + } + ++static inline bool kvmppc_has_cap_nested_kvm_hv(void) ++{ ++ return false; ++} ++ ++static inline int kvmppc_set_cap_nested_kvm_hv(int enable) ++{ ++ return -1; ++} ++ + static inline int kvmppc_enable_hwrng(void) + { + return -1; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ppc-spapr_caps-Don-t-disable-cap_cfpc-on-POWER8-by-d.patch b/SOURCES/kvm-ppc-spapr_caps-Don-t-disable-cap_cfpc-on-POWER8-by-d.patch new file mode 100644 index 0000000..042e4ff --- /dev/null +++ b/SOURCES/kvm-ppc-spapr_caps-Don-t-disable-cap_cfpc-on-POWER8-by-d.patch @@ -0,0 +1,60 @@ +From fc0c82b8d52b1d5a5da0cef66a54b9e277084663 Mon Sep 17 00:00:00 2001 +From: Suraj Jitindar Singh +Date: Thu, 21 Jun 2018 06:56:49 +0200 +Subject: [PATCH 062/268] ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by + default + +RH-Author: Suraj Jitindar Singh +Message-id: <1529564209-30369-4-git-send-email-sursingh@redhat.com> +Patchwork-id: 80929 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 3/3] ppc/spapr_caps: Don't disable cap_cfpc on POWER8 by default +Bugzilla: 1560847 +RH-Acked-by: Laurent Vivier +RH-Acked-by: David Gibson +RH-Acked-by: Miroslav Rezanina + +From: Suraj Jitindar Singh + +In default_caps_with_cpu() we set spapr_cap_cfpc to broken for POWER8 +processors and before. + +Since we no longer require private l1d cache on POWER8 for this cap to +be set to workaround change this to default to broken for POWER7 +processors and before. + +Signed-off-by: Suraj Jitindar Singh +Reviewed-by: David Gibson +Signed-off-by: David Gibson +(cherry picked from commit b2540203bdf4a390c3489146eae82ce237303653) + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1560847 + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: Miroslav Rezanina +--- + hw/ppc/spapr_caps.c | 6 +----- + 1 file changed, 1 insertion(+), 5 deletions(-) + +diff --git a/hw/ppc/spapr_caps.c b/hw/ppc/spapr_caps.c +index 531e145..00e43a9 100644 +--- a/hw/ppc/spapr_caps.c ++++ b/hw/ppc/spapr_caps.c +@@ -335,14 +335,10 @@ static sPAPRCapabilities default_caps_with_cpu(sPAPRMachineState *spapr, + + caps = smc->default_caps; + +- if (!ppc_check_compat(cpu, CPU_POWERPC_LOGICAL_3_00, +- 0, spapr->max_compat_pvr)) { +- caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_BROKEN; +- } +- + if (!ppc_check_compat(cpu, CPU_POWERPC_LOGICAL_2_07, + 0, spapr->max_compat_pvr)) { + caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF; ++ caps.caps[SPAPR_CAP_CFPC] = SPAPR_CAP_BROKEN; + } + + if (!ppc_check_compat(cpu, CPU_POWERPC_LOGICAL_2_06_PLUS, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-helper-Rework-socket-path-handling.patch b/SOURCES/kvm-pr-helper-Rework-socket-path-handling.patch new file mode 100644 index 0000000..0701931 --- /dev/null +++ b/SOURCES/kvm-pr-helper-Rework-socket-path-handling.patch @@ -0,0 +1,141 @@ +From 50e7aa5e8892cd2f0e3197cc5a0bcf289af7f16d Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:58 +0200 +Subject: [PATCH 198/268] pr-helper: Rework socket path handling + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-9-pbonzini@redhat.com> +Patchwork-id: 81252 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 8/9] pr-helper: Rework socket path handling +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +From: Michal Privoznik + +When reviewing Paolo's pr-helper patches I've noticed couple of +problems: + +1) socket_path needs to be calculated at two different places +(one for printing out help, the other if socket activation is NOT +used), + +2) even though the default socket_path is allocated in +compute_default_paths() it is the only default path the function +handles. For instance, pidfile is allocated outside of this +function. And yet again, at different places than 1) + +Signed-off-by: Michal Privoznik +Message-Id: +Signed-off-by: Paolo Bonzini +(cherry picked from commit 2729d79d4993099782002c9a218de1fc12c32c69) +Signed-off-by: Miroslav Rezanina +--- + scsi/qemu-pr-helper.c | 36 ++++++++++-------------------------- + 1 file changed, 10 insertions(+), 26 deletions(-) + +diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c +index c89a446..1528a71 100644 +--- a/scsi/qemu-pr-helper.c ++++ b/scsi/qemu-pr-helper.c +@@ -76,14 +76,12 @@ static int gid = -1; + + static void compute_default_paths(void) + { +- if (!socket_path) { +- socket_path = qemu_get_local_state_pathname("run/qemu-pr-helper.sock"); +- } ++ socket_path = qemu_get_local_state_pathname("run/qemu-pr-helper.sock"); ++ pidfile = qemu_get_local_state_pathname("run/qemu-pr-helper.pid"); + } + + static void usage(const char *name) + { +- compute_default_paths(); + (printf) ( + "Usage: %s [OPTIONS] FILE\n" + "Persistent Reservation helper program for QEMU\n" +@@ -841,19 +839,6 @@ static gboolean accept_client(QIOChannel *ioc, GIOCondition cond, gpointer opaqu + return TRUE; + } + +- +-/* +- * Check socket parameters compatibility when socket activation is used. +- */ +-static const char *socket_activation_validate_opts(void) +-{ +- if (socket_path != NULL) { +- return "Unix socket can't be set when using socket activation"; +- } +- +- return NULL; +-} +- + static void termsig_handler(int signum) + { + atomic_cmpxchg(&state, RUNNING, TERMINATE); +@@ -927,6 +912,7 @@ int main(int argc, char **argv) + char *trace_file = NULL; + bool daemonize = false; + bool pidfile_specified = false; ++ bool socket_path_specified = false; + unsigned socket_activation; + + struct sigaction sa_sigterm; +@@ -943,12 +929,14 @@ int main(int argc, char **argv) + qemu_add_opts(&qemu_trace_opts); + qemu_init_exec_dir(argv[0]); + +- pidfile = qemu_get_local_state_pathname("run/qemu-pr-helper.pid"); ++ compute_default_paths(); + + while ((ch = getopt_long(argc, argv, sopt, lopt, &opt_ind)) != -1) { + switch (ch) { + case 'k': +- socket_path = optarg; ++ g_free(socket_path); ++ socket_path = g_strdup(optarg); ++ socket_path_specified = true; + if (socket_path[0] != '/') { + error_report("socket path must be absolute"); + exit(EXIT_FAILURE); +@@ -1039,10 +1027,9 @@ int main(int argc, char **argv) + socket_activation = check_socket_activation(); + if (socket_activation == 0) { + SocketAddress saddr; +- compute_default_paths(); + saddr = (SocketAddress){ + .type = SOCKET_ADDRESS_TYPE_UNIX, +- .u.q_unix.path = g_strdup(socket_path) ++ .u.q_unix.path = socket_path, + }; + server_ioc = qio_channel_socket_new(); + if (qio_channel_socket_listen_sync(server_ioc, &saddr, &local_err) < 0) { +@@ -1050,12 +1037,10 @@ int main(int argc, char **argv) + error_report_err(local_err); + return 1; + } +- g_free(saddr.u.q_unix.path); + } else { + /* Using socket activation - check user didn't use -p etc. */ +- const char *err_msg = socket_activation_validate_opts(); +- if (err_msg != NULL) { +- error_report("%s", err_msg); ++ if (socket_path_specified) { ++ error_report("Unix socket can't be set when using socket activation"); + exit(EXIT_FAILURE); + } + +@@ -1072,7 +1057,6 @@ int main(int argc, char **argv) + error_get_pretty(local_err)); + exit(EXIT_FAILURE); + } +- socket_path = NULL; + } + + if (qemu_init_main_loop(&local_err)) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-helper-avoid-error-on-PR-IN-command-with-zero-req.patch b/SOURCES/kvm-pr-helper-avoid-error-on-PR-IN-command-with-zero-req.patch new file mode 100644 index 0000000..7c40abf --- /dev/null +++ b/SOURCES/kvm-pr-helper-avoid-error-on-PR-IN-command-with-zero-req.patch @@ -0,0 +1,139 @@ +From aec7894ed126514877d145ad4c3615a0b29e3443 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:57 +0200 +Subject: [PATCH 197/268] pr-helper: avoid error on PR IN command with zero + request size + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-8-pbonzini@redhat.com> +Patchwork-id: 81250 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 7/9] pr-helper: avoid error on PR IN command with zero request size +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +After reading a PR IN command with zero request size in prh_read_request, +the resp->result field will be uninitialized and the resp.sz field will +be also uninitialized when returning to prh_co_entry. + +If resp->result == GOOD (from a previous successful reply or just luck), +then the assert in prh_write_response might not be triggered and +uninitialized response will be sent. + +The fix is to remove the whole handling of sz == 0 in prh_co_entry. +Those errors apply only to PR OUT commands and it's perfectly okay to +catch them later in do_pr_out and multipath_pr_out; the check for +too-short parameters in fact doesn't apply in the easy SG_IO case, as +it can be left to the target firmware even. + +The result is that prh_read_request does not fail requests anymore and +prh_co_entry becomes simpler. + +Reported-by: Dima Stepanov +Signed-off-by: Paolo Bonzini +(cherry picked from commit ee8c13b81474e002db083e9692b11c0e106a9c7f) +Signed-off-by: Miroslav Rezanina +--- + scsi/qemu-pr-helper.c | 63 ++++++++++++++++++++++++--------------------------- + 1 file changed, 30 insertions(+), 33 deletions(-) + +diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c +index 0218d65..c89a446 100644 +--- a/scsi/qemu-pr-helper.c ++++ b/scsi/qemu-pr-helper.c +@@ -455,6 +455,14 @@ static int multipath_pr_out(int fd, const uint8_t *cdb, uint8_t *sense, + char transportids[PR_HELPER_DATA_SIZE]; + int r; + ++ if (sz < PR_OUT_FIXED_PARAM_SIZE) { ++ /* Illegal request, Parameter list length error. This isn't fatal; ++ * we have read the data, send an error without closing the socket. ++ */ ++ scsi_build_sense(sense, SENSE_CODE(INVALID_PARAM_LEN)); ++ return CHECK_CONDITION; ++ } ++ + switch (rq_servact) { + case MPATH_PROUT_REG_SA: + case MPATH_PROUT_RES_SA: +@@ -574,6 +582,12 @@ static int do_pr_out(int fd, const uint8_t *cdb, uint8_t *sense, + const uint8_t *param, int sz) + { + int resp_sz; ++ ++ if ((fcntl(fd, F_GETFL) & O_ACCMODE) == O_RDONLY) { ++ scsi_build_sense(sense, SENSE_CODE(INVALID_OPCODE)); ++ return CHECK_CONDITION; ++ } ++ + #ifdef CONFIG_MPATH + if (is_mpath(fd)) { + return multipath_pr_out(fd, cdb, sense, param, sz); +@@ -690,21 +704,6 @@ static int coroutine_fn prh_read_request(PRHelperClient *client, + errp) < 0) { + goto out_close; + } +- if ((fcntl(client->fd, F_GETFL) & O_ACCMODE) == O_RDONLY) { +- scsi_build_sense(resp->sense, SENSE_CODE(INVALID_OPCODE)); +- sz = 0; +- } else if (sz < PR_OUT_FIXED_PARAM_SIZE) { +- /* Illegal request, Parameter list length error. This isn't fatal; +- * we have read the data, send an error without closing the socket. +- */ +- scsi_build_sense(resp->sense, SENSE_CODE(INVALID_PARAM_LEN)); +- sz = 0; +- } +- if (sz == 0) { +- resp->result = CHECK_CONDITION; +- close(client->fd); +- client->fd = -1; +- } + } + + req->fd = client->fd; +@@ -785,25 +784,23 @@ static void coroutine_fn prh_co_entry(void *opaque) + break; + } + +- if (sz > 0) { +- num_active_sockets++; +- if (req.cdb[0] == PERSISTENT_RESERVE_OUT) { +- r = do_pr_out(req.fd, req.cdb, resp.sense, +- client->data, sz); +- resp.sz = 0; +- } else { +- resp.sz = sizeof(client->data); +- r = do_pr_in(req.fd, req.cdb, resp.sense, +- client->data, &resp.sz); +- resp.sz = MIN(resp.sz, sz); +- } +- num_active_sockets--; +- close(req.fd); +- if (r == -1) { +- break; +- } +- resp.result = r; ++ num_active_sockets++; ++ if (req.cdb[0] == PERSISTENT_RESERVE_OUT) { ++ r = do_pr_out(req.fd, req.cdb, resp.sense, ++ client->data, sz); ++ resp.sz = 0; ++ } else { ++ resp.sz = sizeof(client->data); ++ r = do_pr_in(req.fd, req.cdb, resp.sense, ++ client->data, &resp.sz); ++ resp.sz = MIN(resp.sz, sz); ++ } ++ num_active_sockets--; ++ close(req.fd); ++ if (r == -1) { ++ break; + } ++ resp.result = r; + + if (prh_write_response(client, &req, &resp, &local_err) < 0) { + break; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-helper-fix-assertion-failure-on-failed-multipath-.patch b/SOURCES/kvm-pr-helper-fix-assertion-failure-on-failed-multipath-.patch new file mode 100644 index 0000000..fb8c560 --- /dev/null +++ b/SOURCES/kvm-pr-helper-fix-assertion-failure-on-failed-multipath-.patch @@ -0,0 +1,69 @@ +From 0a37ee43ee671c7cbd02110f9e968dbd530d2a4e Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:52 +0200 +Subject: [PATCH 192/268] pr-helper: fix assertion failure on failed multipath + PERSISTENT RESERVE IN +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-3-pbonzini@redhat.com> +Patchwork-id: 81246 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 2/9] pr-helper: fix assertion failure on failed multipath PERSISTENT RESERVE IN +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +The response size is expected to be zero if the SCSI status is not +"GOOD", but nothing was resetting it. + +This can be reproduced simply by "sg_persist -s /dev/sdb" where /dev/sdb +in the guest is a scsi-block device corresponding to a multipath device +on the host. + +Before: + + PR in (Read full status): Aborted command + +and on the host: + + prh_write_response: Assertion `resp->sz == 0' failed. + +After: + + PR in (Read full status): bad field in cdb or parameter list + (perhaps unsupported service action) + +Reported-by: Jiri Belka +Reviewed-by: Michal Privoznik +Signed-off-by: Paolo Bonzini +Reviewed-by: Paolo Bonzini +Reviewed-by: Philippe Mathieu-Daudé +(cherry picked from commit 86933b4e7879e427e03365bf352c0964640cb37b) +Signed-off-by: Miroslav Rezanina +--- + scsi/qemu-pr-helper.c | 6 +++++- + 1 file changed, 5 insertions(+), 1 deletion(-) + +diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c +index 4057cf3..0218d65 100644 +--- a/scsi/qemu-pr-helper.c ++++ b/scsi/qemu-pr-helper.c +@@ -558,7 +558,11 @@ static int do_pr_in(int fd, const uint8_t *cdb, uint8_t *sense, + #ifdef CONFIG_MPATH + if (is_mpath(fd)) { + /* multipath_pr_in fills the whole input buffer. */ +- return multipath_pr_in(fd, cdb, sense, data, *resp_sz); ++ int r = multipath_pr_in(fd, cdb, sense, data, *resp_sz); ++ if (r != GOOD) { ++ *resp_sz = 0; ++ } ++ return r; + } + #endif + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-helper-fix-socket-path-default-in-help.patch b/SOURCES/kvm-pr-helper-fix-socket-path-default-in-help.patch new file mode 100644 index 0000000..1992ba5 --- /dev/null +++ b/SOURCES/kvm-pr-helper-fix-socket-path-default-in-help.patch @@ -0,0 +1,66 @@ +From b566763838f2f0e09932ceb45583adf64932f001 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:51 +0200 +Subject: [PATCH 191/268] pr-helper: fix --socket-path default in help +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-2-pbonzini@redhat.com> +Patchwork-id: 81244 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 1/9] pr-helper: fix --socket-path default in help +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +Currently --help shows "(default '(null)')" for the -k/--socket-path +option. Fix it by getting the default path in /var/run. + +Signed-off-by: Paolo Bonzini +Reviewed-by: Philippe Mathieu-Daudé +(cherry picked from commit 50fa332516d5e42695811f43396b749185e21b9c) +Signed-off-by: Miroslav Rezanina +--- + scsi/qemu-pr-helper.c | 15 ++++++++------- + 1 file changed, 8 insertions(+), 7 deletions(-) + +diff --git a/scsi/qemu-pr-helper.c b/scsi/qemu-pr-helper.c +index d0f8317..4057cf3 100644 +--- a/scsi/qemu-pr-helper.c ++++ b/scsi/qemu-pr-helper.c +@@ -74,8 +74,16 @@ static int uid = -1; + static int gid = -1; + #endif + ++static void compute_default_paths(void) ++{ ++ if (!socket_path) { ++ socket_path = qemu_get_local_state_pathname("run/qemu-pr-helper.sock"); ++ } ++} ++ + static void usage(const char *name) + { ++ compute_default_paths(); + (printf) ( + "Usage: %s [OPTIONS] FILE\n" + "Persistent Reservation helper program for QEMU\n" +@@ -845,13 +853,6 @@ static const char *socket_activation_validate_opts(void) + return NULL; + } + +-static void compute_default_paths(void) +-{ +- if (!socket_path) { +- socket_path = qemu_get_local_state_pathname("run/qemu-pr-helper.sock"); +- } +-} +- + static void termsig_handler(int signum) + { + atomic_cmpxchg(&state, RUNNING, TERMINATE); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-manager-add-query-pr-managers-QMP-command.patch b/SOURCES/kvm-pr-manager-add-query-pr-managers-QMP-command.patch new file mode 100644 index 0000000..1bdf2a6 --- /dev/null +++ b/SOURCES/kvm-pr-manager-add-query-pr-managers-QMP-command.patch @@ -0,0 +1,201 @@ +From e34b60deac41fc0c642598ea9e84e6c919fe2d02 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:55 +0200 +Subject: [PATCH 195/268] pr-manager: add query-pr-managers QMP command + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-6-pbonzini@redhat.com> +Patchwork-id: 81251 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 5/9] pr-manager: add query-pr-managers QMP command +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +This command lets you query the connection status of each pr-manager-helper +object. + +Signed-off-by: Paolo Bonzini +(cherry picked from commit 5f64089416f0d77c87683401838f064c51a292ed) + +[RHEL: no allow-preconfig yet, so don't specify it] + +Signed-off-by: Miroslav Rezanina +--- + include/scsi/pr-manager.h | 2 ++ + qapi/block.json | 27 +++++++++++++++++++++++++++ + scsi/pr-manager-helper.c | 13 +++++++++++++ + scsi/pr-manager-stub.c | 6 ++++++ + scsi/pr-manager.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ + 5 files changed, 93 insertions(+) + +diff --git a/include/scsi/pr-manager.h b/include/scsi/pr-manager.h +index 71971ae..50a77b0 100644 +--- a/include/scsi/pr-manager.h ++++ b/include/scsi/pr-manager.h +@@ -33,8 +33,10 @@ typedef struct PRManagerClass { + + /* */ + int (*run)(PRManager *pr_mgr, int fd, struct sg_io_hdr *hdr); ++ bool (*is_connected)(PRManager *pr_mgr); + } PRManagerClass; + ++bool pr_manager_is_connected(PRManager *pr_mgr); + BlockAIOCB *pr_manager_execute(PRManager *pr_mgr, + AioContext *ctx, int fd, + struct sg_io_hdr *hdr, +diff --git a/qapi/block.json b/qapi/block.json +index c694524..cf63ea2 100644 +--- a/qapi/block.json ++++ b/qapi/block.json +@@ -78,6 +78,33 @@ + 'data': { 'device': 'str', 'name': 'str' } } + + ## ++# @PRManagerInfo: ++# ++# Information about a persistent reservation manager ++# ++# @id: the identifier of the persistent reservation manager ++# ++# @connected: true if the persistent reservation manager is connected to ++# the underlying storage or helper ++# ++# Since: 3.0 ++## ++{ 'struct': 'PRManagerInfo', ++ 'data': {'id': 'str', 'connected': 'bool'} } ++ ++## ++# @query-pr-managers: ++# ++# Returns a list of information about each persistent reservation manager. ++# ++# Returns: a list of @PRManagerInfo for each persistent reservation manager ++# ++# Since: 3.0 ++## ++{ 'command': 'query-pr-managers', 'returns': ['PRManagerInfo'] } ++ ++ ++## + # @blockdev-snapshot-internal-sync: + # + # Synchronously take an internal snapshot of a block device, when the +diff --git a/scsi/pr-manager-helper.c b/scsi/pr-manager-helper.c +index 0c0fe38..b11481b 100644 +--- a/scsi/pr-manager-helper.c ++++ b/scsi/pr-manager-helper.c +@@ -235,6 +235,18 @@ out: + return ret; + } + ++static bool pr_manager_helper_is_connected(PRManager *p) ++{ ++ PRManagerHelper *pr_mgr = PR_MANAGER_HELPER(p); ++ bool result; ++ ++ qemu_mutex_lock(&pr_mgr->lock); ++ result = (pr_mgr->ioc != NULL); ++ qemu_mutex_unlock(&pr_mgr->lock); ++ ++ return result; ++} ++ + static void pr_manager_helper_complete(UserCreatable *uc, Error **errp) + { + PRManagerHelper *pr_mgr = PR_MANAGER_HELPER(uc); +@@ -284,6 +296,7 @@ static void pr_manager_helper_class_init(ObjectClass *klass, + &error_abort); + uc_klass->complete = pr_manager_helper_complete; + prmgr_klass->run = pr_manager_helper_run; ++ prmgr_klass->is_connected = pr_manager_helper_is_connected; + } + + static const TypeInfo pr_manager_helper_info = { +diff --git a/scsi/pr-manager-stub.c b/scsi/pr-manager-stub.c +index 632f17c..738b6d7 100644 +--- a/scsi/pr-manager-stub.c ++++ b/scsi/pr-manager-stub.c +@@ -22,3 +22,9 @@ PRManager *pr_manager_lookup(const char *id, Error **errp) + error_setg(errp, "No persistent reservation manager with id '%s'", id); + return NULL; + } ++ ++ ++PRManagerInfoList *qmp_query_pr_managers(Error **errp) ++{ ++ return NULL; ++} +diff --git a/scsi/pr-manager.c b/scsi/pr-manager.c +index 87c45db..2a8f300 100644 +--- a/scsi/pr-manager.c ++++ b/scsi/pr-manager.c +@@ -17,6 +17,10 @@ + #include "block/thread-pool.h" + #include "scsi/pr-manager.h" + #include "trace.h" ++#include "qapi/qapi-types-block.h" ++#include "qapi/qapi-commands-block.h" ++ ++#define PR_MANAGER_PATH "/objects" + + typedef struct PRManagerData { + PRManager *pr_mgr; +@@ -64,6 +68,14 @@ BlockAIOCB *pr_manager_execute(PRManager *pr_mgr, + data, complete, opaque); + } + ++bool pr_manager_is_connected(PRManager *pr_mgr) ++{ ++ PRManagerClass *pr_mgr_class = ++ PR_MANAGER_GET_CLASS(pr_mgr); ++ ++ return !pr_mgr_class->is_connected || pr_mgr_class->is_connected(pr_mgr); ++} ++ + static const TypeInfo pr_manager_info = { + .parent = TYPE_OBJECT, + .name = TYPE_PR_MANAGER, +@@ -105,5 +117,38 @@ pr_manager_register_types(void) + type_register_static(&pr_manager_info); + } + ++static int query_one_pr_manager(Object *object, void *opaque) ++{ ++ PRManagerInfoList ***prev = opaque; ++ PRManagerInfoList *elem; ++ PRManagerInfo *info; ++ PRManager *pr_mgr; ++ ++ pr_mgr = (PRManager *)object_dynamic_cast(object, TYPE_PR_MANAGER); ++ if (!pr_mgr) { ++ return 0; ++ } ++ ++ elem = g_new0(PRManagerInfoList, 1); ++ info = g_new0(PRManagerInfo, 1); ++ info->id = object_get_canonical_path_component(object); ++ info->connected = pr_manager_is_connected(pr_mgr); ++ elem->value = info; ++ elem->next = NULL; ++ ++ **prev = elem; ++ *prev = &elem->next; ++ return 0; ++} ++ ++PRManagerInfoList *qmp_query_pr_managers(Error **errp) ++{ ++ PRManagerInfoList *head = NULL; ++ PRManagerInfoList **prev = &head; ++ Object *container = container_get(object_get_root(), PR_MANAGER_PATH); ++ ++ object_child_foreach(container, query_one_pr_manager, &prev); ++ return head; ++} + + type_init(pr_manager_register_types); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-manager-helper-avoid-SIGSEGV-when-writing-to-the-.patch b/SOURCES/kvm-pr-manager-helper-avoid-SIGSEGV-when-writing-to-the-.patch new file mode 100644 index 0000000..1349866 --- /dev/null +++ b/SOURCES/kvm-pr-manager-helper-avoid-SIGSEGV-when-writing-to-the-.patch @@ -0,0 +1,47 @@ +From 3f937c82256585da1abba44c4600b96b0b186df4 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:53 +0200 +Subject: [PATCH 193/268] pr-manager-helper: avoid SIGSEGV when writing to the + socket fail + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-4-pbonzini@redhat.com> +Patchwork-id: 81245 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 3/9] pr-manager-helper: avoid SIGSEGV when writing to the socket fail +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +When writing to the qemu-pr-helper socket failed, the persistent +reservation manager was correctly disconnecting the socket, but it +did not clear pr_mgr->ioc. So the rest of the code did not know +that the socket had been disconnected, accessed pr_mgr->ioc and +happily caused a crash. + +To reproduce, it is enough to stop qemu-pr-helper between QEMU +startup and executing e.g. sg_persist -k /dev/sdb. + +Reviewed-by: Michal Privoznik +Signed-off-by: Paolo Bonzini +(cherry picked from commit aad10040d411d21542dc9ae58a2854c89ccedd78) +Signed-off-by: Miroslav Rezanina +--- + scsi/pr-manager-helper.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/scsi/pr-manager-helper.c b/scsi/pr-manager-helper.c +index 82ff6b6..0c0fe38 100644 +--- a/scsi/pr-manager-helper.c ++++ b/scsi/pr-manager-helper.c +@@ -71,6 +71,7 @@ static int pr_manager_helper_write(PRManagerHelper *pr_mgr, + if (n_written <= 0) { + assert(n_written != QIO_CHANNEL_ERR_BLOCK); + object_unref(OBJECT(pr_mgr->ioc)); ++ pr_mgr->ioc = NULL; + return n_written < 0 ? -EINVAL : 0; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-manager-helper-fix-memory-leak-on-event.patch b/SOURCES/kvm-pr-manager-helper-fix-memory-leak-on-event.patch new file mode 100644 index 0000000..0bd0999 --- /dev/null +++ b/SOURCES/kvm-pr-manager-helper-fix-memory-leak-on-event.patch @@ -0,0 +1,38 @@ +From 14dbdf962d1c5e79c0712ac1a344d82e87c7fdef Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:59 +0200 +Subject: [PATCH 199/268] pr-manager-helper: fix memory leak on event + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-10-pbonzini@redhat.com> +Patchwork-id: 81249 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 9/9] pr-manager-helper: fix memory leak on event +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +Reported by Coverity. + +Signed-off-by: Paolo Bonzini +(cherry picked from commit ea3d77c889cfa8c450da8a716c2bfd6aaea0adb2) +Signed-off-by: Miroslav Rezanina +--- + scsi/pr-manager-helper.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/scsi/pr-manager-helper.c b/scsi/pr-manager-helper.c +index 519a296..3027dde 100644 +--- a/scsi/pr-manager-helper.c ++++ b/scsi/pr-manager-helper.c +@@ -46,6 +46,7 @@ static void pr_manager_send_status_changed_event(PRManagerHelper *pr_mgr) + if (id) { + qapi_event_send_pr_manager_status_changed(id, !!pr_mgr->ioc, + &error_abort); ++ g_free(id); + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-manager-helper-report-event-on-connection-disconn.patch b/SOURCES/kvm-pr-manager-helper-report-event-on-connection-disconn.patch new file mode 100644 index 0000000..211fd32 --- /dev/null +++ b/SOURCES/kvm-pr-manager-helper-report-event-on-connection-disconn.patch @@ -0,0 +1,118 @@ +From 83a0b1b1828bf5e22da506c39848a3b226892c3d Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:56 +0200 +Subject: [PATCH 196/268] pr-manager-helper: report event on + connection/disconnection + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-7-pbonzini@redhat.com> +Patchwork-id: 81247 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 6/9] pr-manager-helper: report event on connection/disconnection +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +Let management know if there were any problems communicating with +qemu-pr-helper. The event is edge-triggered, and is sent every time +the connection status of the pr-manager-helper object changes. + +Signed-off-by: Paolo Bonzini +(cherry picked from commit e2c81a45101fdddfd47477a1805806f2c76639bf) +Signed-off-by: Miroslav Rezanina +--- + qapi/block.json | 24 ++++++++++++++++++++++++ + scsi/pr-manager-helper.c | 14 ++++++++++++++ + 2 files changed, 38 insertions(+) + +diff --git a/qapi/block.json b/qapi/block.json +index cf63ea2..528c1e6 100644 +--- a/qapi/block.json ++++ b/qapi/block.json +@@ -335,6 +335,30 @@ + 'data': { 'device': 'str', 'id': 'str', 'tray-open': 'bool' } } + + ## ++# @PR_MANAGER_STATUS_CHANGED: ++# ++# Emitted whenever the connected status of a persistent reservation ++# manager changes. ++# ++# @id: The id of the PR manager object ++# ++# @connected: true if the PR manager is connected to a backend ++# ++# Since: 3.0 ++# ++# Example: ++# ++# <- { "event": "PR_MANAGER_STATUS_CHANGED", ++# "data": { "id": "pr-helper0", ++# "connected": true ++# }, ++# "timestamp": { "seconds": 1519840375, "microseconds": 450486 } } ++# ++## ++{ 'event': 'PR_MANAGER_STATUS_CHANGED', ++ 'data': { 'id': 'str', 'connected': 'bool' } } ++ ++## + # @QuorumOpType: + # + # An enumeration of the quorum operation types +diff --git a/scsi/pr-manager-helper.c b/scsi/pr-manager-helper.c +index b11481b..519a296 100644 +--- a/scsi/pr-manager-helper.c ++++ b/scsi/pr-manager-helper.c +@@ -17,6 +17,7 @@ + #include "io/channel.h" + #include "io/channel-socket.h" + #include "pr-helper.h" ++#include "qapi/qapi-events-block.h" + + #include + +@@ -38,6 +39,16 @@ typedef struct PRManagerHelper { + QIOChannel *ioc; + } PRManagerHelper; + ++static void pr_manager_send_status_changed_event(PRManagerHelper *pr_mgr) ++{ ++ char *id = object_get_canonical_path_component(OBJECT(pr_mgr)); ++ ++ if (id) { ++ qapi_event_send_pr_manager_status_changed(id, !!pr_mgr->ioc, ++ &error_abort); ++ } ++} ++ + /* Called with lock held. */ + static int pr_manager_helper_read(PRManagerHelper *pr_mgr, + void *buf, int sz, Error **errp) +@@ -47,6 +58,7 @@ static int pr_manager_helper_read(PRManagerHelper *pr_mgr, + if (r < 0) { + object_unref(OBJECT(pr_mgr->ioc)); + pr_mgr->ioc = NULL; ++ pr_manager_send_status_changed_event(pr_mgr); + return -EINVAL; + } + +@@ -72,6 +84,7 @@ static int pr_manager_helper_write(PRManagerHelper *pr_mgr, + assert(n_written != QIO_CHANNEL_ERR_BLOCK); + object_unref(OBJECT(pr_mgr->ioc)); + pr_mgr->ioc = NULL; ++ pr_manager_send_status_changed_event(pr_mgr); + return n_written < 0 ? -EINVAL : 0; + } + +@@ -127,6 +140,7 @@ static int pr_manager_helper_initialize(PRManagerHelper *pr_mgr, + goto out_close; + } + ++ pr_manager_send_status_changed_event(pr_mgr); + return 0; + + out_close: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-pr-manager-put-stubs-in-.c-file.patch b/SOURCES/kvm-pr-manager-put-stubs-in-.c-file.patch new file mode 100644 index 0000000..0b7a2ae --- /dev/null +++ b/SOURCES/kvm-pr-manager-put-stubs-in-.c-file.patch @@ -0,0 +1,86 @@ +From e32685151297109c3b502ffdc7bda2288605029a Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Fri, 6 Jul 2018 17:56:54 +0200 +Subject: [PATCH 194/268] pr-manager: put stubs in .c file + +RH-Author: Paolo Bonzini +Message-id: <20180706175659.30615-5-pbonzini@redhat.com> +Patchwork-id: 81253 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH 4/9] pr-manager: put stubs in .c file +Bugzilla: 1533158 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Michal Privoznik + +Signed-off-by: Paolo Bonzini +(cherry picked from commit 58b3017f7fba15e8c440115dfd5d380f490d0b61) +Signed-off-by: Miroslav Rezanina +--- + include/scsi/pr-manager.h | 9 --------- + scsi/Makefile.objs | 1 + + scsi/pr-manager-stub.c | 24 ++++++++++++++++++++++++ + 3 files changed, 25 insertions(+), 9 deletions(-) + create mode 100644 scsi/pr-manager-stub.c + +diff --git a/include/scsi/pr-manager.h b/include/scsi/pr-manager.h +index 5d2f13a..71971ae 100644 +--- a/include/scsi/pr-manager.h ++++ b/include/scsi/pr-manager.h +@@ -41,15 +41,6 @@ BlockAIOCB *pr_manager_execute(PRManager *pr_mgr, + BlockCompletionFunc *complete, + void *opaque); + +-#ifdef CONFIG_LINUX + PRManager *pr_manager_lookup(const char *id, Error **errp); +-#else +-static inline PRManager *pr_manager_lookup(const char *id, Error **errp) +-{ +- /* The classes do not exist at all! */ +- error_setg(errp, "No persistent reservation manager with id '%s'", id); +- return NULL; +-} +-#endif + + #endif +diff --git a/scsi/Makefile.objs b/scsi/Makefile.objs +index 4d25e47..bb8789c 100644 +--- a/scsi/Makefile.objs ++++ b/scsi/Makefile.objs +@@ -1,3 +1,4 @@ + block-obj-y += utils.o + + block-obj-$(CONFIG_LINUX) += pr-manager.o pr-manager-helper.o ++block-obj-$(call lnot,$(CONFIG_LINUX)) += pr-manager-stub.o +diff --git a/scsi/pr-manager-stub.c b/scsi/pr-manager-stub.c +new file mode 100644 +index 0000000..632f17c +--- /dev/null ++++ b/scsi/pr-manager-stub.c +@@ -0,0 +1,24 @@ ++/* ++ * Persistent reservation manager - stub for non-Linux platforms ++ * ++ * Copyright (c) 2018 Red Hat, Inc. ++ * ++ * Author: Paolo Bonzini ++ * ++ * This code is licensed under the LGPL. ++ * ++ */ ++ ++#include "qemu/osdep.h" ++#include "qapi/error.h" ++#include "scsi/pr-manager.h" ++#include "trace.h" ++#include "qapi/qapi-types-block.h" ++#include "qapi/qapi-commands-block.h" ++ ++PRManager *pr_manager_lookup(const char *id, Error **errp) ++{ ++ /* The classes do not exist at all! */ ++ error_setg(errp, "No persistent reservation manager with id '%s'", id); ++ return NULL; ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-python-futurize-f-lib2to3.fixes.fix_except.patch b/SOURCES/kvm-python-futurize-f-lib2to3.fixes.fix_except.patch new file mode 100644 index 0000000..7d1b950 --- /dev/null +++ b/SOURCES/kvm-python-futurize-f-lib2to3.fixes.fix_except.patch @@ -0,0 +1,51 @@ +From af313514f87fa036f0813f0f4dafdfdd8f259a54 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Mon, 25 Jun 2018 21:23:44 +0100 +Subject: [PATCH 03/15] python: futurize -f lib2to3.fixes.fix_except + +RH-Author: Eduardo Habkost +Message-id: <20180625212346.26440-3-ehabkost@redhat.com> +Patchwork-id: 81050 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 2/4] python: futurize -f lib2to3.fixes.fix_except +Bugzilla: 1571533 +RH-Acked-by: Thomas Huth +RH-Acked-by: Vitaly Kuznetsov +RH-Acked-by: Miroslav Rezanina + +Convert "except X, T" to "except X as T". + +This is necessary for Python 3 compatibility. + +Done using: + + $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ + sort -u | grep -v README.sh4) + $ futurize -w -f lib2to3.fixes.fix_except $py + +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Eduardo Habkost +Message-Id: <20180608122952.2009-10-ehabkost@redhat.com> +Signed-off-by: Eduardo Habkost +(cherry picked from commit bd228083f7364dd2903970ec4dc0cc55bf156104) +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + scripts/simpletrace.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py +index 54e761f..6b46195 100755 +--- a/scripts/simpletrace.py ++++ b/scripts/simpletrace.py +@@ -45,7 +45,7 @@ def get_record(edict, idtoname, rechdr, fobj): + rec = (name, rechdr[1], rechdr[3]) + try: + event = edict[name] +- except KeyError, e: ++ except KeyError as e: + import sys + sys.stderr.write('%s event is logged but is not declared ' \ + 'in the trace events file, try using ' \ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-python-futurize-f-libfuturize.fixes.fix_print_with_i.patch b/SOURCES/kvm-python-futurize-f-libfuturize.fixes.fix_print_with_i.patch new file mode 100644 index 0000000..5a6f755 --- /dev/null +++ b/SOURCES/kvm-python-futurize-f-libfuturize.fixes.fix_print_with_i.patch @@ -0,0 +1,1809 @@ +From 141ae90648a8bb4c26238cfde4d4946e6cff3108 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Mon, 25 Jun 2018 21:23:43 +0100 +Subject: [PATCH 02/15] python: futurize -f + libfuturize.fixes.fix_print_with_import + +RH-Author: Eduardo Habkost +Message-id: <20180625212346.26440-2-ehabkost@redhat.com> +Patchwork-id: 81049 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 1/4] python: futurize -f libfuturize.fixes.fix_print_with_import +Bugzilla: 1571533 +RH-Acked-by: Thomas Huth +RH-Acked-by: Vitaly Kuznetsov +RH-Acked-by: Miroslav Rezanina + +Change all Python code to use print as a function. + +This is necessary for Python 3 compatibility. + +Done using: + + $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ + sort -u | grep -v README.sh4) + $ futurize -w -f libfuturize.fixes.fix_print_with_import $py + +Backport conflicts: + * tests/docker/docker.py: class ProbeCommand doesn't exist + in the RHEL8 tree + +Reviewed-by: Stefan Hajnoczi +Acked-by: Fam Zheng +Signed-off-by: Eduardo Habkost +Message-Id: <20180608122952.2009-2-ehabkost@redhat.com> +[ehabkost: fixup tests/docker/docker.py] +Signed-off-by: Eduardo Habkost +(cherry picked from commit f03868bd5653265e97b253102d77d83ea85efdea) +Signed-off-by: Eduardo Habkost + +Signed-off-by: Danilo C. L. de Paula +--- + scripts/analyse-9p-simpletrace.py | 89 ++++++++++++++++---------------- + scripts/analyse-locks-simpletrace.py | 1 + + scripts/analyze-migration.py | 11 ++-- + scripts/device-crash-test | 3 +- + scripts/dump-guest-memory.py | 1 + + scripts/kvm/kvm_flightrecorder | 21 ++++---- + scripts/kvm/vmxcap | 1 + + scripts/qmp/qemu-ga-client | 1 + + scripts/qmp/qmp | 17 +++--- + scripts/qmp/qmp-shell | 35 +++++++------ + scripts/qmp/qom-get | 7 +-- + scripts/qmp/qom-list | 11 ++-- + scripts/qmp/qom-set | 5 +- + scripts/qmp/qom-tree | 11 ++-- + scripts/replay-dump.py | 21 ++++---- + scripts/signrom.py | 1 + + scripts/simpletrace.py | 3 +- + scripts/vmstate-static-checker.py | 85 +++++++++++++++--------------- + tests/docker/docker.py | 11 ++-- + tests/docker/travis.py | 15 +++--- + tests/guest-debug/test-gdbstub.py | 1 + + tests/image-fuzzer/runner.py | 38 ++++++-------- + tests/migration/guestperf/engine.py | 29 ++++++----- + tests/migration/guestperf/plot.py | 17 +++--- + tests/migration/guestperf/shell.py | 19 +++---- + tests/qemu-iotests/149 | 3 +- + tests/qemu-iotests/165 | 3 +- + tests/qemu-iotests/iotests.py | 5 +- + tests/qemu-iotests/nbd-fault-injector.py | 7 +-- + tests/qemu-iotests/qcow2.py | 39 +++++++------- + tests/qemu-iotests/qed.py | 17 +++--- + tests/vm/basevm.py | 3 +- + 32 files changed, 278 insertions(+), 253 deletions(-) + +diff --git a/scripts/analyse-9p-simpletrace.py b/scripts/analyse-9p-simpletrace.py +index 3c3dee4..710e01a 100755 +--- a/scripts/analyse-9p-simpletrace.py ++++ b/scripts/analyse-9p-simpletrace.py +@@ -3,6 +3,7 @@ + # Usage: ./analyse-9p-simpletrace + # + # Author: Harsh Prateek Bora ++from __future__ import print_function + import os + import simpletrace + +@@ -79,135 +80,135 @@ symbol_9p = { + + class VirtFSRequestTracker(simpletrace.Analyzer): + def begin(self): +- print "Pretty printing 9p simpletrace log ..." ++ print("Pretty printing 9p simpletrace log ...") + + def v9fs_rerror(self, tag, id, err): +- print "RERROR (tag =", tag, ", id =", symbol_9p[id], ", err = \"", os.strerror(err), "\")" ++ print("RERROR (tag =", tag, ", id =", symbol_9p[id], ", err = \"", os.strerror(err), "\")") + + def v9fs_version(self, tag, id, msize, version): +- print "TVERSION (tag =", tag, ", msize =", msize, ", version =", version, ")" ++ print("TVERSION (tag =", tag, ", msize =", msize, ", version =", version, ")") + + def v9fs_version_return(self, tag, id, msize, version): +- print "RVERSION (tag =", tag, ", msize =", msize, ", version =", version, ")" ++ print("RVERSION (tag =", tag, ", msize =", msize, ", version =", version, ")") + + def v9fs_attach(self, tag, id, fid, afid, uname, aname): +- print "TATTACH (tag =", tag, ", fid =", fid, ", afid =", afid, ", uname =", uname, ", aname =", aname, ")" ++ print("TATTACH (tag =", tag, ", fid =", fid, ", afid =", afid, ", uname =", uname, ", aname =", aname, ")") + + def v9fs_attach_return(self, tag, id, type, version, path): +- print "RATTACH (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "})" ++ print("RATTACH (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "})") + + def v9fs_stat(self, tag, id, fid): +- print "TSTAT (tag =", tag, ", fid =", fid, ")" ++ print("TSTAT (tag =", tag, ", fid =", fid, ")") + + def v9fs_stat_return(self, tag, id, mode, atime, mtime, length): +- print "RSTAT (tag =", tag, ", mode =", mode, ", atime =", atime, ", mtime =", mtime, ", length =", length, ")" ++ print("RSTAT (tag =", tag, ", mode =", mode, ", atime =", atime, ", mtime =", mtime, ", length =", length, ")") + + def v9fs_getattr(self, tag, id, fid, request_mask): +- print "TGETATTR (tag =", tag, ", fid =", fid, ", request_mask =", hex(request_mask), ")" ++ print("TGETATTR (tag =", tag, ", fid =", fid, ", request_mask =", hex(request_mask), ")") + + def v9fs_getattr_return(self, tag, id, result_mask, mode, uid, gid): +- print "RGETATTR (tag =", tag, ", result_mask =", hex(result_mask), ", mode =", oct(mode), ", uid =", uid, ", gid =", gid, ")" ++ print("RGETATTR (tag =", tag, ", result_mask =", hex(result_mask), ", mode =", oct(mode), ", uid =", uid, ", gid =", gid, ")") + + def v9fs_walk(self, tag, id, fid, newfid, nwnames): +- print "TWALK (tag =", tag, ", fid =", fid, ", newfid =", newfid, ", nwnames =", nwnames, ")" ++ print("TWALK (tag =", tag, ", fid =", fid, ", newfid =", newfid, ", nwnames =", nwnames, ")") + + def v9fs_walk_return(self, tag, id, nwnames, qids): +- print "RWALK (tag =", tag, ", nwnames =", nwnames, ", qids =", hex(qids), ")" ++ print("RWALK (tag =", tag, ", nwnames =", nwnames, ", qids =", hex(qids), ")") + + def v9fs_open(self, tag, id, fid, mode): +- print "TOPEN (tag =", tag, ", fid =", fid, ", mode =", oct(mode), ")" ++ print("TOPEN (tag =", tag, ", fid =", fid, ", mode =", oct(mode), ")") + + def v9fs_open_return(self, tag, id, type, version, path, iounit): +- print "ROPEN (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, iounit =", iounit, ")" ++ print("ROPEN (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, iounit =", iounit, ")") + + def v9fs_lcreate(self, tag, id, dfid, flags, mode, gid): +- print "TLCREATE (tag =", tag, ", dfid =", dfid, ", flags =", oct(flags), ", mode =", oct(mode), ", gid =", gid, ")" ++ print("TLCREATE (tag =", tag, ", dfid =", dfid, ", flags =", oct(flags), ", mode =", oct(mode), ", gid =", gid, ")") + + def v9fs_lcreate_return(self, tag, id, type, version, path, iounit): +- print "RLCREATE (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, iounit =", iounit, ")" ++ print("RLCREATE (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, iounit =", iounit, ")") + + def v9fs_fsync(self, tag, id, fid, datasync): +- print "TFSYNC (tag =", tag, ", fid =", fid, ", datasync =", datasync, ")" ++ print("TFSYNC (tag =", tag, ", fid =", fid, ", datasync =", datasync, ")") + + def v9fs_clunk(self, tag, id, fid): +- print "TCLUNK (tag =", tag, ", fid =", fid, ")" ++ print("TCLUNK (tag =", tag, ", fid =", fid, ")") + + def v9fs_read(self, tag, id, fid, off, max_count): +- print "TREAD (tag =", tag, ", fid =", fid, ", off =", off, ", max_count =", max_count, ")" ++ print("TREAD (tag =", tag, ", fid =", fid, ", off =", off, ", max_count =", max_count, ")") + + def v9fs_read_return(self, tag, id, count, err): +- print "RREAD (tag =", tag, ", count =", count, ", err =", err, ")" ++ print("RREAD (tag =", tag, ", count =", count, ", err =", err, ")") + + def v9fs_readdir(self, tag, id, fid, offset, max_count): +- print "TREADDIR (tag =", tag, ", fid =", fid, ", offset =", offset, ", max_count =", max_count, ")" ++ print("TREADDIR (tag =", tag, ", fid =", fid, ", offset =", offset, ", max_count =", max_count, ")") + + def v9fs_readdir_return(self, tag, id, count, retval): +- print "RREADDIR (tag =", tag, ", count =", count, ", retval =", retval, ")" ++ print("RREADDIR (tag =", tag, ", count =", count, ", retval =", retval, ")") + + def v9fs_write(self, tag, id, fid, off, count, cnt): +- print "TWRITE (tag =", tag, ", fid =", fid, ", off =", off, ", count =", count, ", cnt =", cnt, ")" ++ print("TWRITE (tag =", tag, ", fid =", fid, ", off =", off, ", count =", count, ", cnt =", cnt, ")") + + def v9fs_write_return(self, tag, id, total, err): +- print "RWRITE (tag =", tag, ", total =", total, ", err =", err, ")" ++ print("RWRITE (tag =", tag, ", total =", total, ", err =", err, ")") + + def v9fs_create(self, tag, id, fid, name, perm, mode): +- print "TCREATE (tag =", tag, ", fid =", fid, ", perm =", oct(perm), ", name =", name, ", mode =", oct(mode), ")" ++ print("TCREATE (tag =", tag, ", fid =", fid, ", perm =", oct(perm), ", name =", name, ", mode =", oct(mode), ")") + + def v9fs_create_return(self, tag, id, type, version, path, iounit): +- print "RCREATE (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, iounit =", iounit, ")" ++ print("RCREATE (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, iounit =", iounit, ")") + + def v9fs_symlink(self, tag, id, fid, name, symname, gid): +- print "TSYMLINK (tag =", tag, ", fid =", fid, ", name =", name, ", symname =", symname, ", gid =", gid, ")" ++ print("TSYMLINK (tag =", tag, ", fid =", fid, ", name =", name, ", symname =", symname, ", gid =", gid, ")") + + def v9fs_symlink_return(self, tag, id, type, version, path): +- print "RSYMLINK (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "})" ++ print("RSYMLINK (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "})") + + def v9fs_flush(self, tag, id, flush_tag): +- print "TFLUSH (tag =", tag, ", flush_tag =", flush_tag, ")" ++ print("TFLUSH (tag =", tag, ", flush_tag =", flush_tag, ")") + + def v9fs_link(self, tag, id, dfid, oldfid, name): +- print "TLINK (tag =", tag, ", dfid =", dfid, ", oldfid =", oldfid, ", name =", name, ")" ++ print("TLINK (tag =", tag, ", dfid =", dfid, ", oldfid =", oldfid, ", name =", name, ")") + + def v9fs_remove(self, tag, id, fid): +- print "TREMOVE (tag =", tag, ", fid =", fid, ")" ++ print("TREMOVE (tag =", tag, ", fid =", fid, ")") + + def v9fs_wstat(self, tag, id, fid, mode, atime, mtime): +- print "TWSTAT (tag =", tag, ", fid =", fid, ", mode =", oct(mode), ", atime =", atime, "mtime =", mtime, ")" ++ print("TWSTAT (tag =", tag, ", fid =", fid, ", mode =", oct(mode), ", atime =", atime, "mtime =", mtime, ")") + + def v9fs_mknod(self, tag, id, fid, mode, major, minor): +- print "TMKNOD (tag =", tag, ", fid =", fid, ", mode =", oct(mode), ", major =", major, ", minor =", minor, ")" ++ print("TMKNOD (tag =", tag, ", fid =", fid, ", mode =", oct(mode), ", major =", major, ", minor =", minor, ")") + + def v9fs_lock(self, tag, id, fid, type, start, length): +- print "TLOCK (tag =", tag, ", fid =", fid, "type =", type, ", start =", start, ", length =", length, ")" ++ print("TLOCK (tag =", tag, ", fid =", fid, "type =", type, ", start =", start, ", length =", length, ")") + + def v9fs_lock_return(self, tag, id, status): +- print "RLOCK (tag =", tag, ", status =", status, ")" ++ print("RLOCK (tag =", tag, ", status =", status, ")") + + def v9fs_getlock(self, tag, id, fid, type, start, length): +- print "TGETLOCK (tag =", tag, ", fid =", fid, "type =", type, ", start =", start, ", length =", length, ")" ++ print("TGETLOCK (tag =", tag, ", fid =", fid, "type =", type, ", start =", start, ", length =", length, ")") + + def v9fs_getlock_return(self, tag, id, type, start, length, proc_id): +- print "RGETLOCK (tag =", tag, "type =", type, ", start =", start, ", length =", length, ", proc_id =", proc_id, ")" ++ print("RGETLOCK (tag =", tag, "type =", type, ", start =", start, ", length =", length, ", proc_id =", proc_id, ")") + + def v9fs_mkdir(self, tag, id, fid, name, mode, gid): +- print "TMKDIR (tag =", tag, ", fid =", fid, ", name =", name, ", mode =", mode, ", gid =", gid, ")" ++ print("TMKDIR (tag =", tag, ", fid =", fid, ", name =", name, ", mode =", mode, ", gid =", gid, ")") + + def v9fs_mkdir_return(self, tag, id, type, version, path, err): +- print "RMKDIR (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, err =", err, ")" ++ print("RMKDIR (tag =", tag, ", qid={type =", type, ", version =", version, ", path =", path, "}, err =", err, ")") + + def v9fs_xattrwalk(self, tag, id, fid, newfid, name): +- print "TXATTRWALK (tag =", tag, ", fid =", fid, ", newfid =", newfid, ", xattr name =", name, ")" ++ print("TXATTRWALK (tag =", tag, ", fid =", fid, ", newfid =", newfid, ", xattr name =", name, ")") + + def v9fs_xattrwalk_return(self, tag, id, size): +- print "RXATTRWALK (tag =", tag, ", xattrsize =", size, ")" ++ print("RXATTRWALK (tag =", tag, ", xattrsize =", size, ")") + + def v9fs_xattrcreate(self, tag, id, fid, name, size, flags): +- print "TXATTRCREATE (tag =", tag, ", fid =", fid, ", name =", name, ", xattrsize =", size, ", flags =", flags, ")" ++ print("TXATTRCREATE (tag =", tag, ", fid =", fid, ", name =", name, ", xattrsize =", size, ", flags =", flags, ")") + + def v9fs_readlink(self, tag, id, fid): +- print "TREADLINK (tag =", tag, ", fid =", fid, ")" ++ print("TREADLINK (tag =", tag, ", fid =", fid, ")") + + def v9fs_readlink_return(self, tag, id, target): +- print "RREADLINK (tag =", tag, ", target =", target, ")" ++ print("RREADLINK (tag =", tag, ", target =", target, ")") + + simpletrace.run(VirtFSRequestTracker()) +diff --git a/scripts/analyse-locks-simpletrace.py b/scripts/analyse-locks-simpletrace.py +index 101e84d..352bc9c 100755 +--- a/scripts/analyse-locks-simpletrace.py ++++ b/scripts/analyse-locks-simpletrace.py +@@ -6,6 +6,7 @@ + # Author: Alex Bennée + # + ++from __future__ import print_function + import os + import simpletrace + import argparse +diff --git a/scripts/analyze-migration.py b/scripts/analyze-migration.py +index 88ff4ad..5c2010c 100755 +--- a/scripts/analyze-migration.py ++++ b/scripts/analyze-migration.py +@@ -17,6 +17,7 @@ + # You should have received a copy of the GNU Lesser General Public + # License along with this library; if not, see . + ++from __future__ import print_function + import numpy as np + import json + import os +@@ -162,7 +163,7 @@ class RamSection(object): + len = self.file.read64() + self.sizeinfo[self.name] = '0x%016x' % len + if self.write_memory: +- print self.name ++ print(self.name) + mkdir_p('./' + os.path.dirname(self.name)) + f = open('./' + self.name, "wb") + f.truncate(0) +@@ -588,7 +589,7 @@ if args.extract: + dump = MigrationDump(args.file) + + dump.read(desc_only = True) +- print "desc.json" ++ print("desc.json") + f = open("desc.json", "wb") + f.truncate() + f.write(jsonenc.encode(dump.vmsd_desc)) +@@ -596,7 +597,7 @@ if args.extract: + + dump.read(write_memory = True) + dict = dump.getDict() +- print "state.json" ++ print("state.json") + f = open("state.json", "wb") + f.truncate() + f.write(jsonenc.encode(dict)) +@@ -605,10 +606,10 @@ elif args.dump == "state": + dump = MigrationDump(args.file) + dump.read(dump_memory = args.memory) + dict = dump.getDict() +- print jsonenc.encode(dict) ++ print(jsonenc.encode(dict)) + elif args.dump == "desc": + dump = MigrationDump(args.file) + dump.read(desc_only = True) +- print jsonenc.encode(dump.vmsd_desc) ++ print(jsonenc.encode(dump.vmsd_desc)) + else: + raise Exception("Please specify either -x, -d state or -d dump") +diff --git a/scripts/device-crash-test b/scripts/device-crash-test +index 5d17dc6..6a85c15 100755 +--- a/scripts/device-crash-test ++++ b/scripts/device-crash-test +@@ -23,6 +23,7 @@ + Run QEMU with all combinations of -machine and -device types, + check for crashes and unexpected errors. + """ ++from __future__ import print_function + + import sys + import os +@@ -557,7 +558,7 @@ def main(): + tc[k] = v + + if len(binariesToTest(args, tc)) == 0: +- print >>sys.stderr, "No QEMU binary found" ++ print("No QEMU binary found", file=sys.stderr) + parser.print_usage(sys.stderr) + return 1 + +diff --git a/scripts/dump-guest-memory.py b/scripts/dump-guest-memory.py +index 276eebf..5a857ce 100644 +--- a/scripts/dump-guest-memory.py ++++ b/scripts/dump-guest-memory.py +@@ -12,6 +12,7 @@ Authors: + This work is licensed under the terms of the GNU GPL, version 2 or later. See + the COPYING file in the top-level directory. + """ ++from __future__ import print_function + + import ctypes + import struct +diff --git a/scripts/kvm/kvm_flightrecorder b/scripts/kvm/kvm_flightrecorder +index 7fb1c2d..54a5674 100755 +--- a/scripts/kvm/kvm_flightrecorder ++++ b/scripts/kvm/kvm_flightrecorder +@@ -32,6 +32,7 @@ + # consuming CPU cycles. No disk I/O is performed since the ring buffer holds a + # fixed-size in-memory trace. + ++from __future__ import print_function + import sys + import os + +@@ -77,8 +78,8 @@ def tail_trace(): + pass + + def usage(): +- print 'Usage: %s start [buffer_size_kb] | stop | dump | tail' % sys.argv[0] +- print 'Control the KVM flight recorder tracing.' ++ print('Usage: %s start [buffer_size_kb] | stop | dump | tail' % sys.argv[0]) ++ print('Control the KVM flight recorder tracing.') + sys.exit(0) + + def main(): +@@ -87,15 +88,15 @@ def main(): + + cmd = sys.argv[1] + if cmd == '--version': +- print 'kvm_flightrecorder version 1.0' ++ print('kvm_flightrecorder version 1.0') + sys.exit(0) + + if not os.path.isdir(tracing_dir): +- print 'Unable to tracing debugfs directory, try:' +- print 'mount -t debugfs none /sys/kernel/debug' ++ print('Unable to tracing debugfs directory, try:') ++ print('mount -t debugfs none /sys/kernel/debug') + sys.exit(1) + if not os.access(tracing_dir, os.W_OK): +- print 'Unable to write to tracing debugfs directory, please run as root' ++ print('Unable to write to tracing debugfs directory, please run as root') + sys.exit(1) + + if cmd == 'start': +@@ -105,16 +106,16 @@ def main(): + try: + buffer_size_kb = int(sys.argv[2]) + except ValueError: +- print 'Invalid per-cpu trace buffer size in KB' ++ print('Invalid per-cpu trace buffer size in KB') + sys.exit(1) + write_file(trace_path('buffer_size_kb'), str(buffer_size_kb)) +- print 'Per-CPU ring buffer size set to %d KB' % buffer_size_kb ++ print('Per-CPU ring buffer size set to %d KB' % buffer_size_kb) + + start_tracing() +- print 'KVM flight recorder enabled' ++ print('KVM flight recorder enabled') + elif cmd == 'stop': + stop_tracing() +- print 'KVM flight recorder disabled' ++ print('KVM flight recorder disabled') + elif cmd == 'dump': + dump_trace() + elif cmd == 'tail': +diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap +index d9a6db0..99a8146 100755 +--- a/scripts/kvm/vmxcap ++++ b/scripts/kvm/vmxcap +@@ -10,6 +10,7 @@ + # This work is licensed under the terms of the GNU GPL, version 2. See + # the COPYING file in the top-level directory. + ++from __future__ import print_function + MSR_IA32_VMX_BASIC = 0x480 + MSR_IA32_VMX_PINBASED_CTLS = 0x481 + MSR_IA32_VMX_PROCBASED_CTLS = 0x482 +diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client +index 7d2a472..8510814 100755 +--- a/scripts/qmp/qemu-ga-client ++++ b/scripts/qmp/qemu-ga-client +@@ -36,6 +36,7 @@ + # See also: https://wiki.qemu.org/Features/QAPI/GuestAgent + # + ++from __future__ import print_function + import base64 + import random + +diff --git a/scripts/qmp/qmp b/scripts/qmp/qmp +index 514b539..16d3bdb 100755 +--- a/scripts/qmp/qmp ++++ b/scripts/qmp/qmp +@@ -10,6 +10,7 @@ + # This work is licensed under the terms of the GNU GPLv2 or later. + # See the COPYING file in the top-level directory. + ++from __future__ import print_function + import sys, os + from qmp import QEMUMonitorProtocol + +@@ -26,9 +27,9 @@ def print_response(rsp, prefix=[]): + print_response(rsp[key], prefix + [key]) + else: + if len(prefix): +- print '%s: %s' % ('.'.join(prefix), rsp) ++ print('%s: %s' % ('.'.join(prefix), rsp)) + else: +- print '%s' % (rsp) ++ print('%s' % (rsp)) + + def main(args): + path = None +@@ -53,21 +54,21 @@ def main(args): + elif arg in ['help']: + os.execlp('man', 'man', 'qmp') + else: +- print 'Unknown argument "%s"' % arg ++ print('Unknown argument "%s"' % arg) + + args = args[1:] + else: + break + + if not path: +- print "QMP path isn't set, use --path=qmp-monitor-address or set QMP_PATH" ++ print("QMP path isn't set, use --path=qmp-monitor-address or set QMP_PATH") + return 1 + + if len(args): + command, args = args[0], args[1:] + else: +- print 'No command found' +- print 'Usage: "qmp [--path=qmp-monitor-address] qmp-cmd arguments"' ++ print('No command found') ++ print('Usage: "qmp [--path=qmp-monitor-address] qmp-cmd arguments"') + return 1 + + if command in ['help']: +@@ -93,7 +94,7 @@ def main(args): + os.execvp(fullcmd, [fullcmd] + args) + except OSError as exc: + if exc.errno == 2: +- print 'Command "%s" not found.' % (fullcmd) ++ print('Command "%s" not found.' % (fullcmd)) + return 1 + raise + return 0 +@@ -104,7 +105,7 @@ def main(args): + arguments = {} + for arg in args: + if not arg.startswith('--'): +- print 'Unknown argument "%s"' % arg ++ print('Unknown argument "%s"' % arg) + return 1 + + arg = arg[2:] +diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell +index be449de..b1cc7e2 100755 +--- a/scripts/qmp/qmp-shell ++++ b/scripts/qmp/qmp-shell +@@ -65,6 +65,7 @@ + # which will echo back the properly formatted JSON-compliant QMP that is being + # sent to QEMU, which is useful for debugging and documentation generation. + ++from __future__ import print_function + import qmp + import json + import ast +@@ -153,14 +154,14 @@ class QMPShell(qmp.QEMUMonitorProtocol): + # File not found. No problem. + pass + else: +- print "Failed to read history '%s'; %s" % (self._histfile, e) ++ print("Failed to read history '%s'; %s" % (self._histfile, e)) + atexit.register(self.__save_history) + + def __save_history(self): + try: + readline.write_history_file(self._histfile) + except Exception as e: +- print "Failed to save history file '%s'; %s" % (self._histfile, e) ++ print("Failed to save history file '%s'; %s" % (self._histfile, e)) + + def __parse_value(self, val): + try: +@@ -258,15 +259,15 @@ class QMPShell(qmp.QEMUMonitorProtocol): + if self._pretty: + indent = 4 + jsobj = json.dumps(qmp, indent=indent) +- print str(jsobj) ++ print(str(jsobj)) + + def _execute_cmd(self, cmdline): + try: + qmpcmd = self.__build_cmd(cmdline) + except Exception as e: +- print 'Error while parsing command line: %s' % e +- print 'command format: ', +- print '[arg-name1=arg1] ... [arg-nameN=argN]' ++ print('Error while parsing command line: %s' % e) ++ print('command format: ', end=' ') ++ print('[arg-name1=arg1] ... [arg-nameN=argN]') + return True + # For transaction mode, we may have just cached the action: + if qmpcmd is None: +@@ -275,7 +276,7 @@ class QMPShell(qmp.QEMUMonitorProtocol): + self._print(qmpcmd) + resp = self.cmd_obj(qmpcmd) + if resp is None: +- print 'Disconnected' ++ print('Disconnected') + return False + self._print(resp) + return True +@@ -285,12 +286,12 @@ class QMPShell(qmp.QEMUMonitorProtocol): + self.__completer_setup() + + def show_banner(self, msg='Welcome to the QMP low-level shell!'): +- print msg ++ print(msg) + if not self._greeting: +- print 'Connected' ++ print('Connected') + return + version = self._greeting['QMP']['version']['qemu'] +- print 'Connected to QEMU %d.%d.%d\n' % (version['major'],version['minor'],version['micro']) ++ print('Connected to QEMU %d.%d.%d\n' % (version['major'],version['minor'],version['micro'])) + + def get_prompt(self): + if self._transmode: +@@ -306,11 +307,11 @@ class QMPShell(qmp.QEMUMonitorProtocol): + try: + cmdline = raw_input(prompt) + except EOFError: +- print ++ print() + return False + if cmdline == '': + for ev in self.get_events(): +- print ev ++ print(ev) + self.clear_events() + return True + else: +@@ -366,24 +367,24 @@ class HMPShell(QMPShell): + try: + idx = int(cmdline.split()[1]) + if not 'return' in self.__cmd_passthrough('info version', idx): +- print 'bad CPU index' ++ print('bad CPU index') + return True + self.__cpu_index = idx + except ValueError: +- print 'cpu command takes an integer argument' ++ print('cpu command takes an integer argument') + return True + resp = self.__cmd_passthrough(cmdline, self.__cpu_index) + if resp is None: +- print 'Disconnected' ++ print('Disconnected') + return False + assert 'return' in resp or 'error' in resp + if 'return' in resp: + # Success + if len(resp['return']) > 0: +- print resp['return'], ++ print(resp['return'], end=' ') + else: + # Error +- print '%s: %s' % (resp['error']['class'], resp['error']['desc']) ++ print('%s: %s' % (resp['error']['class'], resp['error']['desc'])) + return True + + def show_banner(self): +diff --git a/scripts/qmp/qom-get b/scripts/qmp/qom-get +index 0172c69..291c8bf 100755 +--- a/scripts/qmp/qom-get ++++ b/scripts/qmp/qom-get +@@ -11,6 +11,7 @@ + # the COPYING file in the top-level directory. + ## + ++from __future__ import print_function + import sys + import os + from qmp import QEMUMonitorProtocol +@@ -33,7 +34,7 @@ def usage_error(error_msg = "unspecified error"): + + if len(args) > 0: + if args[0] == "-h": +- print usage() ++ print(usage()) + exit(0); + elif args[0] == "-s": + try: +@@ -62,6 +63,6 @@ srv.connect() + rsp = srv.command('qom-get', path=path, property=prop) + if type(rsp) == dict: + for i in rsp.keys(): +- print '%s: %s' % (i, rsp[i]) ++ print('%s: %s' % (i, rsp[i])) + else: +- print rsp ++ print(rsp) +diff --git a/scripts/qmp/qom-list b/scripts/qmp/qom-list +index 1e7cc6c..cd907bb 100755 +--- a/scripts/qmp/qom-list ++++ b/scripts/qmp/qom-list +@@ -11,6 +11,7 @@ + # the COPYING file in the top-level directory. + ## + ++from __future__ import print_function + import sys + import os + from qmp import QEMUMonitorProtocol +@@ -33,7 +34,7 @@ def usage_error(error_msg = "unspecified error"): + + if len(args) > 0: + if args[0] == "-h": +- print usage() ++ print(usage()) + exit(0); + elif args[0] == "-s": + try: +@@ -52,13 +53,13 @@ srv = QEMUMonitorProtocol(socket_path) + srv.connect() + + if len(args) == 0: +- print '/' ++ print('/') + sys.exit(0) + + for item in srv.command('qom-list', path=args[0]): + if item['type'].startswith('child<'): +- print '%s/' % item['name'] ++ print('%s/' % item['name']) + elif item['type'].startswith('link<'): +- print '@%s/' % item['name'] ++ print('@%s/' % item['name']) + else: +- print '%s' % item['name'] ++ print('%s' % item['name']) +diff --git a/scripts/qmp/qom-set b/scripts/qmp/qom-set +index 94e2778..fbe4b3e 100755 +--- a/scripts/qmp/qom-set ++++ b/scripts/qmp/qom-set +@@ -11,6 +11,7 @@ + # the COPYING file in the top-level directory. + ## + ++from __future__ import print_function + import sys + import os + from qmp import QEMUMonitorProtocol +@@ -34,7 +35,7 @@ def usage_error(error_msg = "unspecified error"): + + if len(args) > 0: + if args[0] == "-h": +- print usage() ++ print(usage()) + exit(0); + elif args[0] == "-s": + try: +@@ -61,4 +62,4 @@ else: + srv = QEMUMonitorProtocol(socket_path) + srv.connect() + +-print srv.command('qom-set', path=path, property=prop, value=value) ++print(srv.command('qom-set', path=path, property=prop, value=value)) +diff --git a/scripts/qmp/qom-tree b/scripts/qmp/qom-tree +index 906fcd2..0ffd1ff 100755 +--- a/scripts/qmp/qom-tree ++++ b/scripts/qmp/qom-tree +@@ -13,6 +13,7 @@ + # the COPYING file in the top-level directory. + ## + ++from __future__ import print_function + import sys + import os + from qmp import QEMUMonitorProtocol +@@ -35,7 +36,7 @@ def usage_error(error_msg = "unspecified error"): + + if len(args) > 0: + if args[0] == "-h": +- print usage() ++ print(usage()) + exit(0); + elif args[0] == "-s": + try: +@@ -54,15 +55,15 @@ srv = QEMUMonitorProtocol(socket_path) + srv.connect() + + def list_node(path): +- print '%s' % path ++ print('%s' % path) + items = srv.command('qom-list', path=path) + for item in items: + if not item['type'].startswith('child<'): + try: +- print ' %s: %s (%s)' % (item['name'], srv.command('qom-get', path=path, property=item['name']), item['type']) ++ print(' %s: %s (%s)' % (item['name'], srv.command('qom-get', path=path, property=item['name']), item['type'])) + except: +- print ' %s: (%s)' % (item['name'], item['type']) +- print '' ++ print(' %s: (%s)' % (item['name'], item['type'])) ++ print('') + for item in items: + if item['type'].startswith('child<'): + list_node((path if (path != '/') else '') + '/' + item['name']) +diff --git a/scripts/replay-dump.py b/scripts/replay-dump.py +index e274086..5ae77c8 100755 +--- a/scripts/replay-dump.py ++++ b/scripts/replay-dump.py +@@ -18,6 +18,7 @@ + # You should have received a copy of the GNU Lesser General Public + # License along with this library; if not, see . + ++from __future__ import print_function + import argparse + import struct + from collections import namedtuple +@@ -89,9 +90,9 @@ def call_decode(table, index, dumpfile): + "Search decode table for next step" + decoder = next((d for d in table if d.eid == index), None) + if not decoder: +- print "Could not decode index: %d" % (index) +- print "Entry is: %s" % (decoder) +- print "Decode Table is:\n%s" % (table) ++ print("Could not decode index: %d" % (index)) ++ print("Entry is: %s" % (decoder)) ++ print("Decode Table is:\n%s" % (table)) + return False + else: + return decoder.fn(decoder.eid, decoder.name, dumpfile) +@@ -103,23 +104,23 @@ def print_event(eid, name, string=None, event_count=None): + event_count = replay_state.event_count + + if string: +- print "%d:%s(%d) %s" % (event_count, name, eid, string) ++ print("%d:%s(%d) %s" % (event_count, name, eid, string)) + else: +- print "%d:%s(%d)" % (event_count, name, eid) ++ print("%d:%s(%d)" % (event_count, name, eid)) + + + # Decoders for each event type + + def decode_unimp(eid, name, _unused_dumpfile): + "Unimplimented decoder, will trigger exit" +- print "%s not handled - will now stop" % (name) ++ print("%s not handled - will now stop" % (name)) + return False + + # Checkpoint decoder + def swallow_async_qword(eid, name, dumpfile): + "Swallow a qword of data without looking at it" + step_id = read_qword(dumpfile) +- print " %s(%d) @ %d" % (name, eid, step_id) ++ print(" %s(%d) @ %d" % (name, eid, step_id)) + return True + + async_decode_table = [ Decoder(0, "REPLAY_ASYNC_EVENT_BH", swallow_async_qword), +@@ -139,8 +140,8 @@ def decode_async(eid, name, dumpfile): + async_event_checkpoint = read_byte(dumpfile) + + if async_event_checkpoint != replay_state.current_checkpoint: +- print " mismatch between checkpoint %d and async data %d" % ( +- replay_state.current_checkpoint, async_event_checkpoint) ++ print(" mismatch between checkpoint %d and async data %d" % ( ++ replay_state.current_checkpoint, async_event_checkpoint)) + return True + + return call_decode(async_decode_table, async_event_kind, dumpfile) +@@ -283,7 +284,7 @@ def decode_file(filename): + version = read_dword(dumpfile) + junk = read_qword(dumpfile) + +- print "HEADER: version 0x%x" % (version) ++ print("HEADER: version 0x%x" % (version)) + + if version == 0xe02007: + event_decode_table = v7_event_table +diff --git a/scripts/signrom.py b/scripts/signrom.py +index 0497a1c..313ee28 100644 +--- a/scripts/signrom.py ++++ b/scripts/signrom.py +@@ -1,3 +1,4 @@ ++from __future__ import print_function + # + # Option ROM signing utility + # +diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py +index 46a5fd7..54e761f 100755 +--- a/scripts/simpletrace.py ++++ b/scripts/simpletrace.py +@@ -9,6 +9,7 @@ + # + # For help see docs/devel/tracing.txt + ++from __future__ import print_function + import struct + import re + import inspect +@@ -257,6 +258,6 @@ if __name__ == '__main__': + else: + fields.append('%s=0x%x' % (name, rec[i])) + i += 1 +- print ' '.join(fields) ++ print(' '.join(fields)) + + run(Formatter()) +diff --git a/scripts/vmstate-static-checker.py b/scripts/vmstate-static-checker.py +index ffb13d1..7c97db6 100755 +--- a/scripts/vmstate-static-checker.py ++++ b/scripts/vmstate-static-checker.py +@@ -19,6 +19,7 @@ + # You should have received a copy of the GNU General Public License along + # with this program; if not, see . + ++from __future__ import print_function + import argparse + import json + import sys +@@ -175,10 +176,10 @@ def check_fields(src_fields, dest_fields, desc, sec): + except StopIteration: + if d_iter_list == []: + # We were not in a substruct +- print "Section \"" + sec + "\",", +- print "Description " + "\"" + desc + "\":", +- print "expected field \"" + s_item["field"] + "\",", +- print "while dest has no further fields" ++ print("Section \"" + sec + "\",", end=' ') ++ print("Description " + "\"" + desc + "\":", end=' ') ++ print("expected field \"" + s_item["field"] + "\",", end=' ') ++ print("while dest has no further fields") + bump_taint() + break + +@@ -196,10 +197,10 @@ def check_fields(src_fields, dest_fields, desc, sec): + advance_dest = True + continue + if unused_count < 0: +- print "Section \"" + sec + "\",", +- print "Description \"" + desc + "\":", +- print "unused size mismatch near \"", +- print s_item["field"] + "\"" ++ print("Section \"" + sec + "\",", end=' ') ++ print("Description \"" + desc + "\":", end=' ') ++ print("unused size mismatch near \"", end=' ') ++ print(s_item["field"] + "\"") + bump_taint() + break + continue +@@ -210,10 +211,10 @@ def check_fields(src_fields, dest_fields, desc, sec): + advance_src = True + continue + if unused_count < 0: +- print "Section \"" + sec + "\",", +- print "Description \"" + desc + "\":", +- print "unused size mismatch near \"", +- print d_item["field"] + "\"" ++ print("Section \"" + sec + "\",", end=' ') ++ print("Description \"" + desc + "\":", end=' ') ++ print("unused size mismatch near \"", end=' ') ++ print(d_item["field"] + "\"") + bump_taint() + break + continue +@@ -261,10 +262,10 @@ def check_fields(src_fields, dest_fields, desc, sec): + unused_count = s_item["size"] - d_item["size"] + continue + +- print "Section \"" + sec + "\",", +- print "Description \"" + desc + "\":", +- print "expected field \"" + s_item["field"] + "\",", +- print "got \"" + d_item["field"] + "\"; skipping rest" ++ print("Section \"" + sec + "\",", end=' ') ++ print("Description \"" + desc + "\":", end=' ') ++ print("expected field \"" + s_item["field"] + "\",", end=' ') ++ print("got \"" + d_item["field"] + "\"; skipping rest") + bump_taint() + break + +@@ -288,8 +289,8 @@ def check_subsections(src_sub, dest_sub, desc, sec): + check_descriptions(s_item, d_item, sec) + + if not found: +- print "Section \"" + sec + "\", Description \"" + desc + "\":", +- print "Subsection \"" + s_item["name"] + "\" not found" ++ print("Section \"" + sec + "\", Description \"" + desc + "\":", end=' ') ++ print("Subsection \"" + s_item["name"] + "\" not found") + bump_taint() + + +@@ -298,8 +299,8 @@ def check_description_in_list(s_item, d_item, sec, desc): + return + + if not "Description" in d_item: +- print "Section \"" + sec + "\", Description \"" + desc + "\",", +- print "Field \"" + s_item["field"] + "\": missing description" ++ print("Section \"" + sec + "\", Description \"" + desc + "\",", end=' ') ++ print("Field \"" + s_item["field"] + "\": missing description") + bump_taint() + return + +@@ -310,17 +311,17 @@ def check_descriptions(src_desc, dest_desc, sec): + check_version(src_desc, dest_desc, sec, src_desc["name"]) + + if not check_fields_match(sec, src_desc["name"], dest_desc["name"]): +- print "Section \"" + sec + "\":", +- print "Description \"" + src_desc["name"] + "\"", +- print "missing, got \"" + dest_desc["name"] + "\" instead; skipping" ++ print("Section \"" + sec + "\":", end=' ') ++ print("Description \"" + src_desc["name"] + "\"", end=' ') ++ print("missing, got \"" + dest_desc["name"] + "\" instead; skipping") + bump_taint() + return + + for f in src_desc: + if not f in dest_desc: +- print "Section \"" + sec + "\"", +- print "Description \"" + src_desc["name"] + "\":", +- print "Entry \"" + f + "\" missing" ++ print("Section \"" + sec + "\"", end=' ') ++ print("Description \"" + src_desc["name"] + "\":", end=' ') ++ print("Entry \"" + f + "\" missing") + bump_taint() + continue + +@@ -333,39 +334,39 @@ def check_descriptions(src_desc, dest_desc, sec): + + def check_version(s, d, sec, desc=None): + if s["version_id"] > d["version_id"]: +- print "Section \"" + sec + "\"", ++ print("Section \"" + sec + "\"", end=' ') + if desc: +- print "Description \"" + desc + "\":", +- print "version error:", s["version_id"], ">", d["version_id"] ++ print("Description \"" + desc + "\":", end=' ') ++ print("version error:", s["version_id"], ">", d["version_id"]) + bump_taint() + + if not "minimum_version_id" in d: + return + + if s["version_id"] < d["minimum_version_id"]: +- print "Section \"" + sec + "\"", ++ print("Section \"" + sec + "\"", end=' ') + if desc: +- print "Description \"" + desc + "\":", +- print "minimum version error:", s["version_id"], "<", +- print d["minimum_version_id"] ++ print("Description \"" + desc + "\":", end=' ') ++ print("minimum version error:", s["version_id"], "<", end=' ') ++ print(d["minimum_version_id"]) + bump_taint() + + + def check_size(s, d, sec, desc=None, field=None): + if s["size"] != d["size"]: +- print "Section \"" + sec + "\"", ++ print("Section \"" + sec + "\"", end=' ') + if desc: +- print "Description \"" + desc + "\"", ++ print("Description \"" + desc + "\"", end=' ') + if field: +- print "Field \"" + field + "\"", +- print "size mismatch:", s["size"], ",", d["size"] ++ print("Field \"" + field + "\"", end=' ') ++ print("size mismatch:", s["size"], ",", d["size"]) + bump_taint() + + + def check_machine_type(s, d): + if s["Name"] != d["Name"]: +- print "Warning: checking incompatible machine types:", +- print "\"" + s["Name"] + "\", \"" + d["Name"] + "\"" ++ print("Warning: checking incompatible machine types:", end=' ') ++ print("\"" + s["Name"] + "\", \"" + d["Name"] + "\"") + return + + +@@ -399,7 +400,7 @@ def main(): + # doesn't exist in dest. + dest_sec = get_changed_sec_name(sec) + if not dest_sec in dest_data: +- print "Section \"" + sec + "\" does not exist in dest" ++ print("Section \"" + sec + "\" does not exist in dest") + bump_taint() + continue + +@@ -414,8 +415,8 @@ def main(): + + for entry in s: + if not entry in d: +- print "Section \"" + sec + "\": Entry \"" + entry + "\"", +- print "missing" ++ print("Section \"" + sec + "\": Entry \"" + entry + "\"", end=' ') ++ print("missing") + bump_taint() + continue + +diff --git a/tests/docker/docker.py b/tests/docker/docker.py +index 1246ba9..b43f02e 100755 +--- a/tests/docker/docker.py ++++ b/tests/docker/docker.py +@@ -11,6 +11,7 @@ + # or (at your option) any later version. See the COPYING file in + # the top-level directory. + ++from __future__ import print_function + import os + import sys + sys.path.append(os.path.join(os.path.dirname(__file__), +@@ -87,7 +88,7 @@ def _get_so_libs(executable): + so_lib = search.groups()[1] + libs.append("%s/%s" % (so_path, so_lib)) + except subprocess.CalledProcessError: +- print "%s had no associated libraries (static build?)" % (executable) ++ print("%s had no associated libraries (static build?)" % (executable)) + + return libs + +@@ -161,7 +162,7 @@ class Docker(object): + continue + if only_known and instance_uuid not in self._instances: + continue +- print "Terminating", i ++ print("Terminating", i) + if active: + self._do(["kill", i]) + self._do(["rm", i]) +@@ -288,7 +289,7 @@ class BuildCommand(SubCommand): + if "--no-cache" not in argv and \ + dkr.image_matches_dockerfile(tag, dockerfile): + if not args.quiet: +- print "Image is up to date." ++ print("Image is up to date.") + else: + # Create a docker context directory for the build + docker_dir = tempfile.mkdtemp(prefix="docker_build") +@@ -300,10 +301,10 @@ class BuildCommand(SubCommand): + rc = subprocess.call(os.path.realpath(docker_pre), + cwd=docker_dir, stdout=stdout) + if rc == 3: +- print "Skip" ++ print("Skip") + return 0 + elif rc != 0: +- print "%s exited with code %d" % (docker_pre, rc) ++ print("%s exited with code %d" % (docker_pre, rc)) + return 1 + + # Copy any extra files into the Docker context. These can be +diff --git a/tests/docker/travis.py b/tests/docker/travis.py +index 703a7fd..ea1ef16 100755 +--- a/tests/docker/travis.py ++++ b/tests/docker/travis.py +@@ -11,6 +11,7 @@ + # or (at your option) any later version. See the COPYING file in + # the top-level directory. + ++from __future__ import print_function + import sys + import yaml + import itertools +@@ -34,14 +35,14 @@ def main(): + sys.stderr.write("Usage: %s \n" % sys.argv[0]) + return 1 + conf = load_yaml(sys.argv[1]) +- print "\n".join((": ${%s}" % var for var in conf["env"]["global"])) ++ print("\n".join((": ${%s}" % var for var in conf["env"]["global"]))) + for config in conf_iter(conf): +- print "(" +- print "\n".join(config["env"]) +- print "alias cc=" + config["compiler"] +- print "\n".join(conf["before_script"]) +- print "\n".join(conf["script"]) +- print ")" ++ print("(") ++ print("\n".join(config["env"])) ++ print("alias cc=" + config["compiler"]) ++ print("\n".join(conf["before_script"])) ++ print("\n".join(conf["script"])) ++ print(")") + return 0 + + if __name__ == "__main__": +diff --git a/tests/guest-debug/test-gdbstub.py b/tests/guest-debug/test-gdbstub.py +index 31ba6c9..474d2c5 100644 +--- a/tests/guest-debug/test-gdbstub.py ++++ b/tests/guest-debug/test-gdbstub.py +@@ -1,3 +1,4 @@ ++from __future__ import print_function + # + # This script needs to be run on startup + # qemu -kernel ${KERNEL} -s -S +diff --git a/tests/image-fuzzer/runner.py b/tests/image-fuzzer/runner.py +index 96a1c11..8de6569 100755 +--- a/tests/image-fuzzer/runner.py ++++ b/tests/image-fuzzer/runner.py +@@ -18,6 +18,7 @@ + # along with this program. If not, see . + # + ++from __future__ import print_function + import sys + import os + import signal +@@ -36,9 +37,8 @@ except ImportError: + try: + import simplejson as json + except ImportError: +- print >>sys.stderr, \ +- "Warning: Module for JSON processing is not found.\n" \ +- "'--config' and '--command' options are not supported." ++ print("Warning: Module for JSON processing is not found.\n" \ ++ "'--config' and '--command' options are not supported.", file=sys.stderr) + + # Backing file sizes in MB + MAX_BACKING_FILE_SIZE = 10 +@@ -158,9 +158,8 @@ class TestEnv(object): + try: + os.makedirs(self.current_dir) + except OSError as e: +- print >>sys.stderr, \ +- "Error: The working directory '%s' cannot be used. Reason: %s"\ +- % (self.work_dir, e[1]) ++ print("Error: The working directory '%s' cannot be used. Reason: %s"\ ++ % (self.work_dir, e[1]), file=sys.stderr) + raise TestException + self.log = open(os.path.join(self.current_dir, "test.log"), "w") + self.parent_log = open(run_log, "a") +@@ -277,7 +276,7 @@ class TestEnv(object): + if __name__ == '__main__': + + def usage(): +- print """ ++ print(""" + Usage: runner.py [OPTION...] TEST_DIR IMG_GENERATOR + + Set up test environment in TEST_DIR and run a test in it. A module for +@@ -326,7 +325,7 @@ if __name__ == '__main__': + + If '--config' argument is specified, fields not listed in + the configuration array will not be fuzzed. +- """ ++ """) + + def run_test(test_id, seed, work_dir, run_log, cleanup, log_all, + command, fuzz_config): +@@ -357,8 +356,7 @@ if __name__ == '__main__': + ['command=', 'help', 'seed=', 'config=', + 'keep_passed', 'verbose', 'duration=']) + except getopt.error as e: +- print >>sys.stderr, \ +- "Error: %s\n\nTry 'runner.py --help' for more information" % e ++ print("Error: %s\n\nTry 'runner.py --help' for more information" % e, file=sys.stderr) + sys.exit(1) + + command = None +@@ -375,9 +373,8 @@ if __name__ == '__main__': + try: + command = json.loads(arg) + except (TypeError, ValueError, NameError) as e: +- print >>sys.stderr, \ +- "Error: JSON array of test commands cannot be loaded.\n" \ +- "Reason: %s" % e ++ print("Error: JSON array of test commands cannot be loaded.\n" \ ++ "Reason: %s" % e, file=sys.stderr) + sys.exit(1) + elif opt in ('-k', '--keep_passed'): + cleanup = False +@@ -391,15 +388,13 @@ if __name__ == '__main__': + try: + config = json.loads(arg) + except (TypeError, ValueError, NameError) as e: +- print >>sys.stderr, \ +- "Error: JSON array with the fuzzer configuration cannot" \ +- " be loaded\nReason: %s" % e ++ print("Error: JSON array with the fuzzer configuration cannot" \ ++ " be loaded\nReason: %s" % e, file=sys.stderr) + sys.exit(1) + + if not len(args) == 2: +- print >>sys.stderr, \ +- "Expected two parameters\nTry 'runner.py --help'" \ +- " for more information." ++ print("Expected two parameters\nTry 'runner.py --help'" \ ++ " for more information.", file=sys.stderr) + sys.exit(1) + + work_dir = os.path.realpath(args[0]) +@@ -415,9 +410,8 @@ if __name__ == '__main__': + try: + image_generator = __import__(generator_name) + except ImportError as e: +- print >>sys.stderr, \ +- "Error: The image generator '%s' cannot be imported.\n" \ +- "Reason: %s" % (generator_name, e) ++ print("Error: The image generator '%s' cannot be imported.\n" \ ++ "Reason: %s" % (generator_name, e), file=sys.stderr) + sys.exit(1) + + # Enable core dumps +diff --git a/tests/migration/guestperf/engine.py b/tests/migration/guestperf/engine.py +index e14d432..398e3f2 100644 +--- a/tests/migration/guestperf/engine.py ++++ b/tests/migration/guestperf/engine.py +@@ -1,3 +1,4 @@ ++from __future__ import print_function + # + # Migration test main engine + # +@@ -117,7 +118,7 @@ class Engine(object): + # XXX how to get dst timings on remote host ? + + if self._verbose: +- print "Sleeping %d seconds for initial guest workload run" % self._sleep ++ print("Sleeping %d seconds for initial guest workload run" % self._sleep) + sleep_secs = self._sleep + while sleep_secs > 1: + src_qemu_time.append(self._cpu_timing(src_pid)) +@@ -126,7 +127,7 @@ class Engine(object): + sleep_secs -= 1 + + if self._verbose: +- print "Starting migration" ++ print("Starting migration") + if scenario._auto_converge: + resp = src.command("migrate-set-capabilities", + capabilities = [ +@@ -216,7 +217,7 @@ class Engine(object): + + if progress._status == "completed": + if self._verbose: +- print "Sleeping %d seconds for final guest workload run" % self._sleep ++ print("Sleeping %d seconds for final guest workload run" % self._sleep) + sleep_secs = self._sleep + while sleep_secs > 1: + time.sleep(1) +@@ -227,23 +228,23 @@ class Engine(object): + return [progress_history, src_qemu_time, src_vcpu_time] + + if self._verbose and (loop % 20) == 0: +- print "Iter %d: remain %5dMB of %5dMB (total %5dMB @ %5dMb/sec)" % ( ++ print("Iter %d: remain %5dMB of %5dMB (total %5dMB @ %5dMb/sec)" % ( + progress._ram._iterations, + progress._ram._remaining_bytes / (1024 * 1024), + progress._ram._total_bytes / (1024 * 1024), + progress._ram._transferred_bytes / (1024 * 1024), + progress._ram._transfer_rate_mbs, +- ) ++ )) + + if progress._ram._iterations > scenario._max_iters: + if self._verbose: +- print "No completion after %d iterations over RAM" % scenario._max_iters ++ print("No completion after %d iterations over RAM" % scenario._max_iters) + src.command("migrate_cancel") + continue + + if time.time() > (start + scenario._max_time): + if self._verbose: +- print "No completion after %d seconds" % scenario._max_time ++ print("No completion after %d seconds" % scenario._max_time) + src.command("migrate_cancel") + continue + +@@ -251,7 +252,7 @@ class Engine(object): + progress._ram._iterations >= scenario._post_copy_iters and + not post_copy): + if self._verbose: +- print "Switching to post-copy after %d iterations" % scenario._post_copy_iters ++ print("Switching to post-copy after %d iterations" % scenario._post_copy_iters) + resp = src.command("migrate-start-postcopy") + post_copy = True + +@@ -259,7 +260,7 @@ class Engine(object): + progress._ram._iterations >= scenario._pause_iters and + not paused): + if self._verbose: +- print "Pausing VM after %d iterations" % scenario._pause_iters ++ print("Pausing VM after %d iterations" % scenario._pause_iters) + resp = src.command("stop") + paused = True + +@@ -348,7 +349,7 @@ class Engine(object): + if not log: + return [] + if self._debug: +- print log ++ print(log) + + regex = r"[^\s]+\s\((\d+)\):\sINFO:\s(\d+)ms\scopied\s\d+\sGB\sin\s(\d+)ms" + matcher = re.compile(regex) +@@ -407,7 +408,7 @@ class Engine(object): + if uri[0:5] == "unix:": + os.remove(uri[5:]) + if self._verbose: +- print "Finished migration" ++ print("Finished migration") + + src.shutdown() + dst.shutdown() +@@ -420,7 +421,7 @@ class Engine(object): + self._initrd, self._transport, self._sleep) + except Exception as e: + if self._debug: +- print "Failed: %s" % str(e) ++ print("Failed: %s" % str(e)) + try: + src.shutdown() + except: +@@ -431,7 +432,7 @@ class Engine(object): + pass + + if self._debug: +- print src.get_log() +- print dst.get_log() ++ print(src.get_log()) ++ print(dst.get_log()) + raise + +diff --git a/tests/migration/guestperf/plot.py b/tests/migration/guestperf/plot.py +index bc42249..aa98912 100644 +--- a/tests/migration/guestperf/plot.py ++++ b/tests/migration/guestperf/plot.py +@@ -1,3 +1,4 @@ ++from __future__ import print_function + # + # Migration test graph plotting + # +@@ -588,7 +589,7 @@ class Plot(object): + """ + + def generate_html(self, fh): +- print >>fh, """ ++ print(""" + + +@@ -601,19 +602,19 @@ class Plot(object): +

Migration report

+

Chart summary

+
+-""" % self._generate_style() +- print >>fh, self._generate_chart() +- print >>fh, """ ++""" % self._generate_style(), file=fh) ++ print(self._generate_chart(), file=fh) ++ print(""" +
+

Report details

+
+-""" +- print >>fh, self._generate_report() +- print >>fh, """ ++""", file=fh) ++ print(self._generate_report(), file=fh) ++ print(""" +
+ + +-""" ++""", file=fh) + + def generate(self, filename): + if filename is None: +diff --git a/tests/migration/guestperf/shell.py b/tests/migration/guestperf/shell.py +index b272978..a6b8cec 100644 +--- a/tests/migration/guestperf/shell.py ++++ b/tests/migration/guestperf/shell.py +@@ -1,3 +1,4 @@ ++from __future__ import print_function + # + # Migration test command line shell integration + # +@@ -160,13 +161,13 @@ class Shell(BaseShell): + try: + report = engine.run(hardware, scenario) + if args.output is None: +- print report.to_json() ++ print(report.to_json()) + else: + with open(args.output, "w") as fh: +- print >>fh, report.to_json() ++ print(report.to_json(), file=fh) + return 0 + except Exception as e: +- print >>sys.stderr, "Error: %s" % str(e) ++ print("Error: %s" % str(e), file=sys.stderr) + if args.debug: + raise + return 1 +@@ -199,11 +200,11 @@ class BatchShell(BaseShell): + name = os.path.join(comparison._name, scenario._name) + if not fnmatch.fnmatch(name, args.filter): + if args.verbose: +- print "Skipping %s" % name ++ print("Skipping %s" % name) + continue + + if args.verbose: +- print "Running %s" % name ++ print("Running %s" % name) + + dirname = os.path.join(args.output, comparison._name) + filename = os.path.join(dirname, scenario._name + ".json") +@@ -211,9 +212,9 @@ class BatchShell(BaseShell): + os.makedirs(dirname) + report = engine.run(hardware, scenario) + with open(filename, "w") as fh: +- print >>fh, report.to_json() ++ print(report.to_json(), file=fh) + except Exception as e: +- print >>sys.stderr, "Error: %s" % str(e) ++ print("Error: %s" % str(e), file=sys.stderr) + if args.debug: + raise + +@@ -246,14 +247,14 @@ class PlotShell(object): + + + if len(args.reports) == 0: +- print >>sys.stderr, "At least one report required" ++ print("At least one report required", file=sys.stderr) + return 1 + + if not (args.qemu_cpu or + args.vcpu_cpu or + args.total_guest_cpu or + args.split_guest_cpu): +- print >>sys.stderr, "At least one chart type is required" ++ print("At least one chart type is required", file=sys.stderr) + return 1 + + reports = [] +diff --git a/tests/qemu-iotests/149 b/tests/qemu-iotests/149 +index 223cd68..d3ffa25 100755 +--- a/tests/qemu-iotests/149 ++++ b/tests/qemu-iotests/149 +@@ -20,6 +20,7 @@ + # Exercise the QEMU 'luks' block driver to validate interoperability + # with the Linux dm-crypt + cryptsetup implementation + ++from __future__ import print_function + import subprocess + import os + import os.path +@@ -376,7 +377,7 @@ def test_once(config, qemu_img=False): + finally: + iotests.log("# Delete image") + delete_image(config) +- print ++ print() + + + # Obviously we only work with the luks image format +diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165 +index 2936929..88f62d3 100755 +--- a/tests/qemu-iotests/165 ++++ b/tests/qemu-iotests/165 +@@ -18,6 +18,7 @@ + # along with this program. If not, see . + # + ++from __future__ import print_function + import os + import re + import iotests +@@ -85,7 +86,7 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase): + log = re.sub(r'^\[I \d+\.\d+\] OPENED\n', '', log) + log = re.sub(r'\[I \+\d+\.\d+\] CLOSED\n?$', '', log) + if log: +- print log ++ print(log) + + self.vm = self.mkVm() + self.vm.launch() +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index b25d48a..26e6046 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -1,3 +1,4 @@ ++from __future__ import print_function + # Common utilities and Python wrappers for qemu-iotests + # + # Copyright (C) 2012 IBM Corp. +@@ -209,7 +210,7 @@ def filter_qmp_event(event): + def log(msg, filters=[]): + for flt in filters: + msg = flt(msg) +- print msg ++ print(msg) + + class Timeout: + def __init__(self, seconds, errmsg = "Timeout"): +@@ -525,7 +526,7 @@ def notrun(reason): + seq = os.path.basename(sys.argv[0]) + + open('%s/%s.notrun' % (output_dir, seq), 'wb').write(reason + '\n') +- print '%s not run: %s' % (seq, reason) ++ print('%s not run: %s' % (seq, reason)) + sys.exit(0) + + def verify_image_format(supported_fmts=[], unsupported_fmts=[]): +diff --git a/tests/qemu-iotests/nbd-fault-injector.py b/tests/qemu-iotests/nbd-fault-injector.py +index 8a04d97..f9193c0 100755 +--- a/tests/qemu-iotests/nbd-fault-injector.py ++++ b/tests/qemu-iotests/nbd-fault-injector.py +@@ -43,6 +43,7 @@ + # This work is licensed under the terms of the GNU GPL, version 2 or later. + # See the COPYING file in the top-level directory. + ++from __future__ import print_function + import sys + import socket + import struct +@@ -110,7 +111,7 @@ class FaultInjectionSocket(object): + for rule in self.rules: + if rule.match(event, io): + if rule.when == 0 or bufsize is None: +- print 'Closing connection on rule match %s' % rule.name ++ print('Closing connection on rule match %s' % rule.name) + sys.exit(0) + if rule.when != -1: + return rule.when +@@ -182,7 +183,7 @@ def handle_connection(conn, use_export): + elif req.type == NBD_CMD_DISC: + break + else: +- print 'unrecognized command type %#02x' % req.type ++ print('unrecognized command type %#02x' % req.type) + break + conn.close() + +@@ -242,7 +243,7 @@ def open_socket(path): + sock = socket.socket(socket.AF_UNIX) + sock.bind(path) + sock.listen(0) +- print 'Listening on %s' % path ++ print('Listening on %s' % path) + sys.stdout.flush() # another process may be waiting, show message now + return sock + +diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py +index 9cc4cf7..b95a837 100755 +--- a/tests/qemu-iotests/qcow2.py ++++ b/tests/qemu-iotests/qcow2.py +@@ -1,5 +1,6 @@ + #!/usr/bin/env python + ++from __future__ import print_function + import sys + import struct + import string +@@ -129,8 +130,8 @@ class QcowHeader: + + def dump(self): + for f in QcowHeader.fields: +- print "%-25s" % f[2], f[1] % self.__dict__[f[2]] +- print "" ++ print("%-25s" % f[2], f[1] % self.__dict__[f[2]]) ++ print("") + + def dump_extensions(self): + for ex in self.extensions: +@@ -141,11 +142,11 @@ class QcowHeader: + else: + data = "" + +- print "Header extension:" +- print "%-25s %#x" % ("magic", ex.magic) +- print "%-25s %d" % ("length", ex.length) +- print "%-25s %s" % ("data", data) +- print "" ++ print("Header extension:") ++ print("%-25s %#x" % ("magic", ex.magic)) ++ print("%-25s %d" % ("length", ex.length)) ++ print("%-25s %s" % ("data", data)) ++ print("") + + + def cmd_dump_header(fd): +@@ -157,12 +158,12 @@ def cmd_set_header(fd, name, value): + try: + value = int(value, 0) + except: +- print "'%s' is not a valid number" % value ++ print("'%s' is not a valid number" % value) + sys.exit(1) + + fields = (field[2] for field in QcowHeader.fields) + if not name in fields: +- print "'%s' is not a known header field" % name ++ print("'%s' is not a known header field" % name) + sys.exit(1) + + h = QcowHeader(fd) +@@ -173,7 +174,7 @@ def cmd_add_header_ext(fd, magic, data): + try: + magic = int(magic, 0) + except: +- print "'%s' is not a valid magic number" % magic ++ print("'%s' is not a valid magic number" % magic) + sys.exit(1) + + h = QcowHeader(fd) +@@ -188,7 +189,7 @@ def cmd_del_header_ext(fd, magic): + try: + magic = int(magic, 0) + except: +- print "'%s' is not a valid magic number" % magic ++ print("'%s' is not a valid magic number" % magic) + sys.exit(1) + + h = QcowHeader(fd) +@@ -200,7 +201,7 @@ def cmd_del_header_ext(fd, magic): + h.extensions.remove(ex) + + if not found: +- print "No such header extension" ++ print("No such header extension") + return + + h.update(fd) +@@ -211,7 +212,7 @@ def cmd_set_feature_bit(fd, group, bit): + if bit < 0 or bit >= 64: + raise ValueError + except: +- print "'%s' is not a valid bit number in range [0, 64)" % bit ++ print("'%s' is not a valid bit number in range [0, 64)" % bit) + sys.exit(1) + + h = QcowHeader(fd) +@@ -222,7 +223,7 @@ def cmd_set_feature_bit(fd, group, bit): + elif group == 'autoclear': + h.autoclear_features |= 1 << bit + else: +- print "'%s' is not a valid group, try 'incompatible', 'compatible', or 'autoclear'" % group ++ print("'%s' is not a valid group, try 'incompatible', 'compatible', or 'autoclear'" % group) + sys.exit(1) + + h.update(fd) +@@ -248,16 +249,16 @@ def main(filename, cmd, args): + else: + handler(fd, *args) + return +- print "Unknown command '%s'" % cmd ++ print("Unknown command '%s'" % cmd) + finally: + fd.close() + + def usage(): +- print "Usage: %s [, ...]" % sys.argv[0] +- print "" +- print "Supported commands:" ++ print("Usage: %s [, ...]" % sys.argv[0]) ++ print("") ++ print("Supported commands:") + for name, handler, num_args, desc in cmds: +- print " %-20s - %s" % (name, desc) ++ print(" %-20s - %s" % (name, desc)) + + if __name__ == '__main__': + if len(sys.argv) < 3: +diff --git a/tests/qemu-iotests/qed.py b/tests/qemu-iotests/qed.py +index 748068d..ea469b9 100755 +--- a/tests/qemu-iotests/qed.py ++++ b/tests/qemu-iotests/qed.py +@@ -10,6 +10,7 @@ + # This work is licensed under the terms of the GNU GPL, version 2 or later. + # See the COPYING file in the top-level directory. + ++from __future__ import print_function + import sys + import struct + import random +@@ -108,12 +109,12 @@ def corrupt_table_invalidate(qed, table): + def cmd_show(qed, *args): + '''show [header|l1|l2 ]- Show header or l1/l2 tables''' + if not args or args[0] == 'header': +- print qed.header ++ print(qed.header) + elif args[0] == 'l1': +- print qed.l1_table ++ print(qed.l1_table) + elif len(args) == 2 and args[0] == 'l2': + offset = int(args[1]) +- print qed.read_table(offset) ++ print(qed.read_table(offset)) + else: + err('unrecognized sub-command') + +@@ -146,7 +147,7 @@ def cmd_invalidate(qed, table_level): + def cmd_need_check(qed, *args): + '''need-check [on|off] - Test, set, or clear the QED_F_NEED_CHECK header bit''' + if not args: +- print bool(qed.header['features'] & QED_F_NEED_CHECK) ++ print(bool(qed.header['features'] & QED_F_NEED_CHECK)) + return + + if args[0] == 'on': +@@ -208,11 +209,11 @@ def cmd_copy_metadata(qed, outfile): + out.close() + + def usage(): +- print 'Usage: %s [, ...]' % sys.argv[0] +- print +- print 'Supported commands:' ++ print('Usage: %s [, ...]' % sys.argv[0]) ++ print() ++ print('Supported commands:') + for cmd in sorted(x for x in globals() if x.startswith('cmd_')): +- print globals()[cmd].__doc__ ++ print(globals()[cmd].__doc__) + sys.exit(1) + + def main(): +diff --git a/tests/vm/basevm.py b/tests/vm/basevm.py +index 3a2d508..3643117 100755 +--- a/tests/vm/basevm.py ++++ b/tests/vm/basevm.py +@@ -11,6 +11,7 @@ + # the COPYING file in the top-level directory. + # + ++from __future__ import print_function + import os + import sys + import logging +@@ -222,7 +223,7 @@ def main(vmcls): + try: + args, argv = parse_args(vmcls.name) + if not argv and not args.build_qemu and not args.build_image: +- print "Nothing to do?" ++ print("Nothing to do?") + return 1 + logging.basicConfig(level=(logging.DEBUG if args.debug + else logging.WARN)) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-Add-rendernode-display-option-for-egl-headless.patch b/SOURCES/kvm-qapi-Add-rendernode-display-option-for-egl-headless.patch new file mode 100644 index 0000000..34a6cf6 --- /dev/null +++ b/SOURCES/kvm-qapi-Add-rendernode-display-option-for-egl-headless.patch @@ -0,0 +1,73 @@ +From f1d966bcc5da6bea34ba2052d8645add9db7e057 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Fri, 14 Dec 2018 08:26:40 +0000 +Subject: [PATCH 3/5] qapi: Add "rendernode" display option for egl-headless +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20181214082642.21878-4-kraxel@redhat.com> +Patchwork-id: 83504 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 3/5] qapi: Add "rendernode" display option for egl-headless +Bugzilla: 1652871 +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Erik Skultety + +From: Erik Skultety + +Unlike SPICE, egl-headless doesn't offer a way of specifying the DRM +node used for OpenGL, hence QEMU always selecting the first one that is +available. Thus, add the 'rendernode' option for egl-headless to QAPI. + +Signed-off-by: Erik Skultety +Message-id: 7658e15eca72d520e7a5fb1c2e724702d83d4f7f.1542362949.git.eskultet@redhat.com +Signed-off-by: Gerd Hoffmann +(cherry picked from commit d4dc4ab133b5d7b066aa14036f297ed20398dd32) +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + qapi/ui.json + +Signed-off-by: Danilo C. L. de Paula +--- + qapi/ui.json | 16 +++++++++++++++- + 1 file changed, 15 insertions(+), 1 deletion(-) + +diff --git a/qapi/ui.json b/qapi/ui.json +index 5d01ad4..3e8aeee 100644 +--- a/qapi/ui.json ++++ b/qapi/ui.json +@@ -1020,6 +1020,20 @@ + 'data' : { '*grab-on-hover' : 'bool' } } + + ## ++# @DisplayEGLHeadless: ++# ++# EGL headless display options. ++# ++# @rendernode: Which DRM render node should be used. Default is the first ++# available node on the host. ++# ++# Since: 3.1 ++# ++## ++{ 'struct' : 'DisplayEGLHeadless', ++ 'data' : { '*rendernode' : 'str' } } ++ ++## + # @DisplayType: + # + # Display (user interface) type. +@@ -1054,6 +1068,6 @@ + 'none' : 'DisplayNoOpts', + 'gtk' : 'DisplayGTK', + 'sdl' : 'DisplayNoOpts', +- 'egl-headless' : 'DisplayNoOpts', ++ 'egl-headless' : 'DisplayEGLHeadless', + 'curses' : 'DisplayNoOpts', + 'cocoa' : 'DisplayNoOpts' } } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-add-disabled-parameter-to-block-dirty-bitmap-ad.patch b/SOURCES/kvm-qapi-add-disabled-parameter-to-block-dirty-bitmap-ad.patch new file mode 100644 index 0000000..2ceecb1 --- /dev/null +++ b/SOURCES/kvm-qapi-add-disabled-parameter-to-block-dirty-bitmap-ad.patch @@ -0,0 +1,101 @@ +From 521f0cf0b4830249e6219c27b825311450519213 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:41 +0200 +Subject: [PATCH 223/268] qapi: add disabled parameter to + block-dirty-bitmap-add + +RH-Author: John Snow +Message-id: <20180718225511.14878-6-jsnow@redhat.com> +Patchwork-id: 81404 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 05/35] qapi: add disabled parameter to block-dirty-bitmap-add +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +This is needed, for example, to create a new bitmap and merge several +disabled bitmaps into a new one. Without this flag we will have to +put block-dirty-bitmap-add and block-dirty-bitmap-disable into one +transaction. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Message-id: 20180606182449.1607-6-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit a6e2ca5f6521553681ae136578ec1cb67e1a7973) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + blockdev.c | 10 ++++++++++ + qapi/block-core.json | 6 +++++- + 2 files changed, 15 insertions(+), 1 deletion(-) + +diff --git a/blockdev.c b/blockdev.c +index 837183c..d425746 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2074,6 +2074,7 @@ static void block_dirty_bitmap_add_prepare(BlkActionState *common, + action->has_granularity, action->granularity, + action->has_persistent, action->persistent, + action->has_autoload, action->autoload, ++ action->has_x_disabled, action->x_disabled, + &local_err); + + if (!local_err) { +@@ -2881,6 +2882,7 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, + bool has_granularity, uint32_t granularity, + bool has_persistent, bool persistent, + bool has_autoload, bool autoload, ++ bool has_disabled, bool disabled, + Error **errp) + { + BlockDriverState *bs; +@@ -2915,6 +2917,10 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, + warn_report("Autoload option is deprecated and its value is ignored"); + } + ++ if (!has_disabled) { ++ disabled = false; ++ } ++ + if (persistent && + !bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp)) + { +@@ -2926,6 +2932,10 @@ void qmp_block_dirty_bitmap_add(const char *node, const char *name, + return; + } + ++ if (disabled) { ++ bdrv_disable_dirty_bitmap(bitmap); ++ } ++ + bdrv_dirty_bitmap_set_persistance(bitmap, persistent); + } + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 50a2763..9a9cfa0 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1734,11 +1734,15 @@ + # Currently, all dirty tracking bitmaps are loaded from Qcow2 on + # open. + # ++# @x-disabled: the bitmap is created in the disabled state, which means that ++# it will not track drive changes. The bitmap may be enabled with ++# x-block-dirty-bitmap-enable. Default is false. (Since: 3.0) ++# + # Since: 2.4 + ## + { 'struct': 'BlockDirtyBitmapAdd', + 'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32', +- '*persistent': 'bool', '*autoload': 'bool' } } ++ '*persistent': 'bool', '*autoload': 'bool', '*x-disabled': 'bool' } } + + ## + # @BlockDirtyBitmapMerge: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-add-query-display-options-command.patch b/SOURCES/kvm-qapi-add-query-display-options-command.patch new file mode 100644 index 0000000..49df694 --- /dev/null +++ b/SOURCES/kvm-qapi-add-query-display-options-command.patch @@ -0,0 +1,92 @@ +From 056ced52b46ff725b79c04813467ce0643c1b878 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Fri, 14 Dec 2018 08:26:42 +0000 +Subject: [PATCH 5/5] qapi: add query-display-options command +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20181214082642.21878-6-kraxel@redhat.com> +Patchwork-id: 83503 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 5/5] qapi: add query-display-options command +Bugzilla: 1652871 +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Erik Skultety + +Add query-display-options command, which allows querying the qemu +display configuration. This isn't particularly useful, except it +exposes QAPI type DisplayOptions in query-qmp-schema, so that libvirt +can discover recently added -display parameter rendernode (commit +d4dc4ab133b). Works around lack of sufficiently powerful command line +introspection. + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Eric Blake +Tested-by: Eric Blake +Tested-by: Erik Skultety +Message-id: 20181122071613.2889-1-kraxel@redhat.com + +[ kraxel: reworded commit message as suggested by armbru ] + +(cherry picked from commit e1ca8f7e1915496148f6e0ce1f7c2309af013312) +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + qapi/ui.json + +Signed-off-by: Danilo C. L. de Paula +--- + qapi/ui.json | 13 +++++++++++++ + vl.c | 6 ++++++ + 2 files changed, 19 insertions(+) + +diff --git a/qapi/ui.json b/qapi/ui.json +index 3e8aeee..1475867 100644 +--- a/qapi/ui.json ++++ b/qapi/ui.json +@@ -1071,3 +1071,16 @@ + 'egl-headless' : 'DisplayEGLHeadless', + 'curses' : 'DisplayNoOpts', + 'cocoa' : 'DisplayNoOpts' } } ++ ++## ++# @query-display-options: ++# ++# Returns information about display configuration ++# ++# Returns: @DisplayOptions ++# ++# Since: 3.1 ++# ++## ++{ 'command': 'query-display-options', ++ 'returns': 'DisplayOptions' } +diff --git a/vl.c b/vl.c +index a4d1e3f..9d32921 100644 +--- a/vl.c ++++ b/vl.c +@@ -131,6 +131,7 @@ int main(int argc, char **argv) + #include "qapi/qapi-commands-block-core.h" + #include "qapi/qapi-commands-misc.h" + #include "qapi/qapi-commands-run-state.h" ++#include "qapi/qapi-commands-ui.h" + #include "qapi/qmp/qerror.h" + #include "sysemu/iothread.h" + +@@ -2138,6 +2139,11 @@ static void parse_display_qapi(const char *optarg) + visit_free(v); + } + ++DisplayOptions *qmp_query_display_options(Error **errp) ++{ ++ return QAPI_CLONE(DisplayOptions, &dpy); ++} ++ + static void parse_display(const char *p) + { + const char *opts; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-add-transaction-support-for-x-block-dirty-bitma.patch b/SOURCES/kvm-qapi-add-transaction-support-for-x-block-dirty-bitma.patch new file mode 100644 index 0000000..c29e9d6 --- /dev/null +++ b/SOURCES/kvm-qapi-add-transaction-support-for-x-block-dirty-bitma.patch @@ -0,0 +1,107 @@ +From 5acc09c438937522b0bc4ae5b9a1968ef11e049c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:15 +0000 +Subject: [PATCH 21/35] qapi: add transaction support for + x-block-dirty-bitmap-merge + +RH-Author: John Snow +Message-id: <20181120181828.15132-12-jsnow@redhat.com> +Patchwork-id: 83066 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 11/24] qapi: add transaction support for x-block-dirty-bitmap-merge +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Vladimir Sementsov-Ogievskiy + +New action is like clean action: do the whole thing in .prepare and +undo in .abort. This behavior for bitmap-changing actions is needed +because backup job actions use bitmap in .prepare. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Reviewed-by: John Snow +(cherry picked from commit 6fd2e40789ef7389b17c5fff93b0bf82d4352cb3) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 35 +++++++++++++++++++++++++++++++++++ + qapi/transaction.json | 2 ++ + 2 files changed, 37 insertions(+) + +diff --git a/blockdev.c b/blockdev.c +index c4b9ddd..d2e7e5a 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2222,6 +2222,35 @@ static void block_dirty_bitmap_disable_abort(BlkActionState *common) + } + } + ++static void block_dirty_bitmap_merge_prepare(BlkActionState *common, ++ Error **errp) ++{ ++ BlockDirtyBitmapMerge *action; ++ BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, ++ common, common); ++ BdrvDirtyBitmap *merge_source; ++ ++ if (action_check_completion_mode(common, errp) < 0) { ++ return; ++ } ++ ++ action = common->action->u.x_block_dirty_bitmap_merge.data; ++ state->bitmap = block_dirty_bitmap_lookup(action->node, ++ action->dst_name, ++ &state->bs, ++ errp); ++ if (!state->bitmap) { ++ return; ++ } ++ ++ merge_source = bdrv_find_dirty_bitmap(state->bs, action->src_name); ++ if (!merge_source) { ++ return; ++ } ++ ++ bdrv_merge_dirty_bitmap(state->bitmap, merge_source, &state->backup, errp); ++} ++ + static void abort_prepare(BlkActionState *common, Error **errp) + { + error_setg(errp, "Transaction aborted using Abort action"); +@@ -2293,6 +2322,12 @@ static const BlkActionOps actions[] = { + .prepare = block_dirty_bitmap_disable_prepare, + .abort = block_dirty_bitmap_disable_abort, + }, ++ [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_MERGE] = { ++ .instance_size = sizeof(BlockDirtyBitmapState), ++ .prepare = block_dirty_bitmap_merge_prepare, ++ .commit = block_dirty_bitmap_free_backup, ++ .abort = block_dirty_bitmap_restore, ++ }, + /* Where are transactions for MIRROR, COMMIT and STREAM? + * Although these blockjobs use transaction callbacks like the backup job, + * these jobs do not necessarily adhere to transaction semantics. +diff --git a/qapi/transaction.json b/qapi/transaction.json +index d7e4274..5875cdb 100644 +--- a/qapi/transaction.json ++++ b/qapi/transaction.json +@@ -48,6 +48,7 @@ + # - @block-dirty-bitmap-clear: since 2.5 + # - @x-block-dirty-bitmap-enable: since 3.0 + # - @x-block-dirty-bitmap-disable: since 3.0 ++# - @x-block-dirty-bitmap-merge: since 3.1 + # - @blockdev-backup: since 2.3 + # - @blockdev-snapshot: since 2.5 + # - @blockdev-snapshot-internal-sync: since 1.7 +@@ -63,6 +64,7 @@ + 'block-dirty-bitmap-clear': 'BlockDirtyBitmap', + 'x-block-dirty-bitmap-enable': 'BlockDirtyBitmap', + 'x-block-dirty-bitmap-disable': 'BlockDirtyBitmap', ++ 'x-block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge', + 'blockdev-backup': 'BlockdevBackup', + 'blockdev-snapshot': 'BlockdevSnapshot', + 'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal', +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-add-x-block-dirty-bitmap-enable-disable.patch b/SOURCES/kvm-qapi-add-x-block-dirty-bitmap-enable-disable.patch new file mode 100644 index 0000000..17571d2 --- /dev/null +++ b/SOURCES/kvm-qapi-add-x-block-dirty-bitmap-enable-disable.patch @@ -0,0 +1,142 @@ +From bdb24120ae9af1944a3b9d50ec9830ec075d5cf0 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:38 +0200 +Subject: [PATCH 220/268] qapi: add x-block-dirty-bitmap-enable/disable + +RH-Author: John Snow +Message-id: <20180718225511.14878-3-jsnow@redhat.com> +Patchwork-id: 81406 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 02/35] qapi: add x-block-dirty-bitmap-enable/disable +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Expose the ability to turn bitmaps "on" or "off". This is experimental +and principally for the sake of the Libvirt Checkpoints API, and it may +or may not be committed for 3.0. + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Message-id: 20180606182449.1607-3-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit 5c5d2e50e5ac85234d793f0127a20ea3424a1229) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + blockdev.c | 42 ++++++++++++++++++++++++++++++++++++++++++ + qapi/block-core.json | 42 ++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 84 insertions(+) + +diff --git a/blockdev.c b/blockdev.c +index 721dc9a..9e435f4 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2924,6 +2924,48 @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name, + bdrv_clear_dirty_bitmap(bitmap, NULL); + } + ++void qmp_x_block_dirty_bitmap_enable(const char *node, const char *name, ++ Error **errp) ++{ ++ BlockDriverState *bs; ++ BdrvDirtyBitmap *bitmap; ++ ++ bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp); ++ if (!bitmap) { ++ return; ++ } ++ ++ if (bdrv_dirty_bitmap_frozen(bitmap)) { ++ error_setg(errp, ++ "Bitmap '%s' is currently frozen and cannot be enabled", ++ name); ++ return; ++ } ++ ++ bdrv_enable_dirty_bitmap(bitmap); ++} ++ ++void qmp_x_block_dirty_bitmap_disable(const char *node, const char *name, ++ Error **errp) ++{ ++ BlockDriverState *bs; ++ BdrvDirtyBitmap *bitmap; ++ ++ bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp); ++ if (!bitmap) { ++ return; ++ } ++ ++ if (bdrv_dirty_bitmap_frozen(bitmap)) { ++ error_setg(errp, ++ "Bitmap '%s' is currently frozen and cannot be disabled", ++ name); ++ return; ++ } ++ ++ bdrv_disable_dirty_bitmap(bitmap); ++} ++ + BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node, + const char *name, + Error **errp) +diff --git a/qapi/block-core.json b/qapi/block-core.json +index b0f41f1..6f3fdf4 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1809,6 +1809,48 @@ + 'data': 'BlockDirtyBitmap' } + + ## ++# @x-block-dirty-bitmap-enable: ++# ++# Enables a dirty bitmap so that it will begin tracking disk changes. ++# ++# Returns: nothing on success ++# If @node is not a valid block device, DeviceNotFound ++# If @name is not found, GenericError with an explanation ++# ++# Since: 3.0 ++# ++# Example: ++# ++# -> { "execute": "x-block-dirty-bitmap-enable", ++# "arguments": { "node": "drive0", "name": "bitmap0" } } ++# <- { "return": {} } ++# ++## ++ { 'command': 'x-block-dirty-bitmap-enable', ++ 'data': 'BlockDirtyBitmap' } ++ ++## ++# @x-block-dirty-bitmap-disable: ++# ++# Disables a dirty bitmap so that it will stop tracking disk changes. ++# ++# Returns: nothing on success ++# If @node is not a valid block device, DeviceNotFound ++# If @name is not found, GenericError with an explanation ++# ++# Since: 3.0 ++# ++# Example: ++# ++# -> { "execute": "x-block-dirty-bitmap-disable", ++# "arguments": { "node": "drive0", "name": "bitmap0" } } ++# <- { "return": {} } ++# ++## ++ { 'command': 'x-block-dirty-bitmap-disable', ++ 'data': 'BlockDirtyBitmap' } ++ ++## + # @BlockDirtyBitmapSha256: + # + # SHA256 hash of dirty bitmap data +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-add-x-block-dirty-bitmap-merge.patch b/SOURCES/kvm-qapi-add-x-block-dirty-bitmap-merge.patch new file mode 100644 index 0000000..61b05c5 --- /dev/null +++ b/SOURCES/kvm-qapi-add-x-block-dirty-bitmap-merge.patch @@ -0,0 +1,171 @@ +From 61f7fec68d66c61a6d3058991470cce978fe1e10 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:40 +0200 +Subject: [PATCH 222/268] qapi: add x-block-dirty-bitmap-merge + +RH-Author: John Snow +Message-id: <20180718225511.14878-5-jsnow@redhat.com> +Patchwork-id: 81397 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 04/35] qapi: add x-block-dirty-bitmap-merge +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Message-id: 20180606182449.1607-5-jsnow@redhat.com +Signed-off-by: John Snow +(cherry picked from commit b598e531f1123d2fb72615f1161c66093be751ea) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/dirty-bitmap.c | 18 ++++++++++++++++++ + blockdev.c | 30 ++++++++++++++++++++++++++++++ + include/block/dirty-bitmap.h | 3 ++- + qapi/block-core.json | 38 ++++++++++++++++++++++++++++++++++++++ + 4 files changed, 88 insertions(+), 1 deletion(-) + +diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c +index 5623425..4159d39 100644 +--- a/block/dirty-bitmap.c ++++ b/block/dirty-bitmap.c +@@ -757,3 +757,21 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset) + { + return hbitmap_next_zero(bitmap->bitmap, offset); + } ++ ++void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, ++ Error **errp) ++{ ++ /* only bitmaps from one bds are supported */ ++ assert(dest->mutex == src->mutex); ++ ++ qemu_mutex_lock(dest->mutex); ++ ++ assert(bdrv_dirty_bitmap_enabled(dest)); ++ assert(!bdrv_dirty_bitmap_readonly(dest)); ++ ++ if (!hbitmap_merge(dest->bitmap, src->bitmap)) { ++ error_setg(errp, "Bitmaps are incompatible and can't be merged"); ++ } ++ ++ qemu_mutex_unlock(dest->mutex); ++} +diff --git a/blockdev.c b/blockdev.c +index b74b7d4..837183c 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3045,6 +3045,36 @@ void qmp_x_block_dirty_bitmap_disable(const char *node, const char *name, + bdrv_disable_dirty_bitmap(bitmap); + } + ++void qmp_x_block_dirty_bitmap_merge(const char *node, const char *dst_name, ++ const char *src_name, Error **errp) ++{ ++ BlockDriverState *bs; ++ BdrvDirtyBitmap *dst, *src; ++ ++ dst = block_dirty_bitmap_lookup(node, dst_name, &bs, errp); ++ if (!dst) { ++ return; ++ } ++ ++ if (bdrv_dirty_bitmap_frozen(dst)) { ++ error_setg(errp, "Bitmap '%s' is frozen and cannot be modified", ++ dst_name); ++ return; ++ } else if (bdrv_dirty_bitmap_readonly(dst)) { ++ error_setg(errp, "Bitmap '%s' is readonly and cannot be modified", ++ dst_name); ++ return; ++ } ++ ++ src = bdrv_find_dirty_bitmap(bs, src_name); ++ if (!src) { ++ error_setg(errp, "Dirty bitmap '%s' not found", src_name); ++ return; ++ } ++ ++ bdrv_merge_dirty_bitmap(dst, src, errp); ++} ++ + BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node, + const char *name, + Error **errp) +diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h +index 1ff8949..1e14743 100644 +--- a/include/block/dirty-bitmap.h ++++ b/include/block/dirty-bitmap.h +@@ -70,7 +70,8 @@ void bdrv_dirty_bitmap_set_readonly(BdrvDirtyBitmap *bitmap, bool value); + void bdrv_dirty_bitmap_set_persistance(BdrvDirtyBitmap *bitmap, + bool persistent); + void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked); +- ++void bdrv_merge_dirty_bitmap(BdrvDirtyBitmap *dest, const BdrvDirtyBitmap *src, ++ Error **errp); + + /* Functions that require manual locking. */ + void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap); +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 6f3fdf4..50a2763 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1741,6 +1741,20 @@ + '*persistent': 'bool', '*autoload': 'bool' } } + + ## ++# @BlockDirtyBitmapMerge: ++# ++# @node: name of device/node which the bitmap is tracking ++# ++# @dst_name: name of the destination dirty bitmap ++# ++# @src_name: name of the source dirty bitmap ++# ++# Since: 3.0 ++## ++{ 'struct': 'BlockDirtyBitmapMerge', ++ 'data': { 'node': 'str', 'dst_name': 'str', 'src_name': 'str' } } ++ ++## + # @block-dirty-bitmap-add: + # + # Create a dirty bitmap with a name on the node, and start tracking the writes. +@@ -1851,6 +1865,30 @@ + 'data': 'BlockDirtyBitmap' } + + ## ++# @x-block-dirty-bitmap-merge: ++# ++# Merge @src_name dirty bitmap to @dst_name dirty bitmap. @src_name dirty ++# bitmap is unchanged. On error, @dst_name is unchanged. ++# ++# Returns: nothing on success ++# If @node is not a valid block device, DeviceNotFound ++# If @dst_name or @src_name is not found, GenericError ++# If bitmaps has different sizes or granularities, GenericError ++# ++# Since: 3.0 ++# ++# Example: ++# ++# -> { "execute": "x-block-dirty-bitmap-merge", ++# "arguments": { "node": "drive0", "dst_name": "bitmap0", ++# "src_name": "bitmap1" } } ++# <- { "return": {} } ++# ++## ++ { 'command': 'x-block-dirty-bitmap-merge', ++ 'data': 'BlockDirtyBitmapMerge' } ++ ++## + # @BlockDirtyBitmapSha256: + # + # SHA256 hash of dirty bitmap data +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-block-commit-expose-new-job-properties.patch b/SOURCES/kvm-qapi-block-commit-expose-new-job-properties.patch new file mode 100644 index 0000000..c6ed200 --- /dev/null +++ b/SOURCES/kvm-qapi-block-commit-expose-new-job-properties.patch @@ -0,0 +1,90 @@ +From 3db6ed16ab417b32bcaf9444ab9091f7f599884d Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:27 +0100 +Subject: [PATCH 24/28] qapi/block-commit: expose new job properties + +RH-Author: John Snow +Message-id: <20180925223431.24791-22-jsnow@redhat.com> +Patchwork-id: 82285 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 21/25] qapi/block-commit: expose new job properties +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-13-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 96fbf5345f60a87fab8e7ea79a2406f381027db9) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 8 ++++++++ + qapi/block-core.json | 16 +++++++++++++++- + 2 files changed, 23 insertions(+), 1 deletion(-) + +diff --git a/blockdev.c b/blockdev.c +index 90a50d0..10b5944 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3315,6 +3315,8 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + bool has_backing_file, const char *backing_file, + bool has_speed, int64_t speed, + bool has_filter_node_name, const char *filter_node_name, ++ bool has_auto_finalize, bool auto_finalize, ++ bool has_auto_dismiss, bool auto_dismiss, + Error **errp) + { + BlockDriverState *bs; +@@ -3334,6 +3336,12 @@ void qmp_block_commit(bool has_job_id, const char *job_id, const char *device, + if (!has_filter_node_name) { + filter_node_name = NULL; + } ++ if (has_auto_finalize && !auto_finalize) { ++ job_flags |= JOB_MANUAL_FINALIZE; ++ } ++ if (has_auto_dismiss && !auto_dismiss) { ++ job_flags |= JOB_MANUAL_DISMISS; ++ } + + /* Important Note: + * libvirt relies on the DeviceNotFound error class in order to probe for +diff --git a/qapi/block-core.json b/qapi/block-core.json +index a6399af..c29c4b6 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1475,6 +1475,19 @@ + # above @top. If this option is not given, a node name is + # autogenerated. (Since: 2.9) + # ++# @auto-finalize: When false, this job will wait in a PENDING state after it has ++# finished its work, waiting for @block-job-finalize before ++# making any block graph changes. ++# When true, this job will automatically ++# perform its abort or commit actions. ++# Defaults to true. (Since 3.1) ++# ++# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it ++# has completely ceased all work, and awaits @block-job-dismiss. ++# When true, this job will automatically disappear from the query ++# list without user intervention. ++# Defaults to true. (Since 3.1) ++# + # Returns: Nothing on success + # If commit or stream is already active on this device, DeviceInUse + # If @device does not exist, DeviceNotFound +@@ -1495,7 +1508,8 @@ + { 'command': 'block-commit', + 'data': { '*job-id': 'str', 'device': 'str', '*base': 'str', '*top': 'str', + '*backing-file': 'str', '*speed': 'int', +- '*filter-node-name': 'str' } } ++ '*filter-node-name': 'str', ++ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + + ## + # @drive-backup: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-block-mirror-expose-new-job-properties.patch b/SOURCES/kvm-qapi-block-mirror-expose-new-job-properties.patch new file mode 100644 index 0000000..457d948 --- /dev/null +++ b/SOURCES/kvm-qapi-block-mirror-expose-new-job-properties.patch @@ -0,0 +1,150 @@ +From 47eda5e717db143f5135e00f5bd6a478f613c28d Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:28 +0100 +Subject: [PATCH 25/28] qapi/block-mirror: expose new job properties + +RH-Author: John Snow +Message-id: <20180925223431.24791-23-jsnow@redhat.com> +Patchwork-id: 82274 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 22/25] qapi/block-mirror: expose new job properties +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-14-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit a6b58adec28ff43c0f29ff7c95cdd5d11e87cf61) +Signed-off-by: John Snow + +Conflicts: downstream does not have has_copy_mode. + -blockdev.c + -qapi/block-core.json + +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 14 ++++++++++++++ + qapi/block-core.json | 30 ++++++++++++++++++++++++++++-- + 2 files changed, 42 insertions(+), 2 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index 10b5944..fadafe0 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3707,6 +3707,8 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs, + bool has_unmap, bool unmap, + bool has_filter_node_name, + const char *filter_node_name, ++ bool has_auto_finalize, bool auto_finalize, ++ bool has_auto_dismiss, bool auto_dismiss, + Error **errp) + { + int job_flags = JOB_DEFAULT; +@@ -3732,6 +3734,12 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs, + if (!has_filter_node_name) { + filter_node_name = NULL; + } ++ if (has_auto_finalize && !auto_finalize) { ++ job_flags |= JOB_MANUAL_FINALIZE; ++ } ++ if (has_auto_dismiss && !auto_dismiss) { ++ job_flags |= JOB_MANUAL_DISMISS; ++ } + + if (granularity != 0 && (granularity < 512 || granularity > 1048576 * 64)) { + error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "granularity", +@@ -3908,6 +3916,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp) + arg->has_on_target_error, arg->on_target_error, + arg->has_unmap, arg->unmap, + false, NULL, ++ arg->has_auto_finalize, arg->auto_finalize, ++ arg->has_auto_dismiss, arg->auto_dismiss, + &local_err); + bdrv_unref(target_bs); + error_propagate(errp, local_err); +@@ -3928,6 +3938,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id, + BlockdevOnError on_target_error, + bool has_filter_node_name, + const char *filter_node_name, ++ bool has_auto_finalize, bool auto_finalize, ++ bool has_auto_dismiss, bool auto_dismiss, + Error **errp) + { + BlockDriverState *bs; +@@ -3960,6 +3972,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id, + has_on_target_error, on_target_error, + true, true, + has_filter_node_name, filter_node_name, ++ has_auto_finalize, auto_finalize, ++ has_auto_dismiss, auto_dismiss, + &local_err); + error_propagate(errp, local_err); + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index c29c4b6..3f9ff18 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -1706,6 +1706,18 @@ + # written. Both will result in identical contents. + # Default is true. (Since 2.4) + # ++# @auto-finalize: When false, this job will wait in a PENDING state after it has ++# finished its work, waiting for @block-job-finalize before ++# making any block graph changes. ++# When true, this job will automatically ++# perform its abort or commit actions. ++# Defaults to true. (Since 3.1) ++# ++# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it ++# has completely ceased all work, and awaits @block-job-dismiss. ++# When true, this job will automatically disappear from the query ++# list without user intervention. ++# Defaults to true. (Since 3.1) + # Since: 1.3 + ## + { 'struct': 'DriveMirror', +@@ -1715,7 +1727,8 @@ + '*speed': 'int', '*granularity': 'uint32', + '*buf-size': 'int', '*on-source-error': 'BlockdevOnError', + '*on-target-error': 'BlockdevOnError', +- '*unmap': 'bool' } } ++ '*unmap': 'bool', ++ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + + ## + # @BlockDirtyBitmap: +@@ -1978,6 +1991,18 @@ + # above @device. If this option is not given, a node name is + # autogenerated. (Since: 2.9) + # ++# @auto-finalize: When false, this job will wait in a PENDING state after it has ++# finished its work, waiting for @block-job-finalize before ++# making any block graph changes. ++# When true, this job will automatically ++# perform its abort or commit actions. ++# Defaults to true. (Since 3.1) ++# ++# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it ++# has completely ceased all work, and awaits @block-job-dismiss. ++# When true, this job will automatically disappear from the query ++# list without user intervention. ++# Defaults to true. (Since 3.1) + # Returns: nothing on success. + # + # Since: 2.6 +@@ -1998,7 +2023,8 @@ + '*speed': 'int', '*granularity': 'uint32', + '*buf-size': 'int', '*on-source-error': 'BlockdevOnError', + '*on-target-error': 'BlockdevOnError', +- '*filter-node-name': 'str' } } ++ '*filter-node-name': 'str', ++ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + + ## + # @block_set_io_throttle: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-block-stream-expose-new-job-properties.patch b/SOURCES/kvm-qapi-block-stream-expose-new-job-properties.patch new file mode 100644 index 0000000..cc1bb3b --- /dev/null +++ b/SOURCES/kvm-qapi-block-stream-expose-new-job-properties.patch @@ -0,0 +1,108 @@ +From f86061446c280810e39db23ad22c07161837c429 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:29 +0100 +Subject: [PATCH 26/28] qapi/block-stream: expose new job properties + +RH-Author: John Snow +Message-id: <20180925223431.24791-24-jsnow@redhat.com> +Patchwork-id: 82278 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 23/25] qapi/block-stream: expose new job properties +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-15-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 241ca1ab78542f02e666636e0323bcfe3cb1d5e8) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + blockdev.c | 9 +++++++++ + hmp.c | 5 +++-- + qapi/block-core.json | 16 +++++++++++++++- + 3 files changed, 27 insertions(+), 3 deletions(-) + +diff --git a/blockdev.c b/blockdev.c +index fadafe0..bf026d2 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -3227,6 +3227,8 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device, + bool has_backing_file, const char *backing_file, + bool has_speed, int64_t speed, + bool has_on_error, BlockdevOnError on_error, ++ bool has_auto_finalize, bool auto_finalize, ++ bool has_auto_dismiss, bool auto_dismiss, + Error **errp) + { + BlockDriverState *bs, *iter; +@@ -3296,6 +3298,13 @@ void qmp_block_stream(bool has_job_id, const char *job_id, const char *device, + /* backing_file string overrides base bs filename */ + base_name = has_backing_file ? backing_file : base_name; + ++ if (has_auto_finalize && !auto_finalize) { ++ job_flags |= JOB_MANUAL_FINALIZE; ++ } ++ if (has_auto_dismiss && !auto_dismiss) { ++ job_flags |= JOB_MANUAL_DISMISS; ++ } ++ + stream_start(has_job_id ? job_id : NULL, bs, base_bs, base_name, + job_flags, has_speed ? speed : 0, on_error, &local_err); + if (local_err) { +diff --git a/hmp.c b/hmp.c +index a25c7bd..fefec87 100644 +--- a/hmp.c ++++ b/hmp.c +@@ -1797,8 +1797,9 @@ void hmp_block_stream(Monitor *mon, const QDict *qdict) + int64_t speed = qdict_get_try_int(qdict, "speed", 0); + + qmp_block_stream(true, device, device, base != NULL, base, false, NULL, +- false, NULL, qdict_haskey(qdict, "speed"), speed, +- true, BLOCKDEV_ON_ERROR_REPORT, &error); ++ false, NULL, qdict_haskey(qdict, "speed"), speed, true, ++ BLOCKDEV_ON_ERROR_REPORT, false, false, false, false, ++ &error); + + hmp_handle_error(mon, &error); + } +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 3f9ff18..ba4bba0 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -2290,6 +2290,19 @@ + # 'stop' and 'enospc' can only be used if the block device + # supports io-status (see BlockInfo). Since 1.3. + # ++# @auto-finalize: When false, this job will wait in a PENDING state after it has ++# finished its work, waiting for @block-job-finalize before ++# making any block graph changes. ++# When true, this job will automatically ++# perform its abort or commit actions. ++# Defaults to true. (Since 3.1) ++# ++# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it ++# has completely ceased all work, and awaits @block-job-dismiss. ++# When true, this job will automatically disappear from the query ++# list without user intervention. ++# Defaults to true. (Since 3.1) ++# + # Returns: Nothing on success. If @device does not exist, DeviceNotFound. + # + # Since: 1.1 +@@ -2305,7 +2318,8 @@ + { 'command': 'block-stream', + 'data': { '*job-id': 'str', 'device': 'str', '*base': 'str', + '*base-node': 'str', '*backing-file': 'str', '*speed': 'int', +- '*on-error': 'BlockdevOnError' } } ++ '*on-error': 'BlockdevOnError', ++ '*auto-finalize': 'bool', '*auto-dismiss': 'bool' } } + + ## + # @block-job-set-speed: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qapi-new-qmp-command-nbd-server-add-bitmap.patch b/SOURCES/kvm-qapi-new-qmp-command-nbd-server-add-bitmap.patch new file mode 100644 index 0000000..0b0a791 --- /dev/null +++ b/SOURCES/kvm-qapi-new-qmp-command-nbd-server-add-bitmap.patch @@ -0,0 +1,104 @@ +From 0af062f39429cd402392d6580fcb4c530935c4d3 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:55:04 +0200 +Subject: [PATCH 246/268] qapi: new qmp command nbd-server-add-bitmap + +RH-Author: John Snow +Message-id: <20180718225511.14878-29-jsnow@redhat.com> +Patchwork-id: 81408 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 28/35] qapi: new qmp command nbd-server-add-bitmap +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +For now, the actual command ix x-nbd-server-add-bitmap, reflecting +the fact that we are still working on libvirt code that proves the +command works as needed, and also the fact that we may remove +bitmap-export-name (and just require that the exported name be the +bitmap name). + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-Id: <20180609151758.17343-6-vsementsov@virtuozzo.com> +Reviewed-by: Eric Blake +[eblake: make the command experimental by adding x- prefix] +Signed-off-by: Eric Blake +(cherry picked from commit 767f0c7d6cddedbc97ad700bd1e0229cc2ce5eb5) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + blockdev-nbd.c | 23 +++++++++++++++++++++++ + qapi/block.json | 23 +++++++++++++++++++++++ + 2 files changed, 46 insertions(+) + +diff --git a/blockdev-nbd.c b/blockdev-nbd.c +index 65a8473..1ef1104 100644 +--- a/blockdev-nbd.c ++++ b/blockdev-nbd.c +@@ -220,3 +220,26 @@ void qmp_nbd_server_stop(Error **errp) + nbd_server_free(nbd_server); + nbd_server = NULL; + } ++ ++void qmp_x_nbd_server_add_bitmap(const char *name, const char *bitmap, ++ bool has_bitmap_export_name, ++ const char *bitmap_export_name, ++ Error **errp) ++{ ++ NBDExport *exp; ++ ++ if (!nbd_server) { ++ error_setg(errp, "NBD server not running"); ++ return; ++ } ++ ++ exp = nbd_export_find(name); ++ if (exp == NULL) { ++ error_setg(errp, "Export '%s' is not found", name); ++ return; ++ } ++ ++ nbd_export_bitmap(exp, bitmap, ++ has_bitmap_export_name ? bitmap_export_name : bitmap, ++ errp); ++} +diff --git a/qapi/block.json b/qapi/block.json +index 528c1e6..8b8678b 100644 +--- a/qapi/block.json ++++ b/qapi/block.json +@@ -296,6 +296,29 @@ + 'data': {'name': 'str', '*mode': 'NbdServerRemoveMode'} } + + ## ++# @x-nbd-server-add-bitmap: ++# ++# Expose a dirty bitmap associated with the selected export. The bitmap search ++# starts at the device attached to the export, and includes all backing files. ++# The exported bitmap is then locked until the NBD export is removed. ++# ++# @name: Export name. ++# ++# @bitmap: Bitmap name to search for. ++# ++# @bitmap-export-name: How the bitmap will be seen by nbd clients ++# (default @bitmap) ++# ++# Note: the client must use NBD_OPT_SET_META_CONTEXT with a query of ++# "qemu:dirty-bitmap:NAME" (where NAME matches @bitmap-export-name) to access ++# the exposed bitmap. ++# ++# Since: 3.0 ++## ++ { 'command': 'x-nbd-server-add-bitmap', ++ 'data': {'name': 'str', 'bitmap': 'str', '*bitmap-export-name': 'str'} } ++ ++## + # @nbd-server-stop: + # + # Stop QEMU's embedded NBD server, and unregister all devices previously +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Assign-the-L2-cache-relatively-to-the-image-si.patch b/SOURCES/kvm-qcow2-Assign-the-L2-cache-relatively-to-the-image-si.patch new file mode 100644 index 0000000..304ce28 --- /dev/null +++ b/SOURCES/kvm-qcow2-Assign-the-L2-cache-relatively-to-the-image-si.patch @@ -0,0 +1,231 @@ +From 85dd7b43885d64406c5ea9af9daefe89a6c7475f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:34 +0000 +Subject: [PATCH 09/15] qcow2: Assign the L2 cache relatively to the image size + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-10-kwolf@redhat.com> +Patchwork-id: 83289 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 09/15] qcow2: Assign the L2 cache relatively to the image size +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +Sufficient L2 cache can noticeably improve the performance when using +large images with frequent I/O. + +Previously, unless 'cache-size' was specified and was large enough, the +L2 cache was set to a certain size without taking the virtual image size +into account. + +Now, the L2 cache assignment is aware of the virtual size of the image, +and will cover the entire image, unless the cache size needed for that is +larger than a certain maximum. This maximum is set to 1 MB by default +(enough to cover an 8 GB image with the default cluster size) but can +be increased or decreased using the 'l2-cache-size' option. This option +was previously documented as the *maximum* L2 cache size, and this patch +makes it behave as such, instead of as a constant size. Also, the +existing option 'cache-size' can limit the sum of both L2 and refcount +caches, as previously. + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit b749562d9822d14ef69c9eaa5f85903010b86c30) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 21 +++++++++------------ + block/qcow2.h | 4 +--- + docs/qcow2-cache.txt | 15 ++++++++++----- + qemu-options.hx | 6 +++--- + tests/qemu-iotests/137 | 8 +++++++- + tests/qemu-iotests/137.out | 4 +++- + 6 files changed, 33 insertions(+), 25 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index f3b2860..fc6bddd 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -773,29 +773,35 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + uint64_t *refcount_cache_size, Error **errp) + { + BDRVQcow2State *s = bs->opaque; +- uint64_t combined_cache_size; ++ uint64_t combined_cache_size, l2_cache_max_setting; + bool l2_cache_size_set, refcount_cache_size_set, combined_cache_size_set; + int min_refcount_cache = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; ++ uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; ++ uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8); + + combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE); + l2_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_L2_CACHE_SIZE); + refcount_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_REFCOUNT_CACHE_SIZE); + + combined_cache_size = qemu_opt_get_size(opts, QCOW2_OPT_CACHE_SIZE, 0); +- *l2_cache_size = qemu_opt_get_size(opts, QCOW2_OPT_L2_CACHE_SIZE, 0); ++ l2_cache_max_setting = qemu_opt_get_size(opts, QCOW2_OPT_L2_CACHE_SIZE, ++ DEFAULT_L2_CACHE_MAX_SIZE); + *refcount_cache_size = qemu_opt_get_size(opts, + QCOW2_OPT_REFCOUNT_CACHE_SIZE, 0); + + *l2_cache_entry_size = qemu_opt_get_size( + opts, QCOW2_OPT_L2_CACHE_ENTRY_SIZE, s->cluster_size); + ++ *l2_cache_size = MIN(max_l2_cache, l2_cache_max_setting); ++ + if (combined_cache_size_set) { + if (l2_cache_size_set && refcount_cache_size_set) { + error_setg(errp, QCOW2_OPT_CACHE_SIZE ", " QCOW2_OPT_L2_CACHE_SIZE + " and " QCOW2_OPT_REFCOUNT_CACHE_SIZE " may not be set " + "the same time"); + return; +- } else if (*l2_cache_size > combined_cache_size) { ++ } else if (l2_cache_size_set && ++ (l2_cache_max_setting > combined_cache_size)) { + error_setg(errp, QCOW2_OPT_L2_CACHE_SIZE " may not exceed " + QCOW2_OPT_CACHE_SIZE); + return; +@@ -810,9 +816,6 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + } else if (refcount_cache_size_set) { + *l2_cache_size = combined_cache_size - *refcount_cache_size; + } else { +- uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; +- uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8); +- + /* Assign as much memory as possible to the L2 cache, and + * use the remainder for the refcount cache */ + if (combined_cache_size >= max_l2_cache + min_refcount_cache) { +@@ -824,12 +827,6 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + *l2_cache_size = combined_cache_size - *refcount_cache_size; + } + } +- } else { +- if (!l2_cache_size_set) { +- *l2_cache_size = MAX(DEFAULT_L2_CACHE_SIZE, +- (uint64_t)DEFAULT_L2_CACHE_CLUSTERS +- * s->cluster_size); +- } + } + /* l2_cache_size and refcount_cache_size are ensured to have at least + * their minimum values in qcow2_update_options_prepare() */ +diff --git a/block/qcow2.h b/block/qcow2.h +index f73a48a..d0dd4a2 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -74,9 +74,7 @@ + /* Must be at least 4 to cover all cases of refcount table growth */ + #define MIN_REFCOUNT_CACHE_SIZE 4 /* clusters */ + +-/* Whichever is more */ +-#define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */ +-#define DEFAULT_L2_CACHE_SIZE S_1MiB ++#define DEFAULT_L2_CACHE_MAX_SIZE S_1MiB + + #define DEFAULT_CLUSTER_SIZE S_64KiB + +diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt +index 7e28b41..750447e 100644 +--- a/docs/qcow2-cache.txt ++++ b/docs/qcow2-cache.txt +@@ -125,8 +125,12 @@ There are a few things that need to be taken into account: + - Both caches must have a size that is a multiple of the cluster size + (or the cache entry size: see "Using smaller cache sizes" below). + +- - The default L2 cache size is 8 clusters or 1MB (whichever is more), +- and the minimum is 2 clusters (or 2 cache entries, see below). ++ - The maximum L2 cache size is 1 MB by default (enough for full coverage ++ of 8 GB images, with the default cluster size). This value can be ++ modified using the "l2-cache-size" option. QEMU will not use more memory ++ than needed to hold all of the image's L2 tables, regardless of this max. ++ value. The minimal L2 cache size is 2 clusters (or 2 cache entries, see ++ below). + + - The default (and minimum) refcount cache size is 4 clusters. + +@@ -184,9 +188,10 @@ Some things to take into account: + always uses the cluster size as the entry size. + + - If the L2 cache is big enough to hold all of the image's L2 tables +- (as explained in the "Choosing the right cache sizes" section +- earlier in this document) then none of this is necessary and you +- can omit the "l2-cache-entry-size" parameter altogether. ++ (as explained in the "Choosing the right cache sizes" and "How to ++ configure the cache sizes" sections in this document) then none of ++ this is necessary and you can omit the "l2-cache-entry-size" ++ parameter altogether. + + + Reducing the memory usage +diff --git a/qemu-options.hx b/qemu-options.hx +index 5e15d6f..5e3bd74 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -756,9 +756,9 @@ The maximum total size of the L2 table and refcount block caches in bytes + + @item l2-cache-size + The maximum size of the L2 table cache in bytes +-(default: if cache-size is not defined - 1048576 bytes or 8 clusters, whichever +-is larger; otherwise, as large as possible or needed within the cache-size, +-while permitting the requested or the minimal refcount cache size) ++(default: if cache-size is not specified - 1M; otherwise, as large as possible ++within the cache-size, while permitting the requested or the minimal refcount ++cache size) + + @item refcount-cache-size + The maximum size of the refcount block cache in bytes +diff --git a/tests/qemu-iotests/137 b/tests/qemu-iotests/137 +index 8796562..19e8597 100755 +--- a/tests/qemu-iotests/137 ++++ b/tests/qemu-iotests/137 +@@ -109,7 +109,6 @@ $QEMU_IO \ + -c "reopen -o cache-size=1M,l2-cache-size=64k,refcount-cache-size=64k" \ + -c "reopen -o cache-size=1M,l2-cache-size=2M" \ + -c "reopen -o cache-size=1M,refcount-cache-size=2M" \ +- -c "reopen -o l2-cache-size=256T" \ + -c "reopen -o l2-cache-entry-size=33k" \ + -c "reopen -o l2-cache-entry-size=128k" \ + -c "reopen -o refcount-cache-size=256T" \ +@@ -119,6 +118,13 @@ $QEMU_IO \ + -c "reopen -o cache-clean-interval=-1" \ + "$TEST_IMG" | _filter_qemu_io + ++IMGOPTS="cluster_size=256k" _make_test_img 32P ++$QEMU_IO \ ++ -c "reopen -o l2-cache-entry-size=512,l2-cache-size=1T" \ ++ "$TEST_IMG" | _filter_qemu_io ++ ++_make_test_img 64M ++ + echo + echo === Test transaction semantics === + echo +diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out +index 96724a6..afcc000 100644 +--- a/tests/qemu-iotests/137.out ++++ b/tests/qemu-iotests/137.out +@@ -19,7 +19,6 @@ Parameter 'lazy-refcounts' expects 'on' or 'off' + cache-size, l2-cache-size and refcount-cache-size may not be set the same time + l2-cache-size may not exceed cache-size + refcount-cache-size may not exceed cache-size +-L2 cache size too big + L2 cache entry size must be a power of two between 512 and the cluster size (65536) + L2 cache entry size must be a power of two between 512 and the cluster size (65536) + Refcount cache size too big +@@ -27,6 +26,9 @@ Conflicting values for qcow2 options 'overlap-check' ('constant') and 'overlap-c + Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of the following: none, constant, cached, all + Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of the following: none, constant, cached, all + Cache clean interval too big ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=36028797018963968 ++L2 cache size too big ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 + + === Test transaction semantics === + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Avoid-duplication-in-setting-the-refcount-cach.patch b/SOURCES/kvm-qcow2-Avoid-duplication-in-setting-the-refcount-cach.patch new file mode 100644 index 0000000..8c57ef6 --- /dev/null +++ b/SOURCES/kvm-qcow2-Avoid-duplication-in-setting-the-refcount-cach.patch @@ -0,0 +1,52 @@ +From 3f62574388cc48d3e2e2361e6e32ae1be60fbf42 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:33 +0000 +Subject: [PATCH 08/15] qcow2: Avoid duplication in setting the refcount cache + size + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-9-kwolf@redhat.com> +Patchwork-id: 83296 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 08/15] qcow2: Avoid duplication in setting the refcount cache size +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +The refcount cache size does not need to be set to its minimum value in +read_cache_sizes(), as it is set to at least its minimum value in +qcow2_update_options_prepare(). + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 657ada52abb85140e56949f522ecec527b256450) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 5 ++--- + 1 file changed, 2 insertions(+), 3 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 3859112..f3b2860 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -830,10 +830,9 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + (uint64_t)DEFAULT_L2_CACHE_CLUSTERS + * s->cluster_size); + } +- if (!refcount_cache_size_set) { +- *refcount_cache_size = min_refcount_cache; +- } + } ++ /* l2_cache_size and refcount_cache_size are ensured to have at least ++ * their minimum values in qcow2_update_options_prepare() */ + + if (*l2_cache_entry_size < (1 << MIN_CLUSTER_BITS) || + *l2_cache_entry_size > s->cluster_size || +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Do-not-mark-inactive-images-corrupt.patch b/SOURCES/kvm-qcow2-Do-not-mark-inactive-images-corrupt.patch new file mode 100644 index 0000000..a301393 --- /dev/null +++ b/SOURCES/kvm-qcow2-Do-not-mark-inactive-images-corrupt.patch @@ -0,0 +1,57 @@ +From 425f93585b5a8f45198e91acce68bc0e921c0bf5 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 18:00:54 +0200 +Subject: [PATCH 057/268] qcow2: Do not mark inactive images corrupt + +RH-Author: Max Reitz +Message-id: <20180618180055.22739-3-mreitz@redhat.com> +Patchwork-id: 80793 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/3] qcow2: Do not mark inactive images corrupt +Bugzilla: 1588039 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf +RH-Acked-by: John Snow + +When signaling a corruption on a read-only image, qcow2 already makes +fatal events non-fatal (i.e., they will not result in the image being +closed, and the image header's corrupt flag will not be set). This is +necessary because we cannot set the corrupt flag on read-only images, +and it is possible because further corruption of read-only images is +impossible. + +Inactive images are effectively read-only, too, so we should do the same +for them. bdrv_is_writable() can tell us whether an image can actually +be written to, so use its result instead of !bs->read_only. + +(Otherwise, the assert(!(bs->open_flags & BDRV_O_INACTIVE)) in +bdrv_co_pwritev() will fail, crashing qemu.) + +Cc: qemu-stable@nongnu.org +Signed-off-by: Max Reitz +Message-id: 20180606193702.7113-3-mreitz@redhat.com +Reviewed-by: John Snow +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit ddf3b47ef4b5ed0bf6558d4c2c8ae130b8d8a580) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/qcow2.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 35842c5..26a6a7f 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -4383,7 +4383,7 @@ void qcow2_signal_corruption(BlockDriverState *bs, bool fatal, int64_t offset, + char *message; + va_list ap; + +- fatal = fatal && !bs->read_only; ++ fatal = fatal && bdrv_is_writable(bs); + + if (s->signaled_corruption && + (!fatal || (s->incompatible_features & QCOW2_INCOMPAT_CORRUPT))) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Explicit-number-replaced-by-a-constant.patch b/SOURCES/kvm-qcow2-Explicit-number-replaced-by-a-constant.patch new file mode 100644 index 0000000..1748d3b --- /dev/null +++ b/SOURCES/kvm-qcow2-Explicit-number-replaced-by-a-constant.patch @@ -0,0 +1,52 @@ +From e0ee7e18ae8fb3430202de49faa34091359ef675 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:38 +0000 +Subject: [PATCH 13/15] qcow2: Explicit number replaced by a constant + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-14-kwolf@redhat.com> +Patchwork-id: 83292 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 13/15] qcow2: Explicit number replaced by a constant +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit bd016b912cc68c6f6c68cd5acb2e13126bd9e05c) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index acd076c..114dcdd 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1321,7 +1321,7 @@ static int coroutine_fn qcow2_do_open(BlockDriverState *bs, QDict *options, + /* 2^(s->refcount_order - 3) is the refcount width in bytes */ + s->refcount_block_bits = s->cluster_bits - (s->refcount_order - 3); + s->refcount_block_size = 1 << s->refcount_block_bits; +- bs->total_sectors = header.size / 512; ++ bs->total_sectors = header.size / BDRV_SECTOR_SIZE; + s->csize_shift = (62 - (s->cluster_bits - 8)); + s->csize_mask = (1 << (s->cluster_bits - 8)) - 1; + s->cluster_offset_mask = (1LL << s->csize_shift) - 1; +@@ -3494,7 +3494,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, + goto fail; + } + +- old_length = bs->total_sectors * 512; ++ old_length = bs->total_sectors * BDRV_SECTOR_SIZE; + new_l1_size = size_to_l1(s, offset); + + if (offset < old_length) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Fix-Coverity-warning-when-calculating-the-refc.patch b/SOURCES/kvm-qcow2-Fix-Coverity-warning-when-calculating-the-refc.patch new file mode 100644 index 0000000..12b446f --- /dev/null +++ b/SOURCES/kvm-qcow2-Fix-Coverity-warning-when-calculating-the-refc.patch @@ -0,0 +1,75 @@ +From c0d6740c1219f23f0b9a996f02c1ae9cf824b0b3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:28 +0000 +Subject: [PATCH 03/15] qcow2: Fix Coverity warning when calculating the + refcount cache size + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-4-kwolf@redhat.com> +Patchwork-id: 83295 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 03/15] qcow2: Fix Coverity warning when calculating the refcount cache size +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Alberto Garcia + +MIN_REFCOUNT_CACHE_SIZE is 4 and the cluster size is guaranteed to be +at most 2MB, so the minimum refcount cache size (in bytes) is always +going to fit in a 32-bit integer. + +Coverity doesn't know that, and since we're storing the result in a +uint64_t (*refcount_cache_size) it thinks that we need the 64 bits and +that we probably want to do a 64-bit multiplication to prevent the +result from being truncated. + +This is a false positive in this case, but it's a fair warning. +We could do a 64-bit multiplication to get rid of it, but since we +know that a 32-bit variable is enough to store this value let's simply +reuse min_refcount_cache, make it a normal int and stop doing casts. + +Reported-by: Peter Maydell +Signed-off-by: Alberto Garcia +Reviewed-by: Eric Blake +Signed-off-by: Kevin Wolf +(cherry picked from commit 7af5eea9b34ffb7a9a9fc25ba71998a02b76e159) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 5 ++--- + 1 file changed, 2 insertions(+), 3 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 4b65e4c..a0f7234 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -775,6 +775,7 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + BDRVQcow2State *s = bs->opaque; + uint64_t combined_cache_size; + bool l2_cache_size_set, refcount_cache_size_set, combined_cache_size_set; ++ int min_refcount_cache = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; + + combined_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_CACHE_SIZE); + l2_cache_size_set = qemu_opt_get(opts, QCOW2_OPT_L2_CACHE_SIZE); +@@ -811,8 +812,6 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + } else { + uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; + uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8); +- uint64_t min_refcount_cache = +- (uint64_t) MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; + + /* Assign as much memory as possible to the L2 cache, and + * use the remainder for the refcount cache */ +@@ -832,7 +831,7 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + * s->cluster_size); + } + if (!refcount_cache_size_set) { +- *refcount_cache_size = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; ++ *refcount_cache_size = min_refcount_cache; + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Fix-cache-clean-interval-documentation.patch b/SOURCES/kvm-qcow2-Fix-cache-clean-interval-documentation.patch new file mode 100644 index 0000000..03bb964 --- /dev/null +++ b/SOURCES/kvm-qcow2-Fix-cache-clean-interval-documentation.patch @@ -0,0 +1,95 @@ +From 7fa1062d4c4f6581bbe53a68f4cdd78558bcf5e3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:40 +0000 +Subject: [PATCH 15/15] qcow2: Fix cache-clean-interval documentation + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-16-kwolf@redhat.com> +Patchwork-id: 83298 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 15/15] qcow2: Fix cache-clean-interval documentation +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +Fixing cache-clean-interval documentation following the recent change to +a default of 600 seconds on supported plarforms (only Linux currently). + +Signed-off-by: Leonid Bloch +Reviewed-by: Eric Blake +Signed-off-by: Kevin Wolf +(cherry picked from commit e3a7b4556ee33feba2b396769a9c8354be06b024) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + docs/qcow2-cache.txt | 20 ++++++++++---------- + qapi/block-core.json | 3 ++- + qemu-options.hx | 3 ++- + 3 files changed, 14 insertions(+), 12 deletions(-) + +diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt +index 59358b8..c459bf5 100644 +--- a/docs/qcow2-cache.txt ++++ b/docs/qcow2-cache.txt +@@ -202,18 +202,18 @@ Reducing the memory usage + It is possible to clean unused cache entries in order to reduce the + memory usage during periods of low I/O activity. + +-The parameter "cache-clean-interval" defines an interval (in seconds). +-All cache entries that haven't been accessed during that interval are +-removed from memory. ++The parameter "cache-clean-interval" defines an interval (in seconds), ++after which all the cache entries that haven't been accessed during the ++interval are removed from memory. Setting this parameter to 0 disables this ++feature. + +-This example removes all unused cache entries every 15 minutes: ++The following example removes all unused cache entries every 15 minutes: + + -drive file=hd.qcow2,cache-clean-interval=900 + +-If unset, the default value for this parameter is 600. Setting it to 0 +-disables this feature. ++If unset, the default value for this parameter is 600 on platforms which ++support this functionality, and is 0 (disabled) on other platforms. + +-Note that this functionality currently relies on the MADV_DONTNEED +-argument for madvise() to actually free the memory. This is a +-Linux-specific feature, so cache-clean-interval is not supported in +-other systems. ++This functionality currently relies on the MADV_DONTNEED argument for ++madvise() to actually free the memory. This is a Linux-specific feature, ++so cache-clean-interval is not supported on other systems. +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 5318c9b..db47fb8 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -2867,7 +2867,8 @@ + # + # @cache-clean-interval: clean unused entries in the L2 and refcount + # caches. The interval is in seconds. The default value +-# is 600, and 0 disables this feature. (since 2.5) ++# is 600 on supporting platforms, and 0 on other ++# platforms. 0 disables this feature. (since 2.5) + # + # @encrypt: Image decryption options. Mandatory for + # encrypted images, except when doing a metadata-only +diff --git a/qemu-options.hx b/qemu-options.hx +index 05fabf3..683ab0d 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -767,7 +767,8 @@ it which is not used for the L2 cache) + + @item cache-clean-interval + Clean unused entries in the L2 and refcount caches. The interval is in seconds. +-The default value is 600. Setting it to 0 disables this feature. ++The default value is 600 on supporting platforms, and 0 on other platforms. ++Setting it to 0 disables this feature. + + @item pass-discard-request + Whether discard requests to the qcow2 device should be forwarded to the data +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Fix-qcow2_truncate-error-return-value.patch b/SOURCES/kvm-qcow2-Fix-qcow2_truncate-error-return-value.patch new file mode 100644 index 0000000..1e0a03b --- /dev/null +++ b/SOURCES/kvm-qcow2-Fix-qcow2_truncate-error-return-value.patch @@ -0,0 +1,44 @@ +From f0b9cea2fd575e86a3d2d2c8ca7b7f6b65cd7b08 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 14:42:53 +0200 +Subject: [PATCH 208/268] qcow2: Fix qcow2_truncate() error return value + +RH-Author: Kevin Wolf +Message-id: <20180712144258.17303-2-kwolf@redhat.com> +Patchwork-id: 81325 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/6] qcow2: Fix qcow2_truncate() error return value +Bugzilla: 1595173 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: John Snow + +If qcow2_alloc_clusters_at() returns an error, we do need to negate it +to get back the positive errno code for error_setg_errno(), but we still +need to return the negative error code. + +Fixes: 772d1f973f87269f6a4a4ea4b880680f3779bbdf +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit ae5475e82fd1ebb24f4f77cf28f59ca6548c6136) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/qcow2.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index da74e2a..dbd448c 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3594,7 +3594,7 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + if (clusters_allocated < 0) { + error_setg_errno(errp, -clusters_allocated, + "Failed to allocate data clusters"); +- return -clusters_allocated; ++ return clusters_allocated; + } + + assert(clusters_allocated == nb_new_data_clusters); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Fix-src_offset-in-copy-offloading.patch b/SOURCES/kvm-qcow2-Fix-src_offset-in-copy-offloading.patch new file mode 100644 index 0000000..1a8021f --- /dev/null +++ b/SOURCES/kvm-qcow2-Fix-src_offset-in-copy-offloading.patch @@ -0,0 +1,82 @@ +From a36dfa1ce0b98a3144be892ef6ada29deaf9dcf8 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:51 +0200 +Subject: [PATCH 177/268] qcow2: Fix src_offset in copy offloading + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-12-famz@redhat.com> +Patchwork-id: 81163 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 11/13] qcow2: Fix src_offset in copy offloading +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Not updating src_offset will result in wrong data being written to dst +image. + +Reported-by: Max Reitz +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/qcow2.c | 1 + + tests/qemu-iotests/063 | 9 +++++++++ + tests/qemu-iotests/063.out | 12 ++++++++++++ + 3 files changed, 22 insertions(+) + +diff --git a/block/qcow2.c b/block/qcow2.c +index c85ebcb..4404dc7 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3416,6 +3416,7 @@ qcow2_co_copy_range_to(BlockDriverState *bs, + } + + bytes -= cur_bytes; ++ src_offset += cur_bytes; + dst_offset += cur_bytes; + } + ret = 0; +diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063 +index e4f6ea9..adc037c 100755 +--- a/tests/qemu-iotests/063 ++++ b/tests/qemu-iotests/063 +@@ -91,6 +91,15 @@ if $QEMU_IMG convert -f $IMGFMT -O $IMGFMT -n "$TEST_IMG.orig" "$TEST_IMG" >/dev + exit 1 + fi + ++echo "== Regression testing for copy offloading bug ==" ++ ++_make_test_img 1M ++TEST_IMG="$TEST_IMG.target" _make_test_img 1M ++$QEMU_IO -c 'write -P 1 0 512k' -c 'write -P 2 512k 512k' "$TEST_IMG" | _filter_qemu_io ++$QEMU_IO -c 'write -P 4 512k 512k' -c 'write -P 3 0 512k' "$TEST_IMG.target" | _filter_qemu_io ++$QEMU_IMG convert -n -O $IMGFMT "$TEST_IMG" "$TEST_IMG.target" ++$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG.target" ++ + echo "*** done" + rm -f $seq.full + status=0 +diff --git a/tests/qemu-iotests/063.out b/tests/qemu-iotests/063.out +index de1c99a..7b691b2 100644 +--- a/tests/qemu-iotests/063.out ++++ b/tests/qemu-iotests/063.out +@@ -7,4 +7,16 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 + No errors were found on the image. + == Testing conversion to a smaller file fails == + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2097152 ++== Regression testing for copy offloading bug == ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576 ++Formatting 'TEST_DIR/t.IMGFMT.target', fmt=IMGFMT size=1048576 ++wrote 524288/524288 bytes at offset 0 ++512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 524288/524288 bytes at offset 524288 ++512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 524288/524288 bytes at offset 524288 ++512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++wrote 524288/524288 bytes at offset 0 ++512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) ++Images are identical. + *** done +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Free-allocated-clusters-on-write-error.patch b/SOURCES/kvm-qcow2-Free-allocated-clusters-on-write-error.patch new file mode 100644 index 0000000..801aaea --- /dev/null +++ b/SOURCES/kvm-qcow2-Free-allocated-clusters-on-write-error.patch @@ -0,0 +1,83 @@ +From 175dbc6a2217f7a8cf319d8da487fa80ff68eba4 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Mon, 2 Jul 2018 15:40:07 +0200 +Subject: [PATCH 183/268] qcow2: Free allocated clusters on write error + +RH-Author: Kevin Wolf +Message-id: <20180702154008.15533-3-kwolf@redhat.com> +Patchwork-id: 81185 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/3] qcow2: Free allocated clusters on write error +Bugzilla: 1528541 +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng +RH-Acked-by: Stefan Hajnoczi + +If we managed to allocate the clusters, but then failed to write the +data, there's a good chance that we'll still be able to free the +clusters again in order to avoid cluster leaks (the refcounts are +cached, so even if we can't write them out right now, we may be able to +do so when the VM is resumed after a werror=stop/enospc pause). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Eric Blake +Tested-by: Eric Blake +(cherry picked from commit 8b24cd141549b5b264baeddd4e72902cfb5de23b) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/qcow2-cluster.c | 11 +++++++++++ + block/qcow2.c | 2 ++ + block/qcow2.h | 1 + + 3 files changed, 14 insertions(+) + +diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c +index 1aee726..c90e2ec 100644 +--- a/block/qcow2-cluster.c ++++ b/block/qcow2-cluster.c +@@ -994,6 +994,17 @@ err: + return ret; + } + ++/** ++ * Frees the allocated clusters because the request failed and they won't ++ * actually be linked. ++ */ ++void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m) ++{ ++ BDRVQcow2State *s = bs->opaque; ++ qcow2_free_clusters(bs, m->alloc_offset, m->nb_clusters << s->cluster_bits, ++ QCOW2_DISCARD_NEVER); ++} ++ + /* + * Returns the number of contiguous clusters that can be used for an allocating + * write, but require COW to be performed (this includes yet unallocated space, +diff --git a/block/qcow2.c b/block/qcow2.c +index 4404dc7..da74e2a 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1771,6 +1771,8 @@ static coroutine_fn int qcow2_handle_l2meta(BlockDriverState *bs, + if (ret) { + goto out; + } ++ } else { ++ qcow2_alloc_cluster_abort(bs, l2meta); + } + + /* Take the request off the list of running requests */ +diff --git a/block/qcow2.h b/block/qcow2.h +index adf5c39..b5e2aa3 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -618,6 +618,7 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs, + int compressed_size); + + int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m); ++void qcow2_alloc_cluster_abort(BlockDriverState *bs, QCowL2Meta *m); + int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, enum qcow2_discard_type type, + bool full_discard); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Give-the-refcount-cache-the-minimum-possible-s.patch b/SOURCES/kvm-qcow2-Give-the-refcount-cache-the-minimum-possible-s.patch new file mode 100644 index 0000000..8d86cea --- /dev/null +++ b/SOURCES/kvm-qcow2-Give-the-refcount-cache-the-minimum-possible-s.patch @@ -0,0 +1,152 @@ +From f9faa15ed2a819c8fcf1eaf3534d7162f9cb8290 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:26 +0000 +Subject: [PATCH 01/15] qcow2: Give the refcount cache the minimum possible + size by default + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-2-kwolf@redhat.com> +Patchwork-id: 83284 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 01/15] qcow2: Give the refcount cache the minimum possible size by default +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Alberto Garcia + +The L2 and refcount caches have default sizes that can be overridden +using the l2-cache-size and refcount-cache-size (an additional +parameter named cache-size sets the combined size of both caches). + +Unless forced by one of the aforementioned parameters, QEMU will set +the unspecified sizes so that the L2 cache is 4 times larger than the +refcount cache. + +This is based on the premise that the refcount metadata needs to be +only a fourth of the L2 metadata to cover the same amount of disk +space. This is incorrect for two reasons: + + a) The amount of disk covered by an L2 table depends solely on the + cluster size, but in the case of a refcount block it depends on + the cluster size *and* the width of each refcount entry. + The 4/1 ratio is only valid with 16-bit entries (the default). + + b) When we talk about disk space and L2 tables we are talking about + guest space (L2 tables map guest clusters to host clusters), + whereas refcount blocks are used for host clusters (including + L1/L2 tables and the refcount blocks themselves). On a fully + populated (and uncompressed) qcow2 file, image size > virtual size + so there are more refcount entries than L2 entries. + +Problem (a) could be fixed by adjusting the algorithm to take into +account the refcount entry width. Problem (b) could be fixed by +increasing a bit the refcount cache size to account for the clusters +used for qcow2 metadata. + +However this patch takes a completely different approach and instead +of keeping a ratio between both cache sizes it assigns as much as +possible to the L2 cache and the remainder to the refcount cache. + +The reason is that L2 tables are used for every single I/O request +from the guest and the effect of increasing the cache is significant +and clearly measurable. Refcount blocks are however only used for +cluster allocation and internal snapshots and in practice are accessed +sequentially in most cases, so the effect of increasing the cache is +negligible (even when doing random writes from the guest). + +So, make the refcount cache as small as possible unless the user +explicitly asks for a larger one. + +Signed-off-by: Alberto Garcia +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +Message-id: 9695182c2eb11b77cb319689a1ebaa4e7c9d6591.1523968389.git.berto@igalia.com +Signed-off-by: Max Reitz +(cherry picked from commit 52253998ec3e523c9e20ae81e2a6431d8ff733ba) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 31 +++++++++++++++++++------------ + block/qcow2.h | 4 ---- + tests/qemu-iotests/137.out | 2 +- + 3 files changed, 20 insertions(+), 17 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 36d1152..4b65e4c 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -809,23 +809,30 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + } else if (refcount_cache_size_set) { + *l2_cache_size = combined_cache_size - *refcount_cache_size; + } else { +- *refcount_cache_size = combined_cache_size +- / (DEFAULT_L2_REFCOUNT_SIZE_RATIO + 1); +- *l2_cache_size = combined_cache_size - *refcount_cache_size; ++ uint64_t virtual_disk_size = bs->total_sectors * BDRV_SECTOR_SIZE; ++ uint64_t max_l2_cache = virtual_disk_size / (s->cluster_size / 8); ++ uint64_t min_refcount_cache = ++ (uint64_t) MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; ++ ++ /* Assign as much memory as possible to the L2 cache, and ++ * use the remainder for the refcount cache */ ++ if (combined_cache_size >= max_l2_cache + min_refcount_cache) { ++ *l2_cache_size = max_l2_cache; ++ *refcount_cache_size = combined_cache_size - *l2_cache_size; ++ } else { ++ *refcount_cache_size = ++ MIN(combined_cache_size, min_refcount_cache); ++ *l2_cache_size = combined_cache_size - *refcount_cache_size; ++ } + } + } else { +- if (!l2_cache_size_set && !refcount_cache_size_set) { ++ if (!l2_cache_size_set) { + *l2_cache_size = MAX(DEFAULT_L2_CACHE_BYTE_SIZE, + (uint64_t)DEFAULT_L2_CACHE_CLUSTERS + * s->cluster_size); +- *refcount_cache_size = *l2_cache_size +- / DEFAULT_L2_REFCOUNT_SIZE_RATIO; +- } else if (!l2_cache_size_set) { +- *l2_cache_size = *refcount_cache_size +- * DEFAULT_L2_REFCOUNT_SIZE_RATIO; +- } else if (!refcount_cache_size_set) { +- *refcount_cache_size = *l2_cache_size +- / DEFAULT_L2_REFCOUNT_SIZE_RATIO; ++ } ++ if (!refcount_cache_size_set) { ++ *refcount_cache_size = MIN_REFCOUNT_CACHE_SIZE * s->cluster_size; + } + } + +diff --git a/block/qcow2.h b/block/qcow2.h +index 43163b2..3d92cdb 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -77,10 +77,6 @@ + #define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */ + #define DEFAULT_L2_CACHE_BYTE_SIZE 1048576 /* bytes */ + +-/* The refblock cache needs only a fourth of the L2 cache size to cover as many +- * clusters */ +-#define DEFAULT_L2_REFCOUNT_SIZE_RATIO 4 +- + #define DEFAULT_CLUSTER_SIZE 65536 + + +diff --git a/tests/qemu-iotests/137.out b/tests/qemu-iotests/137.out +index e28e1ea..96724a6 100644 +--- a/tests/qemu-iotests/137.out ++++ b/tests/qemu-iotests/137.out +@@ -22,7 +22,7 @@ refcount-cache-size may not exceed cache-size + L2 cache size too big + L2 cache entry size must be a power of two between 512 and the cluster size (65536) + L2 cache entry size must be a power of two between 512 and the cluster size (65536) +-L2 cache size too big ++Refcount cache size too big + Conflicting values for qcow2 options 'overlap-check' ('constant') and 'overlap-check.template' ('all') + Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of the following: none, constant, cached, all + Unsupported value 'blubb' for qcow2 option 'overlap-check'. Allowed are any of the following: none, constant, cached, all +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Implement-copy-offloading.patch b/SOURCES/kvm-qcow2-Implement-copy-offloading.patch new file mode 100644 index 0000000..4a60c41 --- /dev/null +++ b/SOURCES/kvm-qcow2-Implement-copy-offloading.patch @@ -0,0 +1,301 @@ +From a0e14e757707733db8e3b927b6bcae336ae219d8 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:44 +0200 +Subject: [PATCH 170/268] qcow2: Implement copy offloading + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-5-famz@redhat.com> +Patchwork-id: 81154 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 04/13] qcow2: Implement copy offloading +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +The two callbacks are implemented quite similarly to the read/write +functions: bdrv_co_copy_range_from maps for read and calls into bs->file +or bs->backing depending on the allocation status; bdrv_co_copy_range_to +maps for write and calls into bs->file. + +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Fam Zheng +Message-id: 20180601092648.24614-5-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit fd9fcd37a8645efe322956d94f76e90135522a16) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/qcow2.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++++-------- + 1 file changed, 199 insertions(+), 30 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 092db81..c85ebcb 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1756,6 +1756,39 @@ static int coroutine_fn qcow2_co_block_status(BlockDriverState *bs, + return status; + } + ++static coroutine_fn int qcow2_handle_l2meta(BlockDriverState *bs, ++ QCowL2Meta **pl2meta, ++ bool link_l2) ++{ ++ int ret = 0; ++ QCowL2Meta *l2meta = *pl2meta; ++ ++ while (l2meta != NULL) { ++ QCowL2Meta *next; ++ ++ if (!ret && link_l2) { ++ ret = qcow2_alloc_cluster_link_l2(bs, l2meta); ++ if (ret) { ++ goto out; ++ } ++ } ++ ++ /* Take the request off the list of running requests */ ++ if (l2meta->nb_clusters != 0) { ++ QLIST_REMOVE(l2meta, next_in_flight); ++ } ++ ++ qemu_co_queue_restart_all(&l2meta->dependent_requests); ++ ++ next = l2meta->next; ++ g_free(l2meta); ++ l2meta = next; ++ } ++out: ++ *pl2meta = l2meta; ++ return ret; ++} ++ + static coroutine_fn int qcow2_co_preadv(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, + int flags) +@@ -2042,24 +2075,9 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, + } + } + +- while (l2meta != NULL) { +- QCowL2Meta *next; +- +- ret = qcow2_alloc_cluster_link_l2(bs, l2meta); +- if (ret < 0) { +- goto fail; +- } +- +- /* Take the request off the list of running requests */ +- if (l2meta->nb_clusters != 0) { +- QLIST_REMOVE(l2meta, next_in_flight); +- } +- +- qemu_co_queue_restart_all(&l2meta->dependent_requests); +- +- next = l2meta->next; +- g_free(l2meta); +- l2meta = next; ++ ret = qcow2_handle_l2meta(bs, &l2meta, true); ++ if (ret) { ++ goto fail; + } + + bytes -= cur_bytes; +@@ -2070,18 +2088,7 @@ static coroutine_fn int qcow2_co_pwritev(BlockDriverState *bs, uint64_t offset, + ret = 0; + + fail: +- while (l2meta != NULL) { +- QCowL2Meta *next; +- +- if (l2meta->nb_clusters != 0) { +- QLIST_REMOVE(l2meta, next_in_flight); +- } +- qemu_co_queue_restart_all(&l2meta->dependent_requests); +- +- next = l2meta->next; +- g_free(l2meta); +- l2meta = next; +- } ++ qcow2_handle_l2meta(bs, &l2meta, false); + + qemu_co_mutex_unlock(&s->lock); + +@@ -3264,6 +3271,166 @@ static coroutine_fn int qcow2_co_pdiscard(BlockDriverState *bs, + return ret; + } + ++static int coroutine_fn ++qcow2_co_copy_range_from(BlockDriverState *bs, ++ BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ BDRVQcow2State *s = bs->opaque; ++ int ret; ++ unsigned int cur_bytes; /* number of bytes in current iteration */ ++ BdrvChild *child = NULL; ++ BdrvRequestFlags cur_flags; ++ ++ assert(!bs->encrypted); ++ qemu_co_mutex_lock(&s->lock); ++ ++ while (bytes != 0) { ++ uint64_t copy_offset = 0; ++ /* prepare next request */ ++ cur_bytes = MIN(bytes, INT_MAX); ++ cur_flags = flags; ++ ++ ret = qcow2_get_cluster_offset(bs, src_offset, &cur_bytes, ©_offset); ++ if (ret < 0) { ++ goto out; ++ } ++ ++ switch (ret) { ++ case QCOW2_CLUSTER_UNALLOCATED: ++ if (bs->backing && bs->backing->bs) { ++ int64_t backing_length = bdrv_getlength(bs->backing->bs); ++ if (src_offset >= backing_length) { ++ cur_flags |= BDRV_REQ_ZERO_WRITE; ++ } else { ++ child = bs->backing; ++ cur_bytes = MIN(cur_bytes, backing_length - src_offset); ++ copy_offset = src_offset; ++ } ++ } else { ++ cur_flags |= BDRV_REQ_ZERO_WRITE; ++ } ++ break; ++ ++ case QCOW2_CLUSTER_ZERO_PLAIN: ++ case QCOW2_CLUSTER_ZERO_ALLOC: ++ cur_flags |= BDRV_REQ_ZERO_WRITE; ++ break; ++ ++ case QCOW2_CLUSTER_COMPRESSED: ++ ret = -ENOTSUP; ++ goto out; ++ break; ++ ++ case QCOW2_CLUSTER_NORMAL: ++ child = bs->file; ++ copy_offset += offset_into_cluster(s, src_offset); ++ if ((copy_offset & 511) != 0) { ++ ret = -EIO; ++ goto out; ++ } ++ break; ++ ++ default: ++ abort(); ++ } ++ qemu_co_mutex_unlock(&s->lock); ++ ret = bdrv_co_copy_range_from(child, ++ copy_offset, ++ dst, dst_offset, ++ cur_bytes, cur_flags); ++ qemu_co_mutex_lock(&s->lock); ++ if (ret < 0) { ++ goto out; ++ } ++ ++ bytes -= cur_bytes; ++ src_offset += cur_bytes; ++ dst_offset += cur_bytes; ++ } ++ ret = 0; ++ ++out: ++ qemu_co_mutex_unlock(&s->lock); ++ return ret; ++} ++ ++static int coroutine_fn ++qcow2_co_copy_range_to(BlockDriverState *bs, ++ BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ BDRVQcow2State *s = bs->opaque; ++ int offset_in_cluster; ++ int ret; ++ unsigned int cur_bytes; /* number of sectors in current iteration */ ++ uint64_t cluster_offset; ++ uint8_t *cluster_data = NULL; ++ QCowL2Meta *l2meta = NULL; ++ ++ assert(!bs->encrypted); ++ s->cluster_cache_offset = -1; /* disable compressed cache */ ++ ++ qemu_co_mutex_lock(&s->lock); ++ ++ while (bytes != 0) { ++ ++ l2meta = NULL; ++ ++ offset_in_cluster = offset_into_cluster(s, dst_offset); ++ cur_bytes = MIN(bytes, INT_MAX); ++ ++ /* TODO: ++ * If src->bs == dst->bs, we could simply copy by incrementing ++ * the refcnt, without copying user data. ++ * Or if src->bs == dst->bs->backing->bs, we could copy by discarding. */ ++ ret = qcow2_alloc_cluster_offset(bs, dst_offset, &cur_bytes, ++ &cluster_offset, &l2meta); ++ if (ret < 0) { ++ goto fail; ++ } ++ ++ assert((cluster_offset & 511) == 0); ++ ++ ret = qcow2_pre_write_overlap_check(bs, 0, ++ cluster_offset + offset_in_cluster, cur_bytes); ++ if (ret < 0) { ++ goto fail; ++ } ++ ++ qemu_co_mutex_unlock(&s->lock); ++ ret = bdrv_co_copy_range_to(src, src_offset, ++ bs->file, ++ cluster_offset + offset_in_cluster, ++ cur_bytes, flags); ++ qemu_co_mutex_lock(&s->lock); ++ if (ret < 0) { ++ goto fail; ++ } ++ ++ ret = qcow2_handle_l2meta(bs, &l2meta, true); ++ if (ret) { ++ goto fail; ++ } ++ ++ bytes -= cur_bytes; ++ dst_offset += cur_bytes; ++ } ++ ret = 0; ++ ++fail: ++ qcow2_handle_l2meta(bs, &l2meta, false); ++ ++ qemu_co_mutex_unlock(&s->lock); ++ ++ qemu_vfree(cluster_data); ++ trace_qcow2_writev_done_req(qemu_coroutine_self(), ret); ++ ++ return ret; ++} ++ + static int qcow2_truncate(BlockDriverState *bs, int64_t offset, + PreallocMode prealloc, Error **errp) + { +@@ -4522,6 +4689,8 @@ BlockDriver bdrv_qcow2 = { + + .bdrv_co_pwrite_zeroes = qcow2_co_pwrite_zeroes, + .bdrv_co_pdiscard = qcow2_co_pdiscard, ++ .bdrv_co_copy_range_from = qcow2_co_copy_range_from, ++ .bdrv_co_copy_range_to = qcow2_co_copy_range_to, + .bdrv_truncate = qcow2_truncate, + .bdrv_co_pwritev_compressed = qcow2_co_pwritev_compressed, + .bdrv_make_empty = qcow2_make_empty, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Increase-the-default-upper-limit-on-the-L2-cac.patch b/SOURCES/kvm-qcow2-Increase-the-default-upper-limit-on-the-L2-cac.patch new file mode 100644 index 0000000..7ad1714 --- /dev/null +++ b/SOURCES/kvm-qcow2-Increase-the-default-upper-limit-on-the-L2-cac.patch @@ -0,0 +1,104 @@ +From 64f611dddac96852363e4638954e0722a28420e6 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:35 +0000 +Subject: [PATCH 10/15] qcow2: Increase the default upper limit on the L2 cache + size + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-11-kwolf@redhat.com> +Patchwork-id: 83290 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 10/15] qcow2: Increase the default upper limit on the L2 cache size +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +The upper limit on the L2 cache size is increased from 1 MB to 32 MB +on Linux platforms, and to 8 MB on other platforms (this difference is +caused by the ability to set intervals for cache cleaning on Linux +platforms only). + +This is done in order to allow default full coverage with the L2 cache +for images of up to 256 GB in size (was 8 GB). Note, that only the +needed amount to cover the full image is allocated. The value which is +changed here is just the upper limit on the L2 cache size, beyond which +it will not grow, even if the size of the image will require it to. + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 80668d0fb735f0839a46278a7d42116089b82816) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.h | 6 +++++- + docs/qcow2-cache.txt | 15 +++++++++------ + qemu-options.hx | 6 +++--- + 3 files changed, 17 insertions(+), 10 deletions(-) + +diff --git a/block/qcow2.h b/block/qcow2.h +index d0dd4a2..6c6c742 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -74,7 +74,11 @@ + /* Must be at least 4 to cover all cases of refcount table growth */ + #define MIN_REFCOUNT_CACHE_SIZE 4 /* clusters */ + +-#define DEFAULT_L2_CACHE_MAX_SIZE S_1MiB ++#ifdef CONFIG_LINUX ++#define DEFAULT_L2_CACHE_MAX_SIZE S_32MiB ++#else ++#define DEFAULT_L2_CACHE_MAX_SIZE S_8MiB ++#endif + + #define DEFAULT_CLUSTER_SIZE S_64KiB + +diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt +index 750447e..1fcc065 100644 +--- a/docs/qcow2-cache.txt ++++ b/docs/qcow2-cache.txt +@@ -125,12 +125,15 @@ There are a few things that need to be taken into account: + - Both caches must have a size that is a multiple of the cluster size + (or the cache entry size: see "Using smaller cache sizes" below). + +- - The maximum L2 cache size is 1 MB by default (enough for full coverage +- of 8 GB images, with the default cluster size). This value can be +- modified using the "l2-cache-size" option. QEMU will not use more memory +- than needed to hold all of the image's L2 tables, regardless of this max. +- value. The minimal L2 cache size is 2 clusters (or 2 cache entries, see +- below). ++ - The maximum L2 cache size is 32 MB by default on Linux platforms (enough ++ for full coverage of 256 GB images, with the default cluster size). This ++ value can be modified using the "l2-cache-size" option. QEMU will not use ++ more memory than needed to hold all of the image's L2 tables, regardless ++ of this max. value. ++ On non-Linux platforms the maximal value is smaller by default (8 MB) and ++ this difference stems from the fact that on Linux the cache can be cleared ++ periodically if needed, using the "cache-clean-interval" option (see below). ++ The minimal L2 cache size is 2 clusters (or 2 cache entries, see below). + + - The default (and minimum) refcount cache size is 4 clusters. + +diff --git a/qemu-options.hx b/qemu-options.hx +index 5e3bd74..e3f4e43 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -756,9 +756,9 @@ The maximum total size of the L2 table and refcount block caches in bytes + + @item l2-cache-size + The maximum size of the L2 table cache in bytes +-(default: if cache-size is not specified - 1M; otherwise, as large as possible +-within the cache-size, while permitting the requested or the minimal refcount +-cache size) ++(default: if cache-size is not specified - 32M on Linux platforms, and 8M on ++non-Linux platforms; otherwise, as large as possible within the cache-size, ++while permitting the requested or the minimal refcount cache size) + + @item refcount-cache-size + The maximum size of the refcount block cache in bytes +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Make-sizes-more-humanly-readable.patch b/SOURCES/kvm-qcow2-Make-sizes-more-humanly-readable.patch new file mode 100644 index 0000000..04a5e57 --- /dev/null +++ b/SOURCES/kvm-qcow2-Make-sizes-more-humanly-readable.patch @@ -0,0 +1,82 @@ +From 6067fd2ec96b36cb8cc09dc389f5d49c9549ca73 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:32 +0000 +Subject: [PATCH 07/15] qcow2: Make sizes more humanly readable + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-8-kwolf@redhat.com> +Patchwork-id: 83286 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 07/15] qcow2: Make sizes more humanly readable +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit b6a95c6d10075bb540ce50198bbe22fc0a4392c7) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 2 +- + block/qcow2.h | 9 +++++---- + 2 files changed, 6 insertions(+), 5 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index a0f7234..3859112 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -826,7 +826,7 @@ static void read_cache_sizes(BlockDriverState *bs, QemuOpts *opts, + } + } else { + if (!l2_cache_size_set) { +- *l2_cache_size = MAX(DEFAULT_L2_CACHE_BYTE_SIZE, ++ *l2_cache_size = MAX(DEFAULT_L2_CACHE_SIZE, + (uint64_t)DEFAULT_L2_CACHE_CLUSTERS + * s->cluster_size); + } +diff --git a/block/qcow2.h b/block/qcow2.h +index 3d92cdb..f73a48a 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -27,6 +27,7 @@ + + #include "crypto/block.h" + #include "qemu/coroutine.h" ++#include "qemu/units.h" + + //#define DEBUG_ALLOC + //#define DEBUG_ALLOC2 +@@ -43,11 +44,11 @@ + + /* 8 MB refcount table is enough for 2 PB images at 64k cluster size + * (128 GB for 512 byte clusters, 2 EB for 2 MB clusters) */ +-#define QCOW_MAX_REFTABLE_SIZE 0x800000 ++#define QCOW_MAX_REFTABLE_SIZE S_8MiB + + /* 32 MB L1 table is enough for 2 PB images at 64k cluster size + * (128 GB for 512 byte clusters, 2 EB for 2 MB clusters) */ +-#define QCOW_MAX_L1_SIZE 0x2000000 ++#define QCOW_MAX_L1_SIZE S_32MiB + + /* Allow for an average of 1k per snapshot table entry, should be plenty of + * space for snapshot names and IDs */ +@@ -75,9 +76,9 @@ + + /* Whichever is more */ + #define DEFAULT_L2_CACHE_CLUSTERS 8 /* clusters */ +-#define DEFAULT_L2_CACHE_BYTE_SIZE 1048576 /* bytes */ ++#define DEFAULT_L2_CACHE_SIZE S_1MiB + +-#define DEFAULT_CLUSTER_SIZE 65536 ++#define DEFAULT_CLUSTER_SIZE S_64KiB + + + #define QCOW2_OPT_LAZY_REFCOUNTS "lazy-refcounts" +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Options-documentation-fixes.patch b/SOURCES/kvm-qcow2-Options-documentation-fixes.patch new file mode 100644 index 0000000..1667e3a --- /dev/null +++ b/SOURCES/kvm-qcow2-Options-documentation-fixes.patch @@ -0,0 +1,110 @@ +From a608710425e4de7446a0ca5591f00751af971b0a Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:30 +0000 +Subject: [PATCH 05/15] qcow2: Options' documentation fixes + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-6-kwolf@redhat.com> +Patchwork-id: 83285 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 05/15] qcow2: Options' documentation fixes +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 40fb215d483ce510e211b843352288894eb13285) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + docs/qcow2-cache.txt | 21 ++++++++++++++------- + qemu-options.hx | 9 ++++++--- + 2 files changed, 20 insertions(+), 10 deletions(-) + +diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt +index 8a09a5c..7e28b41 100644 +--- a/docs/qcow2-cache.txt ++++ b/docs/qcow2-cache.txt +@@ -79,14 +79,14 @@ Choosing the right cache sizes + In order to choose the cache sizes we need to know how they relate to + the amount of allocated space. + +-The amount of virtual disk that can be mapped by the L2 and refcount ++The part of the virtual disk that can be mapped by the L2 and refcount + caches (in bytes) is: + + disk_size = l2_cache_size * cluster_size / 8 + disk_size = refcount_cache_size * cluster_size * 8 / refcount_bits + + With the default values for cluster_size (64KB) and refcount_bits +-(16), that is ++(16), this becomes: + + disk_size = l2_cache_size * 8192 + disk_size = refcount_cache_size * 32768 +@@ -97,12 +97,16 @@ need: + l2_cache_size = disk_size_GB * 131072 + refcount_cache_size = disk_size_GB * 32768 + +-QEMU has a default L2 cache of 1MB (1048576 bytes) and a refcount +-cache of 256KB (262144 bytes), so using the formulas we've just seen +-we have ++For example, 1MB of L2 cache is needed to cover every 8 GB of the virtual ++image size (given that the default cluster size is used): + +- 1048576 / 131072 = 8 GB of virtual disk covered by that cache +- 262144 / 32768 = 8 GB ++ 8 GB / 8192 = 1 MB ++ ++The refcount cache is 4 times the cluster size by default. With the default ++cluster size of 64 KB, it is 256 KB (262144 bytes). This is sufficient for ++8 GB of image size: ++ ++ 262144 * 32768 = 8 GB + + + How to configure the cache sizes +@@ -130,6 +134,9 @@ There are a few things that need to be taken into account: + memory as possible to the L2 cache before increasing the refcount + cache size. + ++ - At most two of "l2-cache-size", "refcount-cache-size", and "cache-size" ++ can be set simultaneously. ++ + Unlike L2 tables, refcount blocks are not used during normal I/O but + only during allocations and internal snapshots. In most cases they are + accessed sequentially (even during random guest I/O) so increasing the +diff --git a/qemu-options.hx b/qemu-options.hx +index 2e05112..5e15d6f 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -752,15 +752,18 @@ image file) + + @item cache-size + The maximum total size of the L2 table and refcount block caches in bytes +-(default: 1048576 bytes or 8 clusters, whichever is larger) ++(default: the sum of l2-cache-size and refcount-cache-size) + + @item l2-cache-size + The maximum size of the L2 table cache in bytes +-(default: 4/5 of the total cache size) ++(default: if cache-size is not defined - 1048576 bytes or 8 clusters, whichever ++is larger; otherwise, as large as possible or needed within the cache-size, ++while permitting the requested or the minimal refcount cache size) + + @item refcount-cache-size + The maximum size of the refcount block cache in bytes +-(default: 1/5 of the total cache size) ++(default: 4 times the cluster size; or if cache-size is specified, the part of ++it which is not used for the L2 cache) + + @item cache-clean-interval + Clean unused entries in the L2 and refcount caches. The interval is in seconds. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Remove-coroutine-trampoline-for-preallocate_co.patch b/SOURCES/kvm-qcow2-Remove-coroutine-trampoline-for-preallocate_co.patch new file mode 100644 index 0000000..4e05c69 --- /dev/null +++ b/SOURCES/kvm-qcow2-Remove-coroutine-trampoline-for-preallocate_co.patch @@ -0,0 +1,138 @@ +From 8757520d53257c284c3042279488ec1d00e280c3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 12 Jul 2018 14:42:55 +0200 +Subject: [PATCH 210/268] qcow2: Remove coroutine trampoline for + preallocate_co() + +RH-Author: Kevin Wolf +Message-id: <20180712144258.17303-4-kwolf@redhat.com> +Patchwork-id: 81329 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/6] qcow2: Remove coroutine trampoline for preallocate_co() +Bugzilla: 1595173 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: John Snow + +All callers are coroutine_fns now, so we can just directly call +preallocate_co(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 47e86b868d57ffe13162ca44e5f6a51c15c4a769) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/qcow2.c | 51 ++++++++------------------------------------------- + 1 file changed, 8 insertions(+), 43 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index c5c6ae9..71fbfcd 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -2517,15 +2517,6 @@ static int qcow2_set_up_encryption(BlockDriverState *bs, + return ret; + } + +- +-typedef struct PreallocCo { +- BlockDriverState *bs; +- uint64_t offset; +- uint64_t new_length; +- +- int ret; +-} PreallocCo; +- + /** + * Preallocates metadata structures for data clusters between @offset (in the + * guest disk) and @new_length (which is thus generally the new guest disk +@@ -2533,12 +2524,9 @@ typedef struct PreallocCo { + * + * Returns: 0 on success, -errno on failure. + */ +-static void coroutine_fn preallocate_co(void *opaque) ++static int coroutine_fn preallocate_co(BlockDriverState *bs, uint64_t offset, ++ uint64_t new_length) + { +- PreallocCo *params = opaque; +- BlockDriverState *bs = params->bs; +- uint64_t offset = params->offset; +- uint64_t new_length = params->new_length; + uint64_t bytes; + uint64_t host_offset = 0; + unsigned int cur_bytes; +@@ -2553,7 +2541,7 @@ static void coroutine_fn preallocate_co(void *opaque) + ret = qcow2_alloc_cluster_offset(bs, offset, &cur_bytes, + &host_offset, &meta); + if (ret < 0) { +- goto done; ++ return ret; + } + + while (meta) { +@@ -2563,7 +2551,7 @@ static void coroutine_fn preallocate_co(void *opaque) + if (ret < 0) { + qcow2_free_any_clusters(bs, meta->alloc_offset, + meta->nb_clusters, QCOW2_DISCARD_NEVER); +- goto done; ++ return ret; + } + + /* There are no dependent requests, but we need to remove our +@@ -2590,34 +2578,11 @@ static void coroutine_fn preallocate_co(void *opaque) + ret = bdrv_pwrite(bs->file, (host_offset + cur_bytes) - 1, + &data, 1); + if (ret < 0) { +- goto done; ++ return ret; + } + } + +- ret = 0; +- +-done: +- params->ret = ret; +-} +- +-static int preallocate(BlockDriverState *bs, +- uint64_t offset, uint64_t new_length) +-{ +- PreallocCo params = { +- .bs = bs, +- .offset = offset, +- .new_length = new_length, +- .ret = -EINPROGRESS, +- }; +- +- if (qemu_in_coroutine()) { +- preallocate_co(¶ms); +- } else { +- Coroutine *co = qemu_coroutine_create(preallocate_co, ¶ms); +- bdrv_coroutine_enter(bs, co); +- BDRV_POLL_WHILE(bs, params.ret == -EINPROGRESS); +- } +- return params.ret; ++ return 0; + } + + /* qcow2_refcount_metadata_size: +@@ -3035,7 +3000,7 @@ qcow2_co_create(BlockdevCreateOptions *create_options, Error **errp) + if (qcow2_opts->preallocation != PREALLOC_MODE_OFF) { + BDRVQcow2State *s = blk_bs(blk)->opaque; + qemu_co_mutex_lock(&s->lock); +- ret = preallocate(blk_bs(blk), 0, qcow2_opts->size); ++ ret = preallocate_co(blk_bs(blk), 0, qcow2_opts->size); + qemu_co_mutex_unlock(&s->lock); + + if (ret < 0) { +@@ -3544,7 +3509,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, + break; + + case PREALLOC_MODE_METADATA: +- ret = preallocate(bs, old_length, offset); ++ ret = preallocate_co(bs, old_length, offset); + if (ret < 0) { + error_setg_errno(errp, -ret, "Preallocation failed"); + goto fail; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Remove-dead-check-on-ret.patch b/SOURCES/kvm-qcow2-Remove-dead-check-on-ret.patch new file mode 100644 index 0000000..03978ee --- /dev/null +++ b/SOURCES/kvm-qcow2-Remove-dead-check-on-ret.patch @@ -0,0 +1,51 @@ +From da8f22170113e69fd87bf3acd882ab4e1db005a6 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:48 +0200 +Subject: [PATCH 230/268] qcow2: Remove dead check on !ret +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: John Snow +Message-id: <20180718225511.14878-13-jsnow@redhat.com> +Patchwork-id: 81409 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 12/35] qcow2: Remove dead check on !ret +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Fam Zheng + +In the beginning of the function, we initialize the local variable to 0, +and in the body of the function, we check the assigned values and exit +the loop immediately. So here it can never be non-zero. + +Reported-by: Kevin Wolf +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Philippe Mathieu-Daudé +Signed-off-by: Kevin Wolf +(cherry picked from commit 354d930dc6e90e97599459b79c071ff1b93e433b) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/qcow2.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index c5341a4..e171a99 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1772,7 +1772,7 @@ static coroutine_fn int qcow2_handle_l2meta(BlockDriverState *bs, + while (l2meta != NULL) { + QCowL2Meta *next; + +- if (!ret && link_l2) { ++ if (link_l2) { + ret = qcow2_alloc_cluster_link_l2(bs, l2meta); + if (ret) { + goto out; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Repair-OFLAG_COPIED-when-fixing-leaks.patch b/SOURCES/kvm-qcow2-Repair-OFLAG_COPIED-when-fixing-leaks.patch new file mode 100644 index 0000000..8843e69 --- /dev/null +++ b/SOURCES/kvm-qcow2-Repair-OFLAG_COPIED-when-fixing-leaks.patch @@ -0,0 +1,90 @@ +From 28e64491eecd735d36af79c83226682e097af1ef Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 17:16:54 +0200 +Subject: [PATCH 054/268] qcow2: Repair OFLAG_COPIED when fixing leaks + +RH-Author: Max Reitz +Message-id: <20180618171655.25987-2-mreitz@redhat.com> +Patchwork-id: 80782 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/2] qcow2: Repair OFLAG_COPIED when fixing leaks +Bugzilla: 1527085 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Repairing OFLAG_COPIED is usually safe because it is done after the +refcounts have been repaired. Therefore, it we did not find anyone else +referencing a data or L2 cluster, it makes no sense to not set +OFLAG_COPIED -- and the other direction (clearing OFLAG_COPIED) is +always safe, anyway, it may just induce leaks. + +Furthermore, if OFLAG_COPIED is actually consistent with a wrong (leaky) +refcount, we will decrement the refcount with -r leaks, but OFLAG_COPIED +will then be wrong. qemu-img check should not produce images that are +more corrupted afterwards then they were before. + +Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1527085 +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509200059.31125-2-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 3cce51c919c7b4028cf6676dfcb80a45741b5117) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + block/qcow2-refcount.c | 25 +++++++++++++++++-------- + 1 file changed, 17 insertions(+), 8 deletions(-) + +diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c +index 6b8b635..4e14c0a 100644 +--- a/block/qcow2-refcount.c ++++ b/block/qcow2-refcount.c +@@ -1799,6 +1799,19 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, + int ret; + uint64_t refcount; + int i, j; ++ bool repair; ++ ++ if (fix & BDRV_FIX_ERRORS) { ++ /* Always repair */ ++ repair = true; ++ } else if (fix & BDRV_FIX_LEAKS) { ++ /* Repair only if that seems safe: This function is always ++ * called after the refcounts have been fixed, so the refcount ++ * is accurate if that repair was successful */ ++ repair = !res->check_errors && !res->corruptions && !res->leaks; ++ } else { ++ repair = false; ++ } + + for (i = 0; i < s->l1_size; i++) { + uint64_t l1_entry = s->l1_table[i]; +@@ -1818,10 +1831,8 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, + if ((refcount == 1) != ((l1_entry & QCOW_OFLAG_COPIED) != 0)) { + fprintf(stderr, "%s OFLAG_COPIED L2 cluster: l1_index=%d " + "l1_entry=%" PRIx64 " refcount=%" PRIu64 "\n", +- fix & BDRV_FIX_ERRORS ? "Repairing" : +- "ERROR", +- i, l1_entry, refcount); +- if (fix & BDRV_FIX_ERRORS) { ++ repair ? "Repairing" : "ERROR", i, l1_entry, refcount); ++ if (repair) { + s->l1_table[i] = refcount == 1 + ? l1_entry | QCOW_OFLAG_COPIED + : l1_entry & ~QCOW_OFLAG_COPIED; +@@ -1862,10 +1873,8 @@ static int check_oflag_copied(BlockDriverState *bs, BdrvCheckResult *res, + if ((refcount == 1) != ((l2_entry & QCOW_OFLAG_COPIED) != 0)) { + fprintf(stderr, "%s OFLAG_COPIED data cluster: " + "l2_entry=%" PRIx64 " refcount=%" PRIu64 "\n", +- fix & BDRV_FIX_ERRORS ? "Repairing" : +- "ERROR", +- l2_entry, refcount); +- if (fix & BDRV_FIX_ERRORS) { ++ repair ? "Repairing" : "ERROR", l2_entry, refcount); ++ if (repair) { + l2_table[j] = cpu_to_be64(refcount == 1 + ? l2_entry | QCOW_OFLAG_COPIED + : l2_entry & ~QCOW_OFLAG_COPIED); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Resize-the-cache-upon-image-resizing.patch b/SOURCES/kvm-qcow2-Resize-the-cache-upon-image-resizing.patch new file mode 100644 index 0000000..a5a068e --- /dev/null +++ b/SOURCES/kvm-qcow2-Resize-the-cache-upon-image-resizing.patch @@ -0,0 +1,70 @@ +From 1041ad03a2c47f5bbae0b4c8089bab98ef1ba12d Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:36 +0000 +Subject: [PATCH 11/15] qcow2: Resize the cache upon image resizing + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-12-kwolf@redhat.com> +Patchwork-id: 83293 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 11/15] qcow2: Resize the cache upon image resizing +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +The caches are now recalculated upon image resizing. This is done +because the new default behavior of assigning L2 cache relatively to +the image size, implies that the cache will be adapted accordingly +after an image resize. + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Signed-off-by: Kevin Wolf +(cherry picked from commit 45b4949c7bcdcd998cb42f5c517e80a2657cfd33) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 11 +++++++++++ + 1 file changed, 11 insertions(+) + +diff --git a/block/qcow2.c b/block/qcow2.c +index fc6bddd..72f1ea8 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -3462,6 +3462,7 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, + uint64_t old_length; + int64_t new_l1_size; + int ret; ++ QDict *options; + + if (prealloc != PREALLOC_MODE_OFF && prealloc != PREALLOC_MODE_METADATA && + prealloc != PREALLOC_MODE_FALLOC && prealloc != PREALLOC_MODE_FULL) +@@ -3686,6 +3687,8 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, + } + } + ++ bs->total_sectors = offset / BDRV_SECTOR_SIZE; ++ + /* write updated header.size */ + offset = cpu_to_be64(offset); + ret = bdrv_pwrite_sync(bs->file, offsetof(QCowHeader, size), +@@ -3696,6 +3699,14 @@ static int coroutine_fn qcow2_co_truncate(BlockDriverState *bs, int64_t offset, + } + + s->l1_vm_state_index = new_l1_size; ++ ++ /* Update cache sizes */ ++ options = qdict_clone_shallow(bs->options); ++ ret = qcow2_update_options(bs, options, s->flags, errp); ++ qobject_unref(options); ++ if (ret < 0) { ++ goto fail; ++ } + ret = 0; + fail: + qemu_co_mutex_unlock(&s->lock); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-Set-the-default-cache-clean-interval-to-10-min.patch b/SOURCES/kvm-qcow2-Set-the-default-cache-clean-interval-to-10-min.patch new file mode 100644 index 0000000..268475f --- /dev/null +++ b/SOURCES/kvm-qcow2-Set-the-default-cache-clean-interval-to-10-min.patch @@ -0,0 +1,118 @@ +From 5e5ad26daad9372b9276b18c5b773528b8239c31 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 6 Dec 2018 17:12:37 +0000 +Subject: [PATCH 12/15] qcow2: Set the default cache-clean-interval to 10 + minutes + +RH-Author: Kevin Wolf +Message-id: <20181206171240.5674-13-kwolf@redhat.com> +Patchwork-id: 83288 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 12/15] qcow2: Set the default cache-clean-interval to 10 minutes +Bugzilla: 1656507 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +From: Leonid Bloch + +The default cache-clean-interval is set to 10 minutes, in order to lower +the overhead of the qcow2 caches (before the default was 0, i.e. +disabled). + +* For non-Linux platforms the default is kept at 0, because + cache-clean-interval is not supported there yet. + +Signed-off-by: Leonid Bloch +Reviewed-by: Alberto Garcia +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit e957b50b8daecfc39a1ac09855b0eacb6edfd328) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/qcow2.c | 2 +- + block/qcow2.h | 4 +++- + docs/qcow2-cache.txt | 4 ++-- + qapi/block-core.json | 3 ++- + qemu-options.hx | 2 +- + 5 files changed, 9 insertions(+), 6 deletions(-) + +diff --git a/block/qcow2.c b/block/qcow2.c +index 72f1ea8..acd076c 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -940,7 +940,7 @@ static int qcow2_update_options_prepare(BlockDriverState *bs, + /* New interval for cache cleanup timer */ + r->cache_clean_interval = + qemu_opt_get_number(opts, QCOW2_OPT_CACHE_CLEAN_INTERVAL, +- s->cache_clean_interval); ++ DEFAULT_CACHE_CLEAN_INTERVAL); + #ifndef CONFIG_LINUX + if (r->cache_clean_interval != 0) { + error_setg(errp, QCOW2_OPT_CACHE_CLEAN_INTERVAL +diff --git a/block/qcow2.h b/block/qcow2.h +index 6c6c742..29b041c 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -76,13 +76,15 @@ + + #ifdef CONFIG_LINUX + #define DEFAULT_L2_CACHE_MAX_SIZE S_32MiB ++#define DEFAULT_CACHE_CLEAN_INTERVAL 600 /* seconds */ + #else + #define DEFAULT_L2_CACHE_MAX_SIZE S_8MiB ++/* Cache clean interval is currently available only on Linux, so must be 0 */ ++#define DEFAULT_CACHE_CLEAN_INTERVAL 0 + #endif + + #define DEFAULT_CLUSTER_SIZE S_64KiB + +- + #define QCOW2_OPT_LAZY_REFCOUNTS "lazy-refcounts" + #define QCOW2_OPT_DISCARD_REQUEST "pass-discard-request" + #define QCOW2_OPT_DISCARD_SNAPSHOT "pass-discard-snapshot" +diff --git a/docs/qcow2-cache.txt b/docs/qcow2-cache.txt +index 1fcc065..59358b8 100644 +--- a/docs/qcow2-cache.txt ++++ b/docs/qcow2-cache.txt +@@ -210,8 +210,8 @@ This example removes all unused cache entries every 15 minutes: + + -drive file=hd.qcow2,cache-clean-interval=900 + +-If unset, the default value for this parameter is 0 and it disables +-this feature. ++If unset, the default value for this parameter is 600. Setting it to 0 ++disables this feature. + + Note that this functionality currently relies on the MADV_DONTNEED + argument for madvise() to actually free the memory. This is a +diff --git a/qapi/block-core.json b/qapi/block-core.json +index a6c3977..5318c9b 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -2867,7 +2867,8 @@ + # + # @cache-clean-interval: clean unused entries in the L2 and refcount + # caches. The interval is in seconds. The default value +-# is 0 and it disables this feature (since 2.5) ++# is 600, and 0 disables this feature. (since 2.5) ++# + # @encrypt: Image decryption options. Mandatory for + # encrypted images, except when doing a metadata-only + # probe of the image. (since 2.10) +diff --git a/qemu-options.hx b/qemu-options.hx +index e3f4e43..05fabf3 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -767,7 +767,7 @@ it which is not used for the L2 cache) + + @item cache-clean-interval + Clean unused entries in the L2 and refcount caches. The interval is in seconds. +-The default value is 0 and it disables this feature. ++The default value is 600. Setting it to 0 disables this feature. + + @item pass-discard-request + Whether discard requests to the qcow2 device should be forwarded to the data +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qcow2-add-overlap-check-for-bitmap-directory.patch b/SOURCES/kvm-qcow2-add-overlap-check-for-bitmap-directory.patch new file mode 100644 index 0000000..1fbcef5 --- /dev/null +++ b/SOURCES/kvm-qcow2-add-overlap-check-for-bitmap-directory.patch @@ -0,0 +1,217 @@ +From e7b1acac8246a203ed0ed55a83cec29a4fa6252c Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:45 +0200 +Subject: [PATCH 227/268] qcow2: add overlap check for bitmap directory + +RH-Author: John Snow +Message-id: <20180718225511.14878-10-jsnow@redhat.com> +Patchwork-id: 81394 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 09/35] qcow2: add overlap check for bitmap directory +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Message-id: 20180705151515.779173-1-vsementsov@virtuozzo.com +Signed-off-by: Max Reitz +(cherry picked from commit 0e4e4318eaa56c831001bdf617094807ec6d451c) +Signed-off-by: John Snow +Signed-off-by: Miroslav Rezanina +--- + block/qcow2-bitmap.c | 7 ++++++- + block/qcow2-refcount.c | 10 ++++++++++ + block/qcow2.c | 22 ++++++++++++++-------- + block/qcow2.h | 45 ++++++++++++++++++++++++--------------------- + qapi/block-core.json | 21 ++++++++++++--------- + 5 files changed, 66 insertions(+), 39 deletions(-) + +diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c +index 3e4e4e4..14050eb 100644 +--- a/block/qcow2-bitmap.c ++++ b/block/qcow2-bitmap.c +@@ -775,7 +775,12 @@ static int bitmap_list_store(BlockDriverState *bs, Qcow2BitmapList *bm_list, + } + } + +- ret = qcow2_pre_write_overlap_check(bs, 0, dir_offset, dir_size); ++ /* Actually, even in in-place case ignoring QCOW2_OL_BITMAP_DIRECTORY is not ++ * necessary, because we drop QCOW2_AUTOCLEAR_BITMAPS when updating bitmap ++ * directory in-place (actually, turn-off the extension), which is checked ++ * in qcow2_check_metadata_overlap() */ ++ ret = qcow2_pre_write_overlap_check( ++ bs, in_place ? QCOW2_OL_BITMAP_DIRECTORY : 0, dir_offset, dir_size); + if (ret < 0) { + goto fail; + } +diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c +index 4e14c0a..1307069 100644 +--- a/block/qcow2-refcount.c ++++ b/block/qcow2-refcount.c +@@ -2705,6 +2705,16 @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset, + } + } + ++ if ((chk & QCOW2_OL_BITMAP_DIRECTORY) && ++ (s->autoclear_features & QCOW2_AUTOCLEAR_BITMAPS)) ++ { ++ if (overlaps_with(s->bitmap_directory_offset, ++ s->bitmap_directory_size)) ++ { ++ return QCOW2_OL_BITMAP_DIRECTORY; ++ } ++ } ++ + return 0; + } + +diff --git a/block/qcow2.c b/block/qcow2.c +index 71fbfcd..c5341a4 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -676,6 +676,11 @@ static QemuOptsList qcow2_runtime_opts = { + .help = "Check for unintended writes into an inactive L2 table", + }, + { ++ .name = QCOW2_OPT_OVERLAP_BITMAP_DIRECTORY, ++ .type = QEMU_OPT_BOOL, ++ .help = "Check for unintended writes into the bitmap directory", ++ }, ++ { + .name = QCOW2_OPT_CACHE_SIZE, + .type = QEMU_OPT_SIZE, + .help = "Maximum combined metadata (L2 tables and refcount blocks) " +@@ -708,14 +713,15 @@ static QemuOptsList qcow2_runtime_opts = { + }; + + static const char *overlap_bool_option_names[QCOW2_OL_MAX_BITNR] = { +- [QCOW2_OL_MAIN_HEADER_BITNR] = QCOW2_OPT_OVERLAP_MAIN_HEADER, +- [QCOW2_OL_ACTIVE_L1_BITNR] = QCOW2_OPT_OVERLAP_ACTIVE_L1, +- [QCOW2_OL_ACTIVE_L2_BITNR] = QCOW2_OPT_OVERLAP_ACTIVE_L2, +- [QCOW2_OL_REFCOUNT_TABLE_BITNR] = QCOW2_OPT_OVERLAP_REFCOUNT_TABLE, +- [QCOW2_OL_REFCOUNT_BLOCK_BITNR] = QCOW2_OPT_OVERLAP_REFCOUNT_BLOCK, +- [QCOW2_OL_SNAPSHOT_TABLE_BITNR] = QCOW2_OPT_OVERLAP_SNAPSHOT_TABLE, +- [QCOW2_OL_INACTIVE_L1_BITNR] = QCOW2_OPT_OVERLAP_INACTIVE_L1, +- [QCOW2_OL_INACTIVE_L2_BITNR] = QCOW2_OPT_OVERLAP_INACTIVE_L2, ++ [QCOW2_OL_MAIN_HEADER_BITNR] = QCOW2_OPT_OVERLAP_MAIN_HEADER, ++ [QCOW2_OL_ACTIVE_L1_BITNR] = QCOW2_OPT_OVERLAP_ACTIVE_L1, ++ [QCOW2_OL_ACTIVE_L2_BITNR] = QCOW2_OPT_OVERLAP_ACTIVE_L2, ++ [QCOW2_OL_REFCOUNT_TABLE_BITNR] = QCOW2_OPT_OVERLAP_REFCOUNT_TABLE, ++ [QCOW2_OL_REFCOUNT_BLOCK_BITNR] = QCOW2_OPT_OVERLAP_REFCOUNT_BLOCK, ++ [QCOW2_OL_SNAPSHOT_TABLE_BITNR] = QCOW2_OPT_OVERLAP_SNAPSHOT_TABLE, ++ [QCOW2_OL_INACTIVE_L1_BITNR] = QCOW2_OPT_OVERLAP_INACTIVE_L1, ++ [QCOW2_OL_INACTIVE_L2_BITNR] = QCOW2_OPT_OVERLAP_INACTIVE_L2, ++ [QCOW2_OL_BITMAP_DIRECTORY_BITNR] = QCOW2_OPT_OVERLAP_BITMAP_DIRECTORY, + }; + + static void cache_clean_timer_cb(void *opaque) +diff --git a/block/qcow2.h b/block/qcow2.h +index b5e2aa3..d2c63e4 100644 +--- a/block/qcow2.h ++++ b/block/qcow2.h +@@ -98,6 +98,7 @@ + #define QCOW2_OPT_OVERLAP_SNAPSHOT_TABLE "overlap-check.snapshot-table" + #define QCOW2_OPT_OVERLAP_INACTIVE_L1 "overlap-check.inactive-l1" + #define QCOW2_OPT_OVERLAP_INACTIVE_L2 "overlap-check.inactive-l2" ++#define QCOW2_OPT_OVERLAP_BITMAP_DIRECTORY "overlap-check.bitmap-directory" + #define QCOW2_OPT_CACHE_SIZE "cache-size" + #define QCOW2_OPT_L2_CACHE_SIZE "l2-cache-size" + #define QCOW2_OPT_L2_CACHE_ENTRY_SIZE "l2-cache-entry-size" +@@ -401,34 +402,36 @@ typedef enum QCow2ClusterType { + } QCow2ClusterType; + + typedef enum QCow2MetadataOverlap { +- QCOW2_OL_MAIN_HEADER_BITNR = 0, +- QCOW2_OL_ACTIVE_L1_BITNR = 1, +- QCOW2_OL_ACTIVE_L2_BITNR = 2, +- QCOW2_OL_REFCOUNT_TABLE_BITNR = 3, +- QCOW2_OL_REFCOUNT_BLOCK_BITNR = 4, +- QCOW2_OL_SNAPSHOT_TABLE_BITNR = 5, +- QCOW2_OL_INACTIVE_L1_BITNR = 6, +- QCOW2_OL_INACTIVE_L2_BITNR = 7, +- +- QCOW2_OL_MAX_BITNR = 8, +- +- QCOW2_OL_NONE = 0, +- QCOW2_OL_MAIN_HEADER = (1 << QCOW2_OL_MAIN_HEADER_BITNR), +- QCOW2_OL_ACTIVE_L1 = (1 << QCOW2_OL_ACTIVE_L1_BITNR), +- QCOW2_OL_ACTIVE_L2 = (1 << QCOW2_OL_ACTIVE_L2_BITNR), +- QCOW2_OL_REFCOUNT_TABLE = (1 << QCOW2_OL_REFCOUNT_TABLE_BITNR), +- QCOW2_OL_REFCOUNT_BLOCK = (1 << QCOW2_OL_REFCOUNT_BLOCK_BITNR), +- QCOW2_OL_SNAPSHOT_TABLE = (1 << QCOW2_OL_SNAPSHOT_TABLE_BITNR), +- QCOW2_OL_INACTIVE_L1 = (1 << QCOW2_OL_INACTIVE_L1_BITNR), ++ QCOW2_OL_MAIN_HEADER_BITNR = 0, ++ QCOW2_OL_ACTIVE_L1_BITNR = 1, ++ QCOW2_OL_ACTIVE_L2_BITNR = 2, ++ QCOW2_OL_REFCOUNT_TABLE_BITNR = 3, ++ QCOW2_OL_REFCOUNT_BLOCK_BITNR = 4, ++ QCOW2_OL_SNAPSHOT_TABLE_BITNR = 5, ++ QCOW2_OL_INACTIVE_L1_BITNR = 6, ++ QCOW2_OL_INACTIVE_L2_BITNR = 7, ++ QCOW2_OL_BITMAP_DIRECTORY_BITNR = 8, ++ ++ QCOW2_OL_MAX_BITNR = 9, ++ ++ QCOW2_OL_NONE = 0, ++ QCOW2_OL_MAIN_HEADER = (1 << QCOW2_OL_MAIN_HEADER_BITNR), ++ QCOW2_OL_ACTIVE_L1 = (1 << QCOW2_OL_ACTIVE_L1_BITNR), ++ QCOW2_OL_ACTIVE_L2 = (1 << QCOW2_OL_ACTIVE_L2_BITNR), ++ QCOW2_OL_REFCOUNT_TABLE = (1 << QCOW2_OL_REFCOUNT_TABLE_BITNR), ++ QCOW2_OL_REFCOUNT_BLOCK = (1 << QCOW2_OL_REFCOUNT_BLOCK_BITNR), ++ QCOW2_OL_SNAPSHOT_TABLE = (1 << QCOW2_OL_SNAPSHOT_TABLE_BITNR), ++ QCOW2_OL_INACTIVE_L1 = (1 << QCOW2_OL_INACTIVE_L1_BITNR), + /* NOTE: Checking overlaps with inactive L2 tables will result in bdrv + * reads. */ +- QCOW2_OL_INACTIVE_L2 = (1 << QCOW2_OL_INACTIVE_L2_BITNR), ++ QCOW2_OL_INACTIVE_L2 = (1 << QCOW2_OL_INACTIVE_L2_BITNR), ++ QCOW2_OL_BITMAP_DIRECTORY = (1 << QCOW2_OL_BITMAP_DIRECTORY_BITNR), + } QCow2MetadataOverlap; + + /* Perform all overlap checks which can be done in constant time */ + #define QCOW2_OL_CONSTANT \ + (QCOW2_OL_MAIN_HEADER | QCOW2_OL_ACTIVE_L1 | QCOW2_OL_REFCOUNT_TABLE | \ +- QCOW2_OL_SNAPSHOT_TABLE) ++ QCOW2_OL_SNAPSHOT_TABLE | QCOW2_OL_BITMAP_DIRECTORY) + + /* Perform all overlap checks which don't require disk access */ + #define QCOW2_OL_CACHED \ +diff --git a/qapi/block-core.json b/qapi/block-core.json +index 9a9cfa0..b2de7af 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -2666,18 +2666,21 @@ + # @template: Specifies a template mode which can be adjusted using the other + # flags, defaults to 'cached' + # ++# @bitmap-directory: since 3.0 ++# + # Since: 2.9 + ## + { 'struct': 'Qcow2OverlapCheckFlags', +- 'data': { '*template': 'Qcow2OverlapCheckMode', +- '*main-header': 'bool', +- '*active-l1': 'bool', +- '*active-l2': 'bool', +- '*refcount-table': 'bool', +- '*refcount-block': 'bool', +- '*snapshot-table': 'bool', +- '*inactive-l1': 'bool', +- '*inactive-l2': 'bool' } } ++ 'data': { '*template': 'Qcow2OverlapCheckMode', ++ '*main-header': 'bool', ++ '*active-l1': 'bool', ++ '*active-l2': 'bool', ++ '*refcount-table': 'bool', ++ '*refcount-block': 'bool', ++ '*snapshot-table': 'bool', ++ '*inactive-l1': 'bool', ++ '*inactive-l2': 'bool', ++ '*bitmap-directory': 'bool' } } + + ## + # @Qcow2OverlapChecks: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qdev-add-HotplugHandler-post_plug-callback.patch b/SOURCES/kvm-qdev-add-HotplugHandler-post_plug-callback.patch new file mode 100644 index 0000000..77892fb --- /dev/null +++ b/SOURCES/kvm-qdev-add-HotplugHandler-post_plug-callback.patch @@ -0,0 +1,109 @@ +From 2682f81498f22a9d10bb3fb77a761e613454ce18 Mon Sep 17 00:00:00 2001 +From: Stefan Hajnoczi +Date: Tue, 24 Jul 2018 15:13:07 +0200 +Subject: [PATCH 263/268] qdev: add HotplugHandler->post_plug() callback + +RH-Author: Stefan Hajnoczi +Message-id: <20180724151308.20500-2-stefanha@redhat.com> +Patchwork-id: 81485 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/2] qdev: add HotplugHandler->post_plug() callback +Bugzilla: 1607891 +RH-Acked-by: Igor Mammedov +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Cornelia Huck + +The ->pre_plug() callback is invoked before the device is realized. The +->plug() callback is invoked when the device is being realized but +before it is reset. + +This patch adds a ->post_plug() callback which is invoked after the +device has been reset. This callback is needed by HotplugHandlers that +need to wait until after ->reset(). + +Signed-off-by: Stefan Hajnoczi +Message-Id: <20180716083732.3347-2-stefanha@redhat.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit 25e8978817a54745c44d956d8303e6be6f2c4047) +Signed-off-by: Stefan Hajnoczi +Signed-off-by: Miroslav Rezanina +--- + hw/core/hotplug.c | 10 ++++++++++ + hw/core/qdev.c | 4 ++++ + include/hw/hotplug.h | 11 +++++++++++ + 3 files changed, 25 insertions(+) + +diff --git a/hw/core/hotplug.c b/hw/core/hotplug.c +index 17ac986..2253072 100644 +--- a/hw/core/hotplug.c ++++ b/hw/core/hotplug.c +@@ -35,6 +35,16 @@ void hotplug_handler_plug(HotplugHandler *plug_handler, + } + } + ++void hotplug_handler_post_plug(HotplugHandler *plug_handler, ++ DeviceState *plugged_dev) ++{ ++ HotplugHandlerClass *hdc = HOTPLUG_HANDLER_GET_CLASS(plug_handler); ++ ++ if (hdc->post_plug) { ++ hdc->post_plug(plug_handler, plugged_dev); ++ } ++} ++ + void hotplug_handler_unplug_request(HotplugHandler *plug_handler, + DeviceState *plugged_dev, + Error **errp) +diff --git a/hw/core/qdev.c b/hw/core/qdev.c +index ce7c316..24f1ae7 100644 +--- a/hw/core/qdev.c ++++ b/hw/core/qdev.c +@@ -893,6 +893,10 @@ static void device_set_realized(Object *obj, bool value, Error **errp) + device_reset(dev); + } + dev->pending_deleted_event = false; ++ ++ if (hotplug_ctrl) { ++ hotplug_handler_post_plug(hotplug_ctrl, dev); ++ } + } else if (!value && dev->realized) { + Error **local_errp = NULL; + QLIST_FOREACH(bus, &dev->child_bus, sibling) { +diff --git a/include/hw/hotplug.h b/include/hw/hotplug.h +index 1a0516a..51541d6 100644 +--- a/include/hw/hotplug.h ++++ b/include/hw/hotplug.h +@@ -47,6 +47,8 @@ typedef void (*hotplug_fn)(HotplugHandler *plug_handler, + * @parent: Opaque parent interface. + * @pre_plug: pre plug callback called at start of device.realize(true) + * @plug: plug callback called at end of device.realize(true). ++ * @post_plug: post plug callback called after device.realize(true) and device ++ * reset + * @unplug_request: unplug request callback. + * Used as a means to initiate device unplug for devices that + * require asynchronous unplug handling. +@@ -61,6 +63,7 @@ typedef struct HotplugHandlerClass { + /* */ + hotplug_fn pre_plug; + hotplug_fn plug; ++ void (*post_plug)(HotplugHandler *plug_handler, DeviceState *plugged_dev); + hotplug_fn unplug_request; + hotplug_fn unplug; + } HotplugHandlerClass; +@@ -84,6 +87,14 @@ void hotplug_handler_pre_plug(HotplugHandler *plug_handler, + Error **errp); + + /** ++ * hotplug_handler_post_plug: ++ * ++ * Call #HotplugHandlerClass.post_plug callback of @plug_handler. ++ */ ++void hotplug_handler_post_plug(HotplugHandler *plug_handler, ++ DeviceState *plugged_dev); ++ ++/** + * hotplug_handler_unplug_request: + * + * Calls #HotplugHandlerClass.unplug_request callback of @plug_handler. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-error-introduce-error-warn-_report_once.patch b/SOURCES/kvm-qemu-error-introduce-error-warn-_report_once.patch new file mode 100644 index 0000000..a372ec7 --- /dev/null +++ b/SOURCES/kvm-qemu-error-introduce-error-warn-_report_once.patch @@ -0,0 +1,105 @@ +From cb9a84154c755daf0c9361346302ad41a1804ecc Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Thu, 8 Nov 2018 06:29:32 +0000 +Subject: [PATCH 04/35] qemu-error: introduce {error|warn}_report_once +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Peter Xu +Message-id: <20181108062938.21143-2-peterx@redhat.com> +Patchwork-id: 82959 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/7] qemu-error: introduce {error|warn}_report_once +Bugzilla: 1625173 +RH-Acked-by: Auger Eric +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Laurent Vivier + +Bugzilla: 1625173 + +There are many error_report()s that can be used in frequently called +functions, especially on IO paths. That can be unideal in that +malicious guest can try to trigger the error tons of time which might +use up the log space on the host (e.g., libvirt can capture the stderr +of QEMU and put it persistently onto disk). In VT-d emulation code, we +have trace_vtd_error() tracer. AFAIU all those places can be replaced +by something like error_report() but trace points are mostly used to +avoid the DDOS attack that mentioned above. However using trace points +mean that errors are not dumped if trace not enabled. + +It's not a big deal in most modern server managements since we have +things like logrotate to maintain the logs and make sure the quota is +expected. However it'll still be nice that we just provide another way +to restrict message generations. In most cases, this kind of +error_report()s will only provide valid information on the first message +sent, and all the rest of similar messages will be mostly talking about +the same thing. This patch introduces *_report_once() helpers to allow +a message to be dumped only once during one QEMU process's life cycle. +It will make sure: (1) it's on by deffault, so we can even get something +without turning the trace on and reproducing, and (2) it won't be +affected by DDOS attack. + +To implement it, I stole the printk_once() macro from Linux. + +CC: Eric Blake +CC: Markus Armbruster +Signed-off-by: Peter Xu +Message-Id: <20180815095328.32414-2-peterx@redhat.com> +Reviewed-by: Markus Armbruster +[Whitespace adjusted, comments improved] +Signed-off-by: Markus Armbruster +(cherry picked from commit bc6a69dd4bfa41ae56235dcbb9a28a56e12a7dc6) +Signed-off-by: Peter Xu + +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/error-report.h | 32 ++++++++++++++++++++++++++++++++ + 1 file changed, 32 insertions(+) + +diff --git a/include/qemu/error-report.h b/include/qemu/error-report.h +index e1c8ae1..72fab2b 100644 +--- a/include/qemu/error-report.h ++++ b/include/qemu/error-report.h +@@ -44,6 +44,38 @@ void error_report(const char *fmt, ...) GCC_FMT_ATTR(1, 2); + void warn_report(const char *fmt, ...) GCC_FMT_ATTR(1, 2); + void info_report(const char *fmt, ...) GCC_FMT_ATTR(1, 2); + ++/* ++ * Similar to error_report(), except it prints the message just once. ++ * Return true when it prints, false otherwise. ++ */ ++#define error_report_once(fmt, ...) \ ++ ({ \ ++ static bool print_once_; \ ++ bool ret_print_once_ = !print_once_; \ ++ \ ++ if (!print_once_) { \ ++ print_once_ = true; \ ++ error_report(fmt, ##__VA_ARGS__); \ ++ } \ ++ unlikely(ret_print_once_); \ ++ }) ++ ++/* ++ * Similar to warn_report(), except it prints the message just once. ++ * Return true when it prints, false otherwise. ++ */ ++#define warn_report_once(fmt, ...) \ ++ ({ \ ++ static bool print_once_; \ ++ bool ret_print_once_ = !print_once_; \ ++ \ ++ if (!print_once_) { \ ++ print_once_ = true; \ ++ warn_report(fmt, ##__VA_ARGS__); \ ++ } \ ++ unlikely(ret_print_once_); \ ++ }) ++ + const char *error_get_progname(void); + extern bool enable_timestamp_msg; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-ga-make-get-fsinfo-work-over-pci-bridges.patch b/SOURCES/kvm-qemu-ga-make-get-fsinfo-work-over-pci-bridges.patch new file mode 100644 index 0000000..3e11577 --- /dev/null +++ b/SOURCES/kvm-qemu-ga-make-get-fsinfo-work-over-pci-bridges.patch @@ -0,0 +1,92 @@ +From 0e3b855ea8223c8150192820cde86def8142de29 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Tue, 5 Feb 2019 11:25:50 +0000 +Subject: [PATCH 1/2] qemu-ga: make get-fsinfo work over pci bridges +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20190205112551.14763-2-marcandre.lureau@redhat.com> +Patchwork-id: 84241 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/2] qemu-ga: make get-fsinfo work over pci bridges +Bugzilla: 1666952 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Danilo de Paula + +Iterate over the PCI bridges to lookup the PCI device associated with +the block device. + +This allows to lookup the driver under the following syspath: +/sys/devices/pci0000:00/0000:00:02.2/0000:03:00.0/virtio2/block/vda/vda3 + +It also works with an "old-style" Q35 libvirt hierarchy: root complex +-> DMI-PCI bridge -> PCI-PCI bridge -> virtio controller, ex: +/sys/devices/pci0000:00/0000:00:03.0/0000:01:01.0/0000:02:01.0/virtio1/block/vda/vda3 + +The setup can be reproduced with the following qemu command line +(Thanks Marcel for help): + +qemu-system-x86_64 -M q35 \ + -device i82801b11-bridge,id=dmi2pci_bridge,bus=pcie.0 + -device pci-bridge,id=pci_bridge,bus=dmi2pci_bridge,addr=0x1,chassis_nr=1 + -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,bus=pci_bridge,addr=0x1 + +For consistency with other syspath-related debug messages, replace a +\"%s\" in the message with '%s'. + +Fixes: +https://bugzilla.redhat.com/show_bug.cgi?id=1567041 + +Signed-off-by: Marc-André Lureau +Reviewed-by: Laszlo Ersek +Signed-off-by: Michael Roth + +(cherry picked from commit 743c71d03c20d64f2bae5fba6f26cdf5e4b1bda6) +Signed-off-by: Marc-André Lureau +Signed-off-by: Danilo C. L. de Paula +--- + qga/commands-posix.c | 23 +++++++++++++++++++---- + 1 file changed, 19 insertions(+), 4 deletions(-) + +diff --git a/qga/commands-posix.c b/qga/commands-posix.c +index 82cb120..71cb644 100644 +--- a/qga/commands-posix.c ++++ b/qga/commands-posix.c +@@ -883,13 +883,28 @@ static void build_guest_fsinfo_for_real_device(char const *syspath, + p = strstr(syspath, "/devices/pci"); + if (!p || sscanf(p + 12, "%*x:%*x/%x:%x:%x.%x%n", + pci, pci + 1, pci + 2, pci + 3, &pcilen) < 4) { +- g_debug("only pci device is supported: sysfs path \"%s\"", syspath); ++ g_debug("only pci device is supported: sysfs path '%s'", syspath); + return; + } + +- driver = get_pci_driver(syspath, (p + 12 + pcilen) - syspath, errp); +- if (!driver) { +- goto cleanup; ++ p += 12 + pcilen; ++ while (true) { ++ driver = get_pci_driver(syspath, p - syspath, errp); ++ if (driver && (g_str_equal(driver, "ata_piix") || ++ g_str_equal(driver, "sym53c8xx") || ++ g_str_equal(driver, "virtio-pci") || ++ g_str_equal(driver, "ahci"))) { ++ break; ++ } ++ ++ if (sscanf(p, "/%x:%x:%x.%x%n", ++ pci, pci + 1, pci + 2, pci + 3, &pcilen) == 4) { ++ p += pcilen; ++ continue; ++ } ++ ++ g_debug("unsupported driver or sysfs path '%s'", syspath); ++ return; + } + + p = strstr(syspath, "/target"); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Add-C-option-for-convert-with-copy-offloadi.patch b/SOURCES/kvm-qemu-img-Add-C-option-for-convert-with-copy-offloadi.patch new file mode 100644 index 0000000..d8af77f --- /dev/null +++ b/SOURCES/kvm-qemu-img-Add-C-option-for-convert-with-copy-offloadi.patch @@ -0,0 +1,146 @@ +From 06e8357157c82b57f86ea04ba2d782ddee0e09df Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Thu, 17 Jan 2019 19:11:10 +0000 +Subject: [PATCH 13/14] qemu-img: Add -C option for convert with copy + offloading + +RH-Author: John Snow +Message-id: <20190117191111.30782-2-jsnow@redhat.com> +Patchwork-id: 84041 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/2] qemu-img: Add -C option for convert with copy offloading +Bugzilla: 1623082 +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Laurent Vivier + +From: Fam Zheng + +Signed-off-by: Fam Zheng +Signed-off-by: Kevin Wolf +(cherry picked from commit e11ce12f5eb26438419e486a3ae2c9bb58a23c1f) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + qemu-img-cmds.hx + qemu-img.c + qemu-img.texi + + All relating to downstream addition of -s + argument not present when -C was added. + +Signed-off-by: Danilo C. L. de Paula +--- + qemu-img-cmds.hx | 2 +- + qemu-img.c | 21 +++++++++++++++++---- + qemu-img.texi | 8 +++++++- + 3 files changed, 25 insertions(+), 6 deletions(-) + +diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx +index 2fe3189..48815c0 100644 +--- a/qemu-img-cmds.hx ++++ b/qemu-img-cmds.hx +@@ -41,7 +41,7 @@ STEXI + ETEXI + + DEF("convert", img_convert, +- "convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-s snapshot_id_or_name] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] filename [filename2 [...]] output_filename") ++ "convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-s snapshot_id_or_name] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] filename [filename2 [...]] output_filename") + STEXI + @item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-s @var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename} + ETEXI +diff --git a/qemu-img.c b/qemu-img.c +index b9bd401..f42750a 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -2000,11 +2000,12 @@ static int img_convert(int argc, char **argv) + skip_create = false, progress = false, tgt_image_opts = false; + int64_t ret = -EINVAL; + bool force_share = false; ++ bool explict_min_sparse = false; + + ImgConvertState s = (ImgConvertState) { + /* Need at least 4k of zeros for sparse detection */ + .min_sparse = 8, +- .copy_range = true, ++ .copy_range = false, + .buf_sectors = IO_BUF_SIZE / BDRV_SECTOR_SIZE, + .wr_in_order = true, + .num_coroutines = 8, +@@ -2019,7 +2020,7 @@ static int img_convert(int argc, char **argv) + {"target-image-opts", no_argument, 0, OPTION_TARGET_IMAGE_OPTS}, + {0, 0, 0, 0} + }; +- c = getopt_long(argc, argv, ":hf:O:B:co:s:l:S:pt:T:qnm:WU", ++ c = getopt_long(argc, argv, ":hf:O:B:Cco:s:l:S:pt:T:qnm:WU", + long_options, NULL); + if (c == -1) { + break; +@@ -2043,9 +2044,11 @@ static int img_convert(int argc, char **argv) + case 'B': + out_baseimg = optarg; + break; ++ case 'C': ++ s.copy_range = true; ++ break; + case 'c': + s.compressed = true; +- s.copy_range = false; + break; + case 'o': + if (!is_valid_option_list(optarg)) { +@@ -2087,7 +2090,7 @@ static int img_convert(int argc, char **argv) + } + + s.min_sparse = sval / BDRV_SECTOR_SIZE; +- s.copy_range = false; ++ explict_min_sparse = true; + break; + } + case 'p': +@@ -2152,6 +2155,16 @@ static int img_convert(int argc, char **argv) + goto fail_getopt; + } + ++ if (s.compressed && s.copy_range) { ++ error_report("Cannot enable copy offloading when -c is used"); ++ goto fail_getopt; ++ } ++ ++ if (explict_min_sparse && s.copy_range) { ++ error_report("Cannot enable copy offloading when -S is used"); ++ goto fail_getopt; ++ } ++ + if (tgt_image_opts && !skip_create) { + error_report("--target-image-opts requires use of -n flag"); + goto fail_getopt; +diff --git a/qemu-img.texi b/qemu-img.texi +index 8a26400..74d7290 100644 +--- a/qemu-img.texi ++++ b/qemu-img.texi +@@ -172,6 +172,12 @@ Number of parallel coroutines for the convert process + Allow out-of-order writes to the destination. This option improves performance, + but is only recommended for preallocated devices like host devices or other + raw block devices. ++@item -C ++Try to use copy offloading to move data from source image to target. This may ++improve performance if the data is remote, such as with NFS or iSCSI backends, ++but will not automatically sparsify zero sectors, and may result in a fully ++allocated target image depending on the host support for getting allocation ++information. + @end table + + Parameters to dd subcommand: +@@ -340,7 +346,7 @@ Error on reading data + + @end table + +-@item convert [-c] [-p] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-s @var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-m @var{num_coroutines}] [-W] [-S @var{sparse_size}] @var{filename} [@var{filename2} [...]] @var{output_filename} ++@item convert [-C] [-c] [-p] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-s @var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-m @var{num_coroutines}] [-W] [-S @var{sparse_size}] @var{filename} [@var{filename2} [...]] @var{output_filename} + + Convert the disk image @var{filename} or a snapshot @var{snapshot_param}(@var{snapshot_id_or_name} is deprecated) + to disk image @var{output_filename} using format @var{output_fmt}. It can be optionally compressed (@code{-c} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Add-print_amend_option_help.patch b/SOURCES/kvm-qemu-img-Add-print_amend_option_help.patch new file mode 100644 index 0000000..0c30551 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Add-print_amend_option_help.patch @@ -0,0 +1,238 @@ +From 267e29969f1e835db7a3fb7d5f9e9befe6e8dcad Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:59:40 +0200 +Subject: [PATCH 067/268] qemu-img: Add print_amend_option_help() + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-5-mreitz@redhat.com> +Patchwork-id: 80753 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 4/7] qemu-img: Add print_amend_option_help() +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +The more generic print_block_option_help() function is not really +suitable for qemu-img amend, for a couple of reasons: +(1) We do not need to append the protocol-level options, as amendment + happens only on one node and does not descend downwards to its + children. +(2) print_block_option_help() says those options are "supported". For + option amendment, we do not really know that. So this new function + explicitly says that those options are the creation options, and not + all of them may be supported. +(3) If the driver does not support option amendment, we should not print + anything (except for an error message that amendment is not + supported). + +Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1537956 +Signed-off-by: Max Reitz +Reviewed-by: John Snow +Reviewed-by: Eric Blake +Message-id: 20180509210023.20283-5-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 51641351420e4f9dc6613d0dd03a5f0f214ed5b6) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 30 ++++++++++++++++++++++++++++-- + tests/qemu-iotests/082.out | 44 +++++++++++++++++++++++++++----------------- + 2 files changed, 55 insertions(+), 19 deletions(-) + +diff --git a/qemu-img.c b/qemu-img.c +index cdeb01b..d4cbb63 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -3609,6 +3609,32 @@ static void amend_status_cb(BlockDriverState *bs, + qemu_progress_print(100.f * offset / total_work_size, 0); + } + ++static int print_amend_option_help(const char *format) ++{ ++ BlockDriver *drv; ++ ++ /* Find driver and parse its options */ ++ drv = bdrv_find_format(format); ++ if (!drv) { ++ error_report("Unknown file format '%s'", format); ++ return 1; ++ } ++ ++ if (!drv->bdrv_amend_options) { ++ error_report("Format driver '%s' does not support option amendment", ++ format); ++ return 1; ++ } ++ ++ /* Every driver supporting amendment must have create_opts */ ++ assert(drv->create_opts); ++ ++ printf("Creation options for '%s':\n", format); ++ qemu_opts_print_help(drv->create_opts); ++ printf("\nNote that not all of these options may be amendable.\n"); ++ return 0; ++} ++ + static int img_amend(int argc, char **argv) + { + Error *err = NULL; +@@ -3708,7 +3734,7 @@ static int img_amend(int argc, char **argv) + if (fmt && has_help_option(options)) { + /* If a format is explicitly specified (and possibly no filename is + * given), print option help here */ +- ret = print_block_option_help(filename, fmt); ++ ret = print_amend_option_help(fmt); + goto out; + } + +@@ -3737,7 +3763,7 @@ static int img_amend(int argc, char **argv) + + if (has_help_option(options)) { + /* If the format was auto-detected, print option help here */ +- ret = print_block_option_help(filename, fmt); ++ ret = print_amend_option_help(fmt); + goto out; + } + +diff --git a/tests/qemu-iotests/082.out b/tests/qemu-iotests/082.out +index 1527fbe..4e52dce 100644 +--- a/tests/qemu-iotests/082.out ++++ b/tests/qemu-iotests/082.out +@@ -546,7 +546,7 @@ cluster_size: 65536 + === amend: help for -o === + + Testing: amend -f qcow2 -o help TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -564,10 +564,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o ? TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -585,10 +586,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o cluster_size=4k,help TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -606,10 +608,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o cluster_size=4k,? TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -627,10 +630,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o help,cluster_size=4k TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -648,10 +652,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o ?,cluster_size=4k TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -669,10 +674,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o cluster_size=4k -o help TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -690,10 +696,11 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o cluster_size=4k -o ? TEST_DIR/t.qcow2 +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -711,7 +718,8 @@ cluster_size qcow2 cluster size + preallocation Preallocation mode (allowed values: off, metadata, falloc, full) + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits +-nocow Turn off copy-on-write (valid only on btrfs) ++ ++Note that not all of these options may be amendable. + + Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2,,help TEST_DIR/t.qcow2 + +@@ -731,7 +739,7 @@ Testing: amend -f qcow2 -o backing_file=TEST_DIR/t.qcow2 -o ,, -o help TEST_DIR/ + qemu-img: Invalid option list: ,, + + Testing: amend -f qcow2 -o help +-Supported options: ++Creation options for 'qcow2': + size Virtual disk size + compat Compatibility level (0.10 or 1.1) + backing_file File name of a base image +@@ -750,6 +758,8 @@ preallocation Preallocation mode (allowed values: off, metadata, falloc, full + lazy_refcounts Postpone refcount updates + refcount_bits Width of a reference count entry in bits + ++Note that not all of these options may be amendable. ++ + Testing: convert -o help + Supported options: + size Virtual disk size +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Amendment-support-implies-create_opts.patch b/SOURCES/kvm-qemu-img-Amendment-support-implies-create_opts.patch new file mode 100644 index 0000000..d366fa8 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Amendment-support-implies-create_opts.patch @@ -0,0 +1,61 @@ +From 73102ec1ec9d6f19758734652b4872b9abb249c5 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:59:37 +0200 +Subject: [PATCH 064/268] qemu-img: Amendment support implies create_opts + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-2-mreitz@redhat.com> +Patchwork-id: 80758 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/7] qemu-img: Amendment support implies create_opts +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +Instead of checking whether a driver has a non-NULL create_opts we +should check whether it supports image amendment in the first place. If +it does, it must have create_opts. + +On the other hand, if it does not have create_opts (so it does not +support amendment either), the error message "does not support any +options" is a bit useless. Stating clearly that the driver has no +amendment support whatsoever is probably better. + +Signed-off-by: Max Reitz +Reviewed-by: John Snow +Reviewed-by: Eric Blake +Message-id: 20180509210023.20283-2-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 1f996683ad908fd41c572221a63d9fc31ce04d07) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 7 +++++-- + 1 file changed, 5 insertions(+), 2 deletions(-) + +diff --git a/qemu-img.c b/qemu-img.c +index 60e45ec..2f7c491 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -3740,13 +3740,16 @@ static int img_amend(int argc, char **argv) + goto out; + } + +- if (!bs->drv->create_opts) { +- error_report("Format driver '%s' does not support any options to amend", ++ if (!bs->drv->bdrv_amend_options) { ++ error_report("Format driver '%s' does not support option amendment", + fmt); + ret = -1; + goto out; + } + ++ /* Every driver supporting amendment must have create_opts */ ++ assert(bs->drv->create_opts); ++ + create_opts = qemu_opts_append(create_opts, bs->drv->create_opts); + opts = qemu_opts_create(create_opts, NULL, 0, &error_abort); + qemu_opts_do_parse(opts, options, NULL, &err); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Check-post-truncation-size.patch b/SOURCES/kvm-qemu-img-Check-post-truncation-size.patch new file mode 100644 index 0000000..a8996f1 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Check-post-truncation-size.patch @@ -0,0 +1,100 @@ +From a4029c787c5741de154d9d1ede7e70c7deaba416 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Wed, 16 May 2018 12:00:18 +0200 +Subject: [PATCH 003/268] qemu-img: Check post-truncation size + +RH-Author: Max Reitz +Message-id: <20180516120018.27565-2-mreitz@redhat.com> +Patchwork-id: 80281 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/1] qemu-img: Check post-truncation size +Bugzilla: 1523065 +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf + +Some block drivers (iscsi and file-posix when dealing with device files) +do not actually support truncation, even though they provide a +.bdrv_truncate() method and will happily return success when providing a +new size that does not exceed the current size. This is because these +drivers expect the user to resize the image outside of qemu and then +provide qemu with that information through the block_resize command +(compare cb1b83e740384b4e0d950f3d7c81c02b8ce86c2e). + +Of course, anyone using qemu-img resize will find that behavior useless. +So we should check the actual size of the image after the supposedly +successful truncation took place, emit an error if nothing changed and +emit a warning if the target size was not met. + +Signed-off-by: Max Reitz +Message-id: 20180421163957.29872-1-mreitz@redhat.com +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Max Reitz +(cherry picked from commit 5279b30392da7a3248b320c75f20c61e3a95863c) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 39 +++++++++++++++++++++++++++++++++++---- + 1 file changed, 35 insertions(+), 4 deletions(-) + +diff --git a/qemu-img.c b/qemu-img.c +index 855fa52..8320887 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -3381,7 +3381,7 @@ static int img_resize(int argc, char **argv) + Error *err = NULL; + int c, ret, relative; + const char *filename, *fmt, *size; +- int64_t n, total_size, current_size; ++ int64_t n, total_size, current_size, new_size; + bool quiet = false; + BlockBackend *blk = NULL; + PreallocMode prealloc = PREALLOC_MODE_OFF; +@@ -3557,11 +3557,42 @@ static int img_resize(int argc, char **argv) + } + + ret = blk_truncate(blk, total_size, prealloc, &err); +- if (!ret) { +- qprintf(quiet, "Image resized.\n"); +- } else { ++ if (ret < 0) { + error_report_err(err); ++ goto out; ++ } ++ ++ new_size = blk_getlength(blk); ++ if (new_size < 0) { ++ error_report("Failed to verify truncated image length: %s", ++ strerror(-new_size)); ++ ret = -1; ++ goto out; + } ++ ++ /* Some block drivers implement a truncation method, but only so ++ * the user can cause qemu to refresh the image's size from disk. ++ * The idea is that the user resizes the image outside of qemu and ++ * then invokes block_resize to inform qemu about it. ++ * (This includes iscsi and file-posix for device files.) ++ * Of course, that is not the behavior someone invoking ++ * qemu-img resize would find useful, so we catch that behavior ++ * here and tell the user. */ ++ if (new_size != total_size && new_size == current_size) { ++ error_report("Image was not resized; resizing may not be supported " ++ "for this image"); ++ ret = -1; ++ goto out; ++ } ++ ++ if (new_size != total_size) { ++ warn_report("Image should have been resized to %" PRIi64 ++ " bytes, but was resized to %" PRIi64 " bytes", ++ total_size, new_size); ++ } ++ ++ qprintf(quiet, "Image resized.\n"); ++ + out: + blk_unref(blk); + if (ret) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Convert-with-copy-offloading.patch b/SOURCES/kvm-qemu-img-Convert-with-copy-offloading.patch new file mode 100644 index 0000000..4b354a5 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Convert-with-copy-offloading.patch @@ -0,0 +1,145 @@ +From d4702dc3b114f30be96a1a2b9fe4f7f05fef1fc0 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Wed, 4 Jul 2018 07:56:31 +0200 +Subject: [PATCH 176/268] qemu-img: Convert with copy offloading + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-11-famz@redhat.com> +Patchwork-id: 81158 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 10/13] qemu-img: Convert with copy offloading +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +The new blk_co_copy_range interface offers a more efficient way in the +case of network based storage. Make use of it to allow faster convert +operation. + +Since copy offloading cannot do zero detection ('-S') and compression +(-c), only try it when these options are not used. + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Message-id: 20180601092648.24614-11-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit ee5306d0923377439776e8a30b9fd2de34b5cbfb) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++-- + 1 file changed, 48 insertions(+), 2 deletions(-) + +diff --git a/qemu-img.c b/qemu-img.c +index 9fc8e66..b2ef54e 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -1561,6 +1561,7 @@ typedef struct ImgConvertState { + bool target_has_backing; + int64_t target_backing_sectors; /* negative if unknown */ + bool wr_in_order; ++ bool copy_range; + int min_sparse; + size_t cluster_sectors; + size_t buf_sectors; +@@ -1765,6 +1766,37 @@ static int coroutine_fn convert_co_write(ImgConvertState *s, int64_t sector_num, + return 0; + } + ++static int coroutine_fn convert_co_copy_range(ImgConvertState *s, int64_t sector_num, ++ int nb_sectors) ++{ ++ int n, ret; ++ ++ while (nb_sectors > 0) { ++ BlockBackend *blk; ++ int src_cur; ++ int64_t bs_sectors, src_cur_offset; ++ int64_t offset; ++ ++ convert_select_part(s, sector_num, &src_cur, &src_cur_offset); ++ offset = (sector_num - src_cur_offset) << BDRV_SECTOR_BITS; ++ blk = s->src[src_cur]; ++ bs_sectors = s->src_sectors[src_cur]; ++ ++ n = MIN(nb_sectors, bs_sectors - (sector_num - src_cur_offset)); ++ ++ ret = blk_co_copy_range(blk, offset, s->target, ++ sector_num << BDRV_SECTOR_BITS, ++ n << BDRV_SECTOR_BITS, 0); ++ if (ret < 0) { ++ return ret; ++ } ++ ++ sector_num += n; ++ nb_sectors -= n; ++ } ++ return 0; ++} ++ + static void coroutine_fn convert_co_do_copy(void *opaque) + { + ImgConvertState *s = opaque; +@@ -1787,6 +1819,7 @@ static void coroutine_fn convert_co_do_copy(void *opaque) + int n; + int64_t sector_num; + enum ImgConvertBlockStatus status; ++ bool copy_range; + + qemu_co_mutex_lock(&s->lock); + if (s->ret != -EINPROGRESS || s->sector_num >= s->total_sectors) { +@@ -1816,7 +1849,9 @@ static void coroutine_fn convert_co_do_copy(void *opaque) + s->allocated_sectors, 0); + } + +- if (status == BLK_DATA) { ++retry: ++ copy_range = s->copy_range && s->status == BLK_DATA; ++ if (status == BLK_DATA && !copy_range) { + ret = convert_co_read(s, sector_num, n, buf); + if (ret < 0) { + error_report("error while reading sector %" PRId64 +@@ -1838,7 +1873,15 @@ static void coroutine_fn convert_co_do_copy(void *opaque) + } + + if (s->ret == -EINPROGRESS) { +- ret = convert_co_write(s, sector_num, n, buf, status); ++ if (copy_range) { ++ ret = convert_co_copy_range(s, sector_num, n); ++ if (ret) { ++ s->copy_range = false; ++ goto retry; ++ } ++ } else { ++ ret = convert_co_write(s, sector_num, n, buf, status); ++ } + if (ret < 0) { + error_report("error while writing sector %" PRId64 + ": %s", sector_num, strerror(-ret)); +@@ -1961,6 +2004,7 @@ static int img_convert(int argc, char **argv) + ImgConvertState s = (ImgConvertState) { + /* Need at least 4k of zeros for sparse detection */ + .min_sparse = 8, ++ .copy_range = true, + .buf_sectors = IO_BUF_SIZE / BDRV_SECTOR_SIZE, + .wr_in_order = true, + .num_coroutines = 8, +@@ -2001,6 +2045,7 @@ static int img_convert(int argc, char **argv) + break; + case 'c': + s.compressed = true; ++ s.copy_range = false; + break; + case 'o': + if (!is_valid_option_list(optarg)) { +@@ -2042,6 +2087,7 @@ static int img_convert(int argc, char **argv) + } + + s.min_sparse = sval / BDRV_SECTOR_SIZE; ++ s.copy_range = false; + break; + } + case 'p': +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Fix-assert-when-mapping-unaligned-raw-file.patch b/SOURCES/kvm-qemu-img-Fix-assert-when-mapping-unaligned-raw-file.patch new file mode 100644 index 0000000..2e750b7 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Fix-assert-when-mapping-unaligned-raw-file.patch @@ -0,0 +1,61 @@ +From 0ad5340a73448535ca73d6e551955e2339c9e110 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 15 Oct 2018 17:44:45 +0100 +Subject: [PATCH 48/49] qemu-img: Fix assert when mapping unaligned raw file + +RH-Author: Max Reitz +Message-id: <20181015174446.31974-2-mreitz@redhat.com> +Patchwork-id: 82706 +O-Subject: [RHEL-8 qemu-kvm PATCH v2 1/2] qemu-img: Fix assert when mapping unaligned raw file +Bugzilla: 1639374 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Thomas Huth + +From: Eric Blake + +Commit a290f085 exposed a latent bug in qemu-img map introduced +during the conversion of block status to be byte-based. Earlier in +commit 5e344dd8, the internal interface get_block_status() switched +to take byte-based parameters, but still called a sector-based +block layer function; as such, rounding was added in the lone +caller to obey the contract. However, commit 237d78f8 changed +get_block_status() to truly be byte-based, at which point rounding +to sector boundaries can result in calling bdrv_block_status() with +'bytes == 0' (a coding error) when the boundary between data and a +hole falls mid-sector (true for the past-EOF implicit hole present +in POSIX files). Fix things by removing the rounding that is now +no longer necessary. + +See also https://bugzilla.redhat.com/1589738 + +Fixes: 237d78f8 +Reported-by: Dan Kenigsberg +Reported-by: Nir Soffer +Reported-by: Maor Lipchuk +CC: qemu-stable@nongnu.org +Signed-off-by: Eric Blake +Signed-off-by: Kevin Wolf +(cherry picked from commit e0b371ed5e2db079051139136fd0478728b6a58f) +Signed-off-by: Max Reitz +Signed-off-by: Danilo C. L. de Paula +--- + qemu-img.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/qemu-img.c b/qemu-img.c +index eaee6d6..b9bd401 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -2912,7 +2912,7 @@ static int img_map(int argc, char **argv) + int64_t n; + + /* Probe up to 1 GiB at a time. */ +- n = QEMU_ALIGN_DOWN(MIN(1 << 30, length - offset), BDRV_SECTOR_SIZE); ++ n = MIN(1 << 30, length - offset); + ret = get_block_status(bs, offset, n, &next); + + if (ret < 0) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Recognize-no-creation-support-in-o-help.patch b/SOURCES/kvm-qemu-img-Recognize-no-creation-support-in-o-help.patch new file mode 100644 index 0000000..42de472 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Recognize-no-creation-support-in-o-help.patch @@ -0,0 +1,74 @@ +From bdd2925eaebdbba0400da3ebef201f402c3cee90 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:59:41 +0200 +Subject: [PATCH 068/268] qemu-img: Recognize no creation support in -o help + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-6-mreitz@redhat.com> +Patchwork-id: 80759 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 5/7] qemu-img: Recognize no creation support in -o help +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +The only users of print_block_option_help() are qemu-img create and +qemu-img convert for the output image, so this function is always used +for image creation (it used to be used for amendment also, but that is +no longer the case). + +So if image creation is not supported by either the format or the +protocol, there is no need to print any option description, because the +user cannot create an image like this anyway. + +This also fixes an assertion failure: + + $ qemu-img create -f bochs -o help + Supported options: + qemu-img: util/qemu-option.c:219: + qemu_opts_print_help: Assertion `list' failed. + [1] 24831 abort (core dumped) qemu-img create -f bochs -o help + +Signed-off-by: Max Reitz +Reviewed-by: John Snow +Reviewed-by: Eric Blake +Message-id: 20180509210023.20283-6-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit d402b6a21a825a5c07aac9251990860723d49f5d) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 10 ++++++++++ + 1 file changed, 10 insertions(+) + +diff --git a/qemu-img.c b/qemu-img.c +index d4cbb63..4bcf157 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -250,6 +250,11 @@ static int print_block_option_help(const char *filename, const char *fmt) + return 1; + } + ++ if (!drv->create_opts) { ++ error_report("Format driver '%s' does not support image creation", fmt); ++ return 1; ++ } ++ + create_opts = qemu_opts_append(create_opts, drv->create_opts); + if (filename) { + proto_drv = bdrv_find_protocol(filename, true, &local_err); +@@ -258,6 +263,11 @@ static int print_block_option_help(const char *filename, const char *fmt) + qemu_opts_free(create_opts); + return 1; + } ++ if (!proto_drv->create_opts) { ++ error_report("Protocal driver '%s' does not support image creation", ++ proto_drv->format_name); ++ return 1; ++ } + create_opts = qemu_opts_append(create_opts, proto_drv->create_opts); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Resolve-relative-backing-paths-in-rebase.patch b/SOURCES/kvm-qemu-img-Resolve-relative-backing-paths-in-rebase.patch new file mode 100644 index 0000000..a796ddc --- /dev/null +++ b/SOURCES/kvm-qemu-img-Resolve-relative-backing-paths-in-rebase.patch @@ -0,0 +1,90 @@ +From 68af7bd1c3085fb25f50a7ad8cada0c33dbba748 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 17:48:32 +0200 +Subject: [PATCH 071/268] qemu-img: Resolve relative backing paths in rebase + +RH-Author: Max Reitz +Message-id: <20180618174833.19439-2-mreitz@redhat.com> +Patchwork-id: 80790 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/2] qemu-img: Resolve relative backing paths in rebase +Bugzilla: 1569835 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +Currently, rebase interprets a relative path for the new backing image +as follows: +(1) Open the new backing image with the given relative path (thus relative to + qemu-img's working directory). +(2) Write it directly into the overlay's backing path field (thus + relative to the overlay). + +If the overlay is not in qemu-img's working directory, both will be +different interpretations, which may either lead to an error somewhere +(either rebase fails because it cannot open the new backing image, or +your overlay becomes unusable because its backing path does not point to +a file), or, even worse, it may result in your rebase being performed +for a different backing file than what your overlay will point to after +the rebase. + +Fix this by interpreting the target backing path as relative to the +overlay, like qemu-img does everywhere else. + +Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1569835 +Cc: qemu-stable@nongnu.org +Signed-off-by: Max Reitz +Message-id: 20180509182002.8044-2-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit d16699b64671466b42079c45b89127aeea1ca565) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 23 ++++++++++++++++++++++- + 1 file changed, 22 insertions(+), 1 deletion(-) + +diff --git a/qemu-img.c b/qemu-img.c +index 4bcf157..76d6e47 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -3202,6 +3202,9 @@ static int img_rebase(int argc, char **argv) + } + + if (out_baseimg[0]) { ++ const char *overlay_filename; ++ char *out_real_path; ++ + options = qdict_new(); + if (out_basefmt) { + qdict_put_str(options, "driver", out_basefmt); +@@ -3210,8 +3213,26 @@ static int img_rebase(int argc, char **argv) + qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true); + } + +- blk_new_backing = blk_new_open(out_baseimg, NULL, ++ overlay_filename = bs->exact_filename[0] ? bs->exact_filename ++ : bs->filename; ++ out_real_path = g_malloc(PATH_MAX); ++ ++ bdrv_get_full_backing_filename_from_filename(overlay_filename, ++ out_baseimg, ++ out_real_path, ++ PATH_MAX, ++ &local_err); ++ if (local_err) { ++ error_reportf_err(local_err, ++ "Could not resolve backing filename: "); ++ ret = -1; ++ g_free(out_real_path); ++ goto out; ++ } ++ ++ blk_new_backing = blk_new_open(out_real_path, NULL, + options, src_flags, &local_err); ++ g_free(out_real_path); + if (!blk_new_backing) { + error_reportf_err(local_err, + "Could not open new backing file '%s': ", +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Special-post-backing-convert-handling.patch b/SOURCES/kvm-qemu-img-Special-post-backing-convert-handling.patch new file mode 100644 index 0000000..0399176 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Special-post-backing-convert-handling.patch @@ -0,0 +1,113 @@ +From 25b5ec4399eec0a03d91a103658a4fba4263ba4d Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 18:04:50 +0200 +Subject: [PATCH 073/268] qemu-img: Special post-backing convert handling + +RH-Author: Max Reitz +Message-id: <20180618180451.23808-2-mreitz@redhat.com> +Patchwork-id: 80795 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/2] qemu-img: Special post-backing convert handling +Bugzilla: 1527898 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Jeffrey Cody + +Currently, qemu-img convert writes zeroes when it reads zeroes. +Sometimes it does not because the target is initialized to zeroes +anyway, so we do not need to overwrite (and thus potentially allocate) +it. This is never the case for targets with backing files, though. But +even they may have an area that is initialized to zeroes, and that is +the area past the end of the backing file (if that is shorter than the +overlay). + +So if the target format's unallocated blocks are zero and there is a gap +between the target's backing file's end and the target's end, we do not +have to explicitly write zeroes there. + +Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1527898 +Signed-off-by: Max Reitz +Message-id: 20180501165750.19242-2-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 351c8efff9ad809c822d55620df54d575d536f68) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 26 +++++++++++++++++++++++++- + 1 file changed, 25 insertions(+), 1 deletion(-) + +diff --git a/qemu-img.c b/qemu-img.c +index 76d6e47..e2395b9 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -1553,7 +1553,9 @@ typedef struct ImgConvertState { + BlockBackend *target; + bool has_zero_init; + bool compressed; ++ bool unallocated_blocks_are_zero; + bool target_has_backing; ++ int64_t target_backing_sectors; /* negative if unknown */ + bool wr_in_order; + int min_sparse; + size_t cluster_sectors; +@@ -1582,12 +1584,23 @@ static int convert_iteration_sectors(ImgConvertState *s, int64_t sector_num) + { + int64_t src_cur_offset; + int ret, n, src_cur; ++ bool post_backing_zero = false; + + convert_select_part(s, sector_num, &src_cur, &src_cur_offset); + + assert(s->total_sectors > sector_num); + n = MIN(s->total_sectors - sector_num, BDRV_REQUEST_MAX_SECTORS); + ++ if (s->target_backing_sectors >= 0) { ++ if (sector_num >= s->target_backing_sectors) { ++ post_backing_zero = s->unallocated_blocks_are_zero; ++ } else if (sector_num + n > s->target_backing_sectors) { ++ /* Split requests around target_backing_sectors (because ++ * starting from there, zeros are handled differently) */ ++ n = s->target_backing_sectors - sector_num; ++ } ++ } ++ + if (s->sector_next_status <= sector_num) { + int64_t count = n * BDRV_SECTOR_SIZE; + +@@ -1609,7 +1622,7 @@ static int convert_iteration_sectors(ImgConvertState *s, int64_t sector_num) + n = DIV_ROUND_UP(count, BDRV_SECTOR_SIZE); + + if (ret & BDRV_BLOCK_ZERO) { +- s->status = BLK_ZERO; ++ s->status = post_backing_zero ? BLK_BACKING_FILE : BLK_ZERO; + } else if (ret & BDRV_BLOCK_DATA) { + s->status = BLK_DATA; + } else { +@@ -2330,6 +2343,16 @@ static int img_convert(int argc, char **argv) + } + } + ++ if (s.target_has_backing) { ++ /* Errors are treated as "backing length unknown" (which means ++ * s.target_backing_sectors has to be negative, which it will ++ * be automatically). The backing file length is used only ++ * for optimizations, so such a case is not fatal. */ ++ s.target_backing_sectors = bdrv_nb_sectors(out_bs->backing->bs); ++ } else { ++ s.target_backing_sectors = -1; ++ } ++ + ret = bdrv_get_info(out_bs, &bdi); + if (ret < 0) { + if (s.compressed) { +@@ -2339,6 +2362,7 @@ static int img_convert(int argc, char **argv) + } else { + s.compressed = s.compressed || bdi.needs_compressed_writes; + s.cluster_sectors = bdi.cluster_size / BDRV_SECTOR_SIZE; ++ s.unallocated_blocks_are_zero = bdi.unallocated_blocks_are_zero; + } + + ret = convert_do_copy(&s); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-img-Use-only-string-options-in-img_open_opts.patch b/SOURCES/kvm-qemu-img-Use-only-string-options-in-img_open_opts.patch new file mode 100644 index 0000000..025e3a0 --- /dev/null +++ b/SOURCES/kvm-qemu-img-Use-only-string-options-in-img_open_opts.patch @@ -0,0 +1,61 @@ +From f939ed0a804f86ba8cde7f2401ade2a3b108c81d Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 25 Jun 2018 13:06:56 +0200 +Subject: [PATCH 047/268] qemu-img: Use only string options in img_open_opts + +RH-Author: Max Reitz +Message-id: <20180618163106.23010-3-mreitz@redhat.com> +Patchwork-id: 80774 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/3] qemu-img: Use only string options in img_open_opts +Bugzilla: 1576598 +RH-Acked-by: Kevin Wolf +RH-Acked-by: Fam Zheng +RH-Acked-by: Miroslav Rezanina + +img_open_opts() takes a QemuOpts and converts them to a QDict, so all +values therein are strings. Then it may try to call qdict_get_bool(), +however, which will fail with a segmentation fault every time: + +$ ./qemu-img info -U --image-opts \ + driver=file,filename=/dev/null,force-share=off +[1] 27869 segmentation fault (core dumped) ./qemu-img info -U +--image-opts driver=file,filename=/dev/null,force-share=off + +Fix this by using qdict_get_str() and comparing the value as a string. +Also, when adding a force-share value to the QDict, add it as a string +so it fits the rest of the dict. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Max Reitz +Message-id: 20180502202051.15493-3-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 4615f87832d2fcb7a544bedeece2741bf8c21f94) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/qemu-img.c b/qemu-img.c +index 62b29e7..60e45ec 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -277,12 +277,12 @@ static BlockBackend *img_open_opts(const char *optstr, + options = qemu_opts_to_qdict(opts, NULL); + if (force_share) { + if (qdict_haskey(options, BDRV_OPT_FORCE_SHARE) +- && !qdict_get_bool(options, BDRV_OPT_FORCE_SHARE)) { ++ && strcmp(qdict_get_str(options, BDRV_OPT_FORCE_SHARE), "on")) { + error_report("--force-share/-U conflicts with image options"); + qobject_unref(options); + return NULL; + } +- qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true); ++ qdict_put_str(options, BDRV_OPT_FORCE_SHARE, "on"); + } + blk = blk_new_open(NULL, NULL, options, flags, &local_err); + if (!blk) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-io-Drop-command-functions-return-values.patch b/SOURCES/kvm-qemu-io-Drop-command-functions-return-values.patch new file mode 100644 index 0000000..6ab4aff --- /dev/null +++ b/SOURCES/kvm-qemu-io-Drop-command-functions-return-values.patch @@ -0,0 +1,1336 @@ +From fed57d13b5e402e6d385189acb78aa5f84343fe4 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 25 Jun 2018 13:09:51 +0200 +Subject: [PATCH 049/268] qemu-io: Drop command functions' return values + +RH-Author: Max Reitz +Message-id: <20180618164312.24423-2-mreitz@redhat.com> +Patchwork-id: 80778 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/5] qemu-io: Drop command functions' return values +Bugzilla: 1519617 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +For qemu-io, a function returns an integer with two possible values: 0 +for "qemu-io may continue execution", or 1 for "qemu-io should exit". +However, there is only a single command that returns 1, and that is +"quit". + +So let's turn this case into a global variable instead so we can make +better use of the return value in a later patch. + +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509194302.21585-2-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit b444d0e9d1ae323fed1ef7c35a359ce064f36b32) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + include/qemu-io.h | 6 +- + qemu-io-cmds.c | 294 +++++++++++++++++++++++++----------------------------- + qemu-io.c | 36 +++---- + 3 files changed, 157 insertions(+), 179 deletions(-) + +diff --git a/include/qemu-io.h b/include/qemu-io.h +index 196fde0..06cdfbf 100644 +--- a/include/qemu-io.h ++++ b/include/qemu-io.h +@@ -22,7 +22,7 @@ + + #define CMD_FLAG_GLOBAL ((int)0x80000000) /* don't iterate "args" */ + +-typedef int (*cfunc_t)(BlockBackend *blk, int argc, char **argv); ++typedef void (*cfunc_t)(BlockBackend *blk, int argc, char **argv); + typedef void (*helpfunc_t)(void); + + typedef struct cmdinfo { +@@ -41,10 +41,10 @@ typedef struct cmdinfo { + + extern bool qemuio_misalign; + +-bool qemuio_command(BlockBackend *blk, const char *cmd); ++void qemuio_command(BlockBackend *blk, const char *cmd); + + void qemuio_add_command(const cmdinfo_t *ci); +-int qemuio_command_usage(const cmdinfo_t *ci); ++void qemuio_command_usage(const cmdinfo_t *ci); + void qemuio_complete_command(const char *input, + void (*fn)(const char *cmd, void *opaque), + void *opaque); +diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c +index 9b3cd00..c2fbaae 100644 +--- a/qemu-io-cmds.c ++++ b/qemu-io-cmds.c +@@ -48,10 +48,9 @@ void qemuio_add_command(const cmdinfo_t *ci) + qsort(cmdtab, ncmds, sizeof(*cmdtab), compare_cmdname); + } + +-int qemuio_command_usage(const cmdinfo_t *ci) ++void qemuio_command_usage(const cmdinfo_t *ci) + { + printf("%s %s -- %s\n", ci->name, ci->args, ci->oneline); +- return 0; + } + + static int init_check_command(BlockBackend *blk, const cmdinfo_t *ct) +@@ -66,13 +65,13 @@ static int init_check_command(BlockBackend *blk, const cmdinfo_t *ct) + return 1; + } + +-static int command(BlockBackend *blk, const cmdinfo_t *ct, int argc, +- char **argv) ++static void command(BlockBackend *blk, const cmdinfo_t *ct, int argc, ++ char **argv) + { + char *cmd = argv[0]; + + if (!init_check_command(blk, ct)) { +- return 0; ++ return; + } + + if (argc - 1 < ct->argmin || (ct->argmax != -1 && argc - 1 > ct->argmax)) { +@@ -89,7 +88,7 @@ static int command(BlockBackend *blk, const cmdinfo_t *ct, int argc, + "bad argument count %d to %s, expected between %d and %d arguments\n", + argc-1, cmd, ct->argmin, ct->argmax); + } +- return 0; ++ return; + } + + /* Request additional permissions if necessary for this command. The caller +@@ -109,13 +108,13 @@ static int command(BlockBackend *blk, const cmdinfo_t *ct, int argc, + ret = blk_set_perm(blk, new_perm, orig_shared_perm, &local_err); + if (ret < 0) { + error_report_err(local_err); +- return 0; ++ return; + } + } + } + + optind = 0; +- return ct->cfunc(blk, argc, argv); ++ ct->cfunc(blk, argc, argv); + } + + static const cmdinfo_t *find_command(const char *cmd) +@@ -634,7 +633,7 @@ static void read_help(void) + "\n"); + } + +-static int read_f(BlockBackend *blk, int argc, char **argv); ++static void read_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t read_cmd = { + .name = "read", +@@ -647,7 +646,7 @@ static const cmdinfo_t read_cmd = { + .help = read_help, + }; + +-static int read_f(BlockBackend *blk, int argc, char **argv) ++static void read_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false, vflag = false; +@@ -674,7 +673,7 @@ static int read_f(BlockBackend *blk, int argc, char **argv) + pattern_count = cvtnum(optarg); + if (pattern_count < 0) { + print_cvtnum_err(pattern_count, optarg); +- return 0; ++ return; + } + break; + case 'p': +@@ -684,7 +683,7 @@ static int read_f(BlockBackend *blk, int argc, char **argv) + Pflag = true; + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return 0; ++ return; + } + break; + case 'q': +@@ -695,40 +694,43 @@ static int read_f(BlockBackend *blk, int argc, char **argv) + pattern_offset = cvtnum(optarg); + if (pattern_offset < 0) { + print_cvtnum_err(pattern_offset, optarg); +- return 0; ++ return; + } + break; + case 'v': + vflag = true; + break; + default: +- return qemuio_command_usage(&read_cmd); ++ qemuio_command_usage(&read_cmd); ++ return; + } + } + + if (optind != argc - 2) { +- return qemuio_command_usage(&read_cmd); ++ qemuio_command_usage(&read_cmd); ++ return; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return 0; ++ return; + } + + optind++; + count = cvtnum(argv[optind]); + if (count < 0) { + print_cvtnum_err(count, argv[optind]); +- return 0; ++ return; + } else if (count > BDRV_REQUEST_MAX_BYTES) { + printf("length cannot exceed %" PRIu64 ", given %s\n", + (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]); +- return 0; ++ return; + } + + if (!Pflag && (lflag || sflag)) { +- return qemuio_command_usage(&read_cmd); ++ qemuio_command_usage(&read_cmd); ++ return; + } + + if (!lflag) { +@@ -737,19 +739,19 @@ static int read_f(BlockBackend *blk, int argc, char **argv) + + if ((pattern_count < 0) || (pattern_count + pattern_offset > count)) { + printf("pattern verification range exceeds end of read data\n"); +- return 0; ++ return; + } + + if (bflag) { + if (!QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)) { + printf("%" PRId64 " is not a sector-aligned value for 'offset'\n", + offset); +- return 0; ++ return; + } + if (!QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)) { + printf("%"PRId64" is not a sector-aligned value for 'count'\n", + count); +- return 0; ++ return; + } + } + +@@ -793,8 +795,6 @@ static int read_f(BlockBackend *blk, int argc, char **argv) + + out: + qemu_io_free(buf); +- +- return 0; + } + + static void readv_help(void) +@@ -816,7 +816,7 @@ static void readv_help(void) + "\n"); + } + +-static int readv_f(BlockBackend *blk, int argc, char **argv); ++static void readv_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t readv_cmd = { + .name = "readv", +@@ -828,7 +828,7 @@ static const cmdinfo_t readv_cmd = { + .help = readv_help, + }; + +-static int readv_f(BlockBackend *blk, int argc, char **argv) ++static void readv_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false, vflag = false; +@@ -851,7 +851,7 @@ static int readv_f(BlockBackend *blk, int argc, char **argv) + Pflag = true; + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return 0; ++ return; + } + break; + case 'q': +@@ -861,26 +861,28 @@ static int readv_f(BlockBackend *blk, int argc, char **argv) + vflag = true; + break; + default: +- return qemuio_command_usage(&readv_cmd); ++ qemuio_command_usage(&readv_cmd); ++ return; + } + } + + if (optind > argc - 2) { +- return qemuio_command_usage(&readv_cmd); ++ qemuio_command_usage(&readv_cmd); ++ return; + } + + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return 0; ++ return; + } + optind++; + + nr_iov = argc - optind; + buf = create_iovec(blk, &qiov, &argv[optind], nr_iov, 0xab); + if (buf == NULL) { +- return 0; ++ return; + } + + gettimeofday(&t1, NULL); +@@ -917,7 +919,6 @@ static int readv_f(BlockBackend *blk, int argc, char **argv) + out: + qemu_iovec_destroy(&qiov); + qemu_io_free(buf); +- return 0; + } + + static void write_help(void) +@@ -943,7 +944,7 @@ static void write_help(void) + "\n"); + } + +-static int write_f(BlockBackend *blk, int argc, char **argv); ++static void write_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t write_cmd = { + .name = "write", +@@ -957,7 +958,7 @@ static const cmdinfo_t write_cmd = { + .help = write_help, + }; + +-static int write_f(BlockBackend *blk, int argc, char **argv) ++static void write_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false, bflag = false; +@@ -992,7 +993,7 @@ static int write_f(BlockBackend *blk, int argc, char **argv) + Pflag = true; + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return 0; ++ return; + } + break; + case 'q': +@@ -1005,62 +1006,64 @@ static int write_f(BlockBackend *blk, int argc, char **argv) + zflag = true; + break; + default: +- return qemuio_command_usage(&write_cmd); ++ qemuio_command_usage(&write_cmd); ++ return; + } + } + + if (optind != argc - 2) { +- return qemuio_command_usage(&write_cmd); ++ qemuio_command_usage(&write_cmd); ++ return; + } + + if (bflag && zflag) { + printf("-b and -z cannot be specified at the same time\n"); +- return 0; ++ return; + } + + if ((flags & BDRV_REQ_FUA) && (bflag || cflag)) { + printf("-f and -b or -c cannot be specified at the same time\n"); +- return 0; ++ return; + } + + if ((flags & BDRV_REQ_MAY_UNMAP) && !zflag) { + printf("-u requires -z to be specified\n"); +- return 0; ++ return; + } + + if (zflag && Pflag) { + printf("-z and -P cannot be specified at the same time\n"); +- return 0; ++ return; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return 0; ++ return; + } + + optind++; + count = cvtnum(argv[optind]); + if (count < 0) { + print_cvtnum_err(count, argv[optind]); +- return 0; ++ return; + } else if (count > BDRV_REQUEST_MAX_BYTES) { + printf("length cannot exceed %" PRIu64 ", given %s\n", + (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]); +- return 0; ++ return; + } + + if (bflag || cflag) { + if (!QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)) { + printf("%" PRId64 " is not a sector-aligned value for 'offset'\n", + offset); +- return 0; ++ return; + } + + if (!QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)) { + printf("%"PRId64" is not a sector-aligned value for 'count'\n", + count); +- return 0; ++ return; + } + } + +@@ -1097,8 +1100,6 @@ out: + if (!zflag) { + qemu_io_free(buf); + } +- +- return 0; + } + + static void +@@ -1120,7 +1121,7 @@ writev_help(void) + "\n"); + } + +-static int writev_f(BlockBackend *blk, int argc, char **argv); ++static void writev_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t writev_cmd = { + .name = "writev", +@@ -1133,7 +1134,7 @@ static const cmdinfo_t writev_cmd = { + .help = writev_help, + }; + +-static int writev_f(BlockBackend *blk, int argc, char **argv) ++static void writev_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false; +@@ -1161,29 +1162,31 @@ static int writev_f(BlockBackend *blk, int argc, char **argv) + case 'P': + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return 0; ++ return; + } + break; + default: +- return qemuio_command_usage(&writev_cmd); ++ qemuio_command_usage(&writev_cmd); ++ return; + } + } + + if (optind > argc - 2) { +- return qemuio_command_usage(&writev_cmd); ++ qemuio_command_usage(&writev_cmd); ++ return; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return 0; ++ return; + } + optind++; + + nr_iov = argc - optind; + buf = create_iovec(blk, &qiov, &argv[optind], nr_iov, pattern); + if (buf == NULL) { +- return 0; ++ return; + } + + gettimeofday(&t1, NULL); +@@ -1205,7 +1208,6 @@ static int writev_f(BlockBackend *blk, int argc, char **argv) + out: + qemu_iovec_destroy(&qiov); + qemu_io_free(buf); +- return 0; + } + + struct aio_ctx { +@@ -1320,7 +1322,7 @@ static void aio_read_help(void) + "\n"); + } + +-static int aio_read_f(BlockBackend *blk, int argc, char **argv); ++static void aio_read_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t aio_read_cmd = { + .name = "aio_read", +@@ -1332,7 +1334,7 @@ static const cmdinfo_t aio_read_cmd = { + .help = aio_read_help, + }; + +-static int aio_read_f(BlockBackend *blk, int argc, char **argv) ++static void aio_read_f(BlockBackend *blk, int argc, char **argv) + { + int nr_iov, c; + struct aio_ctx *ctx = g_new0(struct aio_ctx, 1); +@@ -1348,14 +1350,14 @@ static int aio_read_f(BlockBackend *blk, int argc, char **argv) + ctx->pattern = parse_pattern(optarg); + if (ctx->pattern < 0) { + g_free(ctx); +- return 0; ++ return; + } + break; + case 'i': + printf("injecting invalid read request\n"); + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_READ); + g_free(ctx); +- return 0; ++ return; + case 'q': + ctx->qflag = true; + break; +@@ -1364,20 +1366,22 @@ static int aio_read_f(BlockBackend *blk, int argc, char **argv) + break; + default: + g_free(ctx); +- return qemuio_command_usage(&aio_read_cmd); ++ qemuio_command_usage(&aio_read_cmd); ++ return; + } + } + + if (optind > argc - 2) { + g_free(ctx); +- return qemuio_command_usage(&aio_read_cmd); ++ qemuio_command_usage(&aio_read_cmd); ++ return; + } + + ctx->offset = cvtnum(argv[optind]); + if (ctx->offset < 0) { + print_cvtnum_err(ctx->offset, argv[optind]); + g_free(ctx); +- return 0; ++ return; + } + optind++; + +@@ -1386,14 +1390,13 @@ static int aio_read_f(BlockBackend *blk, int argc, char **argv) + if (ctx->buf == NULL) { + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_READ); + g_free(ctx); +- return 0; ++ return; + } + + gettimeofday(&ctx->t1, NULL); + block_acct_start(blk_get_stats(blk), &ctx->acct, ctx->qiov.size, + BLOCK_ACCT_READ); + blk_aio_preadv(blk, ctx->offset, &ctx->qiov, 0, aio_read_done, ctx); +- return 0; + } + + static void aio_write_help(void) +@@ -1420,7 +1423,7 @@ static void aio_write_help(void) + "\n"); + } + +-static int aio_write_f(BlockBackend *blk, int argc, char **argv); ++static void aio_write_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t aio_write_cmd = { + .name = "aio_write", +@@ -1433,7 +1436,7 @@ static const cmdinfo_t aio_write_cmd = { + .help = aio_write_help, + }; + +-static int aio_write_f(BlockBackend *blk, int argc, char **argv) ++static void aio_write_f(BlockBackend *blk, int argc, char **argv) + { + int nr_iov, c; + int pattern = 0xcd; +@@ -1459,51 +1462,53 @@ static int aio_write_f(BlockBackend *blk, int argc, char **argv) + pattern = parse_pattern(optarg); + if (pattern < 0) { + g_free(ctx); +- return 0; ++ return; + } + break; + case 'i': + printf("injecting invalid write request\n"); + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_WRITE); + g_free(ctx); +- return 0; ++ return; + case 'z': + ctx->zflag = true; + break; + default: + g_free(ctx); +- return qemuio_command_usage(&aio_write_cmd); ++ qemuio_command_usage(&aio_write_cmd); ++ return; + } + } + + if (optind > argc - 2) { + g_free(ctx); +- return qemuio_command_usage(&aio_write_cmd); ++ qemuio_command_usage(&aio_write_cmd); ++ return; + } + + if (ctx->zflag && optind != argc - 2) { + printf("-z supports only a single length parameter\n"); + g_free(ctx); +- return 0; ++ return; + } + + if ((flags & BDRV_REQ_MAY_UNMAP) && !ctx->zflag) { + printf("-u requires -z to be specified\n"); + g_free(ctx); +- return 0; ++ return; + } + + if (ctx->zflag && ctx->Pflag) { + printf("-z and -P cannot be specified at the same time\n"); + g_free(ctx); +- return 0; ++ return; + } + + ctx->offset = cvtnum(argv[optind]); + if (ctx->offset < 0) { + print_cvtnum_err(ctx->offset, argv[optind]); + g_free(ctx); +- return 0; ++ return; + } + optind++; + +@@ -1512,7 +1517,7 @@ static int aio_write_f(BlockBackend *blk, int argc, char **argv) + if (count < 0) { + print_cvtnum_err(count, argv[optind]); + g_free(ctx); +- return 0; ++ return; + } + + ctx->qiov.size = count; +@@ -1525,7 +1530,7 @@ static int aio_write_f(BlockBackend *blk, int argc, char **argv) + if (ctx->buf == NULL) { + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_WRITE); + g_free(ctx); +- return 0; ++ return; + } + + gettimeofday(&ctx->t1, NULL); +@@ -1535,16 +1540,14 @@ static int aio_write_f(BlockBackend *blk, int argc, char **argv) + blk_aio_pwritev(blk, ctx->offset, &ctx->qiov, flags, aio_write_done, + ctx); + } +- return 0; + } + +-static int aio_flush_f(BlockBackend *blk, int argc, char **argv) ++static void aio_flush_f(BlockBackend *blk, int argc, char **argv) + { + BlockAcctCookie cookie; + block_acct_start(blk_get_stats(blk), &cookie, 0, BLOCK_ACCT_FLUSH); + blk_drain_all(); + block_acct_done(blk_get_stats(blk), &cookie); +- return 0; + } + + static const cmdinfo_t aio_flush_cmd = { +@@ -1553,10 +1556,9 @@ static const cmdinfo_t aio_flush_cmd = { + .oneline = "completes all outstanding aio requests" + }; + +-static int flush_f(BlockBackend *blk, int argc, char **argv) ++static void flush_f(BlockBackend *blk, int argc, char **argv) + { + blk_flush(blk); +- return 0; + } + + static const cmdinfo_t flush_cmd = { +@@ -1566,7 +1568,7 @@ static const cmdinfo_t flush_cmd = { + .oneline = "flush all in-core file state to disk", + }; + +-static int truncate_f(BlockBackend *blk, int argc, char **argv) ++static void truncate_f(BlockBackend *blk, int argc, char **argv) + { + Error *local_err = NULL; + int64_t offset; +@@ -1575,16 +1577,14 @@ static int truncate_f(BlockBackend *blk, int argc, char **argv) + offset = cvtnum(argv[1]); + if (offset < 0) { + print_cvtnum_err(offset, argv[1]); +- return 0; ++ return; + } + + ret = blk_truncate(blk, offset, PREALLOC_MODE_OFF, &local_err); + if (ret < 0) { + error_report_err(local_err); +- return 0; ++ return; + } +- +- return 0; + } + + static const cmdinfo_t truncate_cmd = { +@@ -1598,7 +1598,7 @@ static const cmdinfo_t truncate_cmd = { + .oneline = "truncates the current file at the given offset", + }; + +-static int length_f(BlockBackend *blk, int argc, char **argv) ++static void length_f(BlockBackend *blk, int argc, char **argv) + { + int64_t size; + char s1[64]; +@@ -1606,12 +1606,11 @@ static int length_f(BlockBackend *blk, int argc, char **argv) + size = blk_getlength(blk); + if (size < 0) { + printf("getlength: %s\n", strerror(-size)); +- return 0; ++ return; + } + + cvtstr(size, s1, sizeof(s1)); + printf("%s\n", s1); +- return 0; + } + + +@@ -1623,7 +1622,7 @@ static const cmdinfo_t length_cmd = { + }; + + +-static int info_f(BlockBackend *blk, int argc, char **argv) ++static void info_f(BlockBackend *blk, int argc, char **argv) + { + BlockDriverState *bs = blk_bs(blk); + BlockDriverInfo bdi; +@@ -1640,7 +1639,7 @@ static int info_f(BlockBackend *blk, int argc, char **argv) + + ret = bdrv_get_info(bs, &bdi); + if (ret) { +- return 0; ++ return; + } + + cvtstr(bdi.cluster_size, s1, sizeof(s1)); +@@ -1655,8 +1654,6 @@ static int info_f(BlockBackend *blk, int argc, char **argv) + bdrv_image_info_specific_dump(fprintf, stdout, spec_info); + qapi_free_ImageInfoSpecific(spec_info); + } +- +- return 0; + } + + +@@ -1683,7 +1680,7 @@ static void discard_help(void) + "\n"); + } + +-static int discard_f(BlockBackend *blk, int argc, char **argv); ++static void discard_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t discard_cmd = { + .name = "discard", +@@ -1697,7 +1694,7 @@ static const cmdinfo_t discard_cmd = { + .help = discard_help, + }; + +-static int discard_f(BlockBackend *blk, int argc, char **argv) ++static void discard_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false; +@@ -1713,30 +1710,32 @@ static int discard_f(BlockBackend *blk, int argc, char **argv) + qflag = true; + break; + default: +- return qemuio_command_usage(&discard_cmd); ++ qemuio_command_usage(&discard_cmd); ++ return; + } + } + + if (optind != argc - 2) { +- return qemuio_command_usage(&discard_cmd); ++ qemuio_command_usage(&discard_cmd); ++ return; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return 0; ++ return; + } + + optind++; + bytes = cvtnum(argv[optind]); + if (bytes < 0) { + print_cvtnum_err(bytes, argv[optind]); +- return 0; ++ return; + } else if (bytes >> BDRV_SECTOR_BITS > BDRV_REQUEST_MAX_SECTORS) { + printf("length cannot exceed %"PRIu64", given %s\n", + (uint64_t)BDRV_REQUEST_MAX_SECTORS << BDRV_SECTOR_BITS, + argv[optind]); +- return 0; ++ return; + } + + gettimeofday(&t1, NULL); +@@ -1745,7 +1744,7 @@ static int discard_f(BlockBackend *blk, int argc, char **argv) + + if (ret < 0) { + printf("discard failed: %s\n", strerror(-ret)); +- goto out; ++ return; + } + + /* Finally, report back -- -C gives a parsable format */ +@@ -1753,12 +1752,9 @@ static int discard_f(BlockBackend *blk, int argc, char **argv) + t2 = tsub(t2, t1); + print_report("discard", &t2, offset, bytes, bytes, 1, Cflag); + } +- +-out: +- return 0; + } + +-static int alloc_f(BlockBackend *blk, int argc, char **argv) ++static void alloc_f(BlockBackend *blk, int argc, char **argv) + { + BlockDriverState *bs = blk_bs(blk); + int64_t offset, start, remaining, count; +@@ -1769,14 +1765,14 @@ static int alloc_f(BlockBackend *blk, int argc, char **argv) + start = offset = cvtnum(argv[1]); + if (offset < 0) { + print_cvtnum_err(offset, argv[1]); +- return 0; ++ return; + } + + if (argc == 3) { + count = cvtnum(argv[2]); + if (count < 0) { + print_cvtnum_err(count, argv[2]); +- return 0; ++ return; + } + } else { + count = BDRV_SECTOR_SIZE; +@@ -1788,7 +1784,7 @@ static int alloc_f(BlockBackend *blk, int argc, char **argv) + ret = bdrv_is_allocated(bs, offset, remaining, &num); + if (ret < 0) { + printf("is_allocated failed: %s\n", strerror(-ret)); +- return 0; ++ return; + } + offset += num; + remaining -= num; +@@ -1805,7 +1801,6 @@ static int alloc_f(BlockBackend *blk, int argc, char **argv) + + printf("%"PRId64"/%"PRId64" bytes allocated at offset %s\n", + sum_alloc, count, s1); +- return 0; + } + + static const cmdinfo_t alloc_cmd = { +@@ -1851,7 +1846,7 @@ static int map_is_allocated(BlockDriverState *bs, int64_t offset, + return firstret; + } + +-static int map_f(BlockBackend *blk, int argc, char **argv) ++static void map_f(BlockBackend *blk, int argc, char **argv) + { + int64_t offset, bytes; + char s1[64], s2[64]; +@@ -1863,17 +1858,17 @@ static int map_f(BlockBackend *blk, int argc, char **argv) + bytes = blk_getlength(blk); + if (bytes < 0) { + error_report("Failed to query image length: %s", strerror(-bytes)); +- return 0; ++ return; + } + + while (bytes) { + ret = map_is_allocated(blk_bs(blk), offset, bytes, &num); + if (ret < 0) { + error_report("Failed to get allocation status: %s", strerror(-ret)); +- return 0; ++ return; + } else if (!num) { + error_report("Unexpected end of image"); +- return 0; ++ return; + } + + retstr = ret ? " allocated" : "not allocated"; +@@ -1885,8 +1880,6 @@ static int map_f(BlockBackend *blk, int argc, char **argv) + offset += num; + bytes -= num; + } +- +- return 0; + } + + static const cmdinfo_t map_cmd = { +@@ -1914,7 +1907,7 @@ static void reopen_help(void) + "\n"); + } + +-static int reopen_f(BlockBackend *blk, int argc, char **argv); ++static void reopen_f(BlockBackend *blk, int argc, char **argv); + + static QemuOptsList reopen_opts = { + .name = "reopen", +@@ -1936,7 +1929,7 @@ static const cmdinfo_t reopen_cmd = { + .help = reopen_help, + }; + +-static int reopen_f(BlockBackend *blk, int argc, char **argv) ++static void reopen_f(BlockBackend *blk, int argc, char **argv) + { + BlockDriverState *bs = blk_bs(blk); + QemuOpts *qopts; +@@ -1954,19 +1947,19 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv) + case 'c': + if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) { + error_report("Invalid cache option: %s", optarg); +- return 0; ++ return; + } + break; + case 'o': + if (!qemu_opts_parse_noisily(&reopen_opts, optarg, 0)) { + qemu_opts_reset(&reopen_opts); +- return 0; ++ return; + } + break; + case 'r': + if (has_rw_option) { + error_report("Only one -r/-w option may be given"); +- return 0; ++ return; + } + flags &= ~BDRV_O_RDWR; + has_rw_option = true; +@@ -1974,20 +1967,22 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv) + case 'w': + if (has_rw_option) { + error_report("Only one -r/-w option may be given"); +- return 0; ++ return; + } + flags |= BDRV_O_RDWR; + has_rw_option = true; + break; + default: + qemu_opts_reset(&reopen_opts); +- return qemuio_command_usage(&reopen_cmd); ++ qemuio_command_usage(&reopen_cmd); ++ return; + } + } + + if (optind != argc) { + qemu_opts_reset(&reopen_opts); +- return qemuio_command_usage(&reopen_cmd); ++ qemuio_command_usage(&reopen_cmd); ++ return; + } + + if (writethrough != blk_enable_write_cache(blk) && +@@ -1995,7 +1990,7 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv) + { + error_report("Cannot change cache.writeback: Device attached"); + qemu_opts_reset(&reopen_opts); +- return 0; ++ return; + } + + if (!(flags & BDRV_O_RDWR)) { +@@ -2024,11 +2019,9 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv) + } else { + blk_set_enable_write_cache(blk, !writethrough); + } +- +- return 0; + } + +-static int break_f(BlockBackend *blk, int argc, char **argv) ++static void break_f(BlockBackend *blk, int argc, char **argv) + { + int ret; + +@@ -2036,11 +2029,9 @@ static int break_f(BlockBackend *blk, int argc, char **argv) + if (ret < 0) { + printf("Could not set breakpoint: %s\n", strerror(-ret)); + } +- +- return 0; + } + +-static int remove_break_f(BlockBackend *blk, int argc, char **argv) ++static void remove_break_f(BlockBackend *blk, int argc, char **argv) + { + int ret; + +@@ -2048,8 +2039,6 @@ static int remove_break_f(BlockBackend *blk, int argc, char **argv) + if (ret < 0) { + printf("Could not remove breakpoint %s: %s\n", argv[1], strerror(-ret)); + } +- +- return 0; + } + + static const cmdinfo_t break_cmd = { +@@ -2071,7 +2060,7 @@ static const cmdinfo_t remove_break_cmd = { + .oneline = "remove a breakpoint by tag", + }; + +-static int resume_f(BlockBackend *blk, int argc, char **argv) ++static void resume_f(BlockBackend *blk, int argc, char **argv) + { + int ret; + +@@ -2079,8 +2068,6 @@ static int resume_f(BlockBackend *blk, int argc, char **argv) + if (ret < 0) { + printf("Could not resume request: %s\n", strerror(-ret)); + } +- +- return 0; + } + + static const cmdinfo_t resume_cmd = { +@@ -2092,13 +2079,11 @@ static const cmdinfo_t resume_cmd = { + .oneline = "resumes the request tagged as tag", + }; + +-static int wait_break_f(BlockBackend *blk, int argc, char **argv) ++static void wait_break_f(BlockBackend *blk, int argc, char **argv) + { + while (!bdrv_debug_is_suspended(blk_bs(blk), argv[1])) { + aio_poll(blk_get_aio_context(blk), true); + } +- +- return 0; + } + + static const cmdinfo_t wait_break_cmd = { +@@ -2110,7 +2095,7 @@ static const cmdinfo_t wait_break_cmd = { + .oneline = "waits for the suspension of a request", + }; + +-static int abort_f(BlockBackend *blk, int argc, char **argv) ++static void abort_f(BlockBackend *blk, int argc, char **argv) + { + abort(); + } +@@ -2136,7 +2121,7 @@ static void sigraise_help(void) + "\n", SIGTERM); + } + +-static int sigraise_f(BlockBackend *blk, int argc, char **argv); ++static void sigraise_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t sigraise_cmd = { + .name = "sigraise", +@@ -2149,16 +2134,16 @@ static const cmdinfo_t sigraise_cmd = { + .help = sigraise_help, + }; + +-static int sigraise_f(BlockBackend *blk, int argc, char **argv) ++static void sigraise_f(BlockBackend *blk, int argc, char **argv) + { + int64_t sig = cvtnum(argv[1]); + if (sig < 0) { + print_cvtnum_err(sig, argv[1]); +- return 0; ++ return; + } else if (sig > NSIG) { + printf("signal argument '%s' is too large to be a valid signal\n", + argv[1]); +- return 0; ++ return; + } + + /* Using raise() to kill this process does not necessarily flush all open +@@ -2168,7 +2153,6 @@ static int sigraise_f(BlockBackend *blk, int argc, char **argv) + fflush(stderr); + + raise(sig); +- return 0; + } + + static void sleep_cb(void *opaque) +@@ -2177,7 +2161,7 @@ static void sleep_cb(void *opaque) + *expired = true; + } + +-static int sleep_f(BlockBackend *blk, int argc, char **argv) ++static void sleep_f(BlockBackend *blk, int argc, char **argv) + { + char *endptr; + long ms; +@@ -2187,7 +2171,7 @@ static int sleep_f(BlockBackend *blk, int argc, char **argv) + ms = strtol(argv[1], &endptr, 0); + if (ms < 0 || *endptr != '\0') { + printf("%s is not a valid number\n", argv[1]); +- return 0; ++ return; + } + + timer = timer_new_ns(QEMU_CLOCK_HOST, sleep_cb, &expired); +@@ -2198,8 +2182,6 @@ static int sleep_f(BlockBackend *blk, int argc, char **argv) + } + + timer_free(timer); +- +- return 0; + } + + static const cmdinfo_t sleep_cmd = { +@@ -2246,23 +2228,22 @@ static void help_all(void) + printf("\nUse 'help commandname' for extended help.\n"); + } + +-static int help_f(BlockBackend *blk, int argc, char **argv) ++static void help_f(BlockBackend *blk, int argc, char **argv) + { + const cmdinfo_t *ct; + + if (argc == 1) { + help_all(); +- return 0; ++ return; + } + + ct = find_command(argv[1]); + if (ct == NULL) { + printf("command %s not found\n", argv[1]); +- return 0; ++ return; + } + + help_onecmd(argv[1], ct); +- return 0; + } + + static const cmdinfo_t help_cmd = { +@@ -2276,14 +2257,13 @@ static const cmdinfo_t help_cmd = { + .oneline = "help for one or all commands", + }; + +-bool qemuio_command(BlockBackend *blk, const char *cmd) ++void qemuio_command(BlockBackend *blk, const char *cmd) + { + AioContext *ctx; + char *input; + const cmdinfo_t *ct; + char **v; + int c; +- bool done = false; + + input = g_strdup(cmd); + v = breakline(input, &c); +@@ -2292,7 +2272,7 @@ bool qemuio_command(BlockBackend *blk, const char *cmd) + if (ct) { + ctx = blk ? blk_get_aio_context(blk) : qemu_get_aio_context(); + aio_context_acquire(ctx); +- done = command(blk, ct, c, v); ++ command(blk, ct, c, v); + aio_context_release(ctx); + } else { + fprintf(stderr, "command \"%s\" not found\n", v[0]); +@@ -2300,8 +2280,6 @@ bool qemuio_command(BlockBackend *blk, const char *cmd) + } + g_free(input); + g_free(v); +- +- return done; + } + + static void __attribute((constructor)) init_qemuio_commands(void) +diff --git a/qemu-io.c b/qemu-io.c +index 73c638f..02a67c9 100644 +--- a/qemu-io.c ++++ b/qemu-io.c +@@ -37,6 +37,7 @@ + static char *progname; + + static BlockBackend *qemuio_blk; ++static bool quit_qemu_io; + + /* qemu-io commands passed using -c */ + static int ncmdline; +@@ -65,11 +66,10 @@ static int get_eof_char(void) + #endif + } + +-static int close_f(BlockBackend *blk, int argc, char **argv) ++static void close_f(BlockBackend *blk, int argc, char **argv) + { + blk_unref(qemuio_blk); + qemuio_blk = NULL; +- return 0; + } + + static const cmdinfo_t close_cmd = { +@@ -136,7 +136,7 @@ static void open_help(void) + "\n"); + } + +-static int open_f(BlockBackend *blk, int argc, char **argv); ++static void open_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t open_cmd = { + .name = "open", +@@ -160,7 +160,7 @@ static QemuOptsList empty_opts = { + }, + }; + +-static int open_f(BlockBackend *blk, int argc, char **argv) ++static void open_f(BlockBackend *blk, int argc, char **argv) + { + int flags = BDRV_O_UNMAP; + int readonly = 0; +@@ -192,25 +192,25 @@ static int open_f(BlockBackend *blk, int argc, char **argv) + if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) { + error_report("Invalid cache option: %s", optarg); + qemu_opts_reset(&empty_opts); +- return 0; ++ return; + } + break; + case 'd': + if (bdrv_parse_discard_flags(optarg, &flags) < 0) { + error_report("Invalid discard option: %s", optarg); + qemu_opts_reset(&empty_opts); +- return 0; ++ return; + } + break; + case 'o': + if (imageOpts) { + printf("--image-opts and 'open -o' are mutually exclusive\n"); + qemu_opts_reset(&empty_opts); +- return 0; ++ return; + } + if (!qemu_opts_parse_noisily(&empty_opts, optarg, false)) { + qemu_opts_reset(&empty_opts); +- return 0; ++ return; + } + break; + case 'U': +@@ -218,7 +218,8 @@ static int open_f(BlockBackend *blk, int argc, char **argv) + break; + default: + qemu_opts_reset(&empty_opts); +- return qemuio_command_usage(&open_cmd); ++ qemuio_command_usage(&open_cmd); ++ return; + } + } + +@@ -229,7 +230,7 @@ static int open_f(BlockBackend *blk, int argc, char **argv) + if (imageOpts && (optind == argc - 1)) { + if (!qemu_opts_parse_noisily(&empty_opts, argv[optind], false)) { + qemu_opts_reset(&empty_opts); +- return 0; ++ return; + } + optind++; + } +@@ -246,12 +247,11 @@ static int open_f(BlockBackend *blk, int argc, char **argv) + qobject_unref(opts); + qemuio_command_usage(&open_cmd); + } +- return 0; + } + +-static int quit_f(BlockBackend *blk, int argc, char **argv) ++static void quit_f(BlockBackend *blk, int argc, char **argv) + { +- return 1; ++ quit_qemu_io = true; + } + + static const cmdinfo_t quit_cmd = { +@@ -392,18 +392,18 @@ static void prep_fetchline(void *opaque) + + static void command_loop(void) + { +- int i, done = 0, fetchable = 0, prompted = 0; ++ int i, fetchable = 0, prompted = 0; + char *input; + +- for (i = 0; !done && i < ncmdline; i++) { +- done = qemuio_command(qemuio_blk, cmdline[i]); ++ for (i = 0; !quit_qemu_io && i < ncmdline; i++) { ++ qemuio_command(qemuio_blk, cmdline[i]); + } + if (cmdline) { + g_free(cmdline); + return; + } + +- while (!done) { ++ while (!quit_qemu_io) { + if (!prompted) { + printf("%s", get_prompt()); + fflush(stdout); +@@ -421,7 +421,7 @@ static void command_loop(void) + if (input == NULL) { + break; + } +- done = qemuio_command(qemuio_blk, input); ++ qemuio_command(qemuio_blk, input); + g_free(input); + + prompted = 0; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-io-Exit-with-error-when-a-command-failed.patch b/SOURCES/kvm-qemu-io-Exit-with-error-when-a-command-failed.patch new file mode 100644 index 0000000..724d9fd --- /dev/null +++ b/SOURCES/kvm-qemu-io-Exit-with-error-when-a-command-failed.patch @@ -0,0 +1,116 @@ +From c9498bf0eae922fb4673d024904a8d93bdf35d2d Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 16:43:10 +0200 +Subject: [PATCH 051/268] qemu-io: Exit with error when a command failed + +RH-Author: Max Reitz +Message-id: <20180618164312.24423-4-mreitz@redhat.com> +Patchwork-id: 80780 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/5] qemu-io: Exit with error when a command failed +Bugzilla: 1519617 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +Currently, qemu-io basically always returns success when it gets to +interactive mode (so once the whole command line has been parsed; even +before the commands on the command line are interpreted). That is not +very useful. + +This patch makes qemu-io return failure when any of the executed +commands failed. + +Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1519617 +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509194302.21585-4-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 6b3aa8485ca8e61b168f51d465188855cf549bd2) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-io.c | 28 ++++++++++++++++++++++------ + 1 file changed, 22 insertions(+), 6 deletions(-) + +diff --git a/qemu-io.c b/qemu-io.c +index ec66838..13829f5 100644 +--- a/qemu-io.c ++++ b/qemu-io.c +@@ -400,17 +400,21 @@ static void prep_fetchline(void *opaque) + *fetchable= 1; + } + +-static void command_loop(void) ++static int command_loop(void) + { + int i, fetchable = 0, prompted = 0; ++ int ret, last_error = 0; + char *input; + + for (i = 0; !quit_qemu_io && i < ncmdline; i++) { +- qemuio_command(qemuio_blk, cmdline[i]); ++ ret = qemuio_command(qemuio_blk, cmdline[i]); ++ if (ret < 0) { ++ last_error = ret; ++ } + } + if (cmdline) { + g_free(cmdline); +- return; ++ return last_error; + } + + while (!quit_qemu_io) { +@@ -431,13 +435,19 @@ static void command_loop(void) + if (input == NULL) { + break; + } +- qemuio_command(qemuio_blk, input); ++ ret = qemuio_command(qemuio_blk, input); + g_free(input); + ++ if (ret < 0) { ++ last_error = ret; ++ } ++ + prompted = 0; + fetchable = 0; + } + qemu_set_fd_handler(STDIN_FILENO, NULL, NULL, NULL); ++ ++ return last_error; + } + + static void add_user_command(char *optarg) +@@ -502,6 +512,7 @@ int main(int argc, char **argv) + int c; + int opt_index = 0; + int flags = BDRV_O_UNMAP; ++ int ret; + bool writethrough = true; + Error *local_error = NULL; + QDict *opts = NULL; +@@ -663,7 +674,7 @@ int main(int argc, char **argv) + } + } + } +- command_loop(); ++ ret = command_loop(); + + /* + * Make sure all outstanding requests complete before the program exits. +@@ -672,5 +683,10 @@ int main(int argc, char **argv) + + blk_unref(qemuio_blk); + g_free(readline_state); +- return 0; ++ ++ if (ret < 0) { ++ return 1; ++ } else { ++ return 0; ++ } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-io-Let-command-functions-return-error-code.patch b/SOURCES/kvm-qemu-io-Let-command-functions-return-error-code.patch new file mode 100644 index 0000000..b4413af --- /dev/null +++ b/SOURCES/kvm-qemu-io-Let-command-functions-return-error-code.patch @@ -0,0 +1,1452 @@ +From acfb8ed23ede36db229a3cd6e26162eb644c21fd Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 25 Jun 2018 13:12:14 +0200 +Subject: [PATCH 050/268] qemu-io: Let command functions return error code + +RH-Author: Max Reitz +Message-id: <20180618164312.24423-3-mreitz@redhat.com> +Patchwork-id: 80781 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 2/5] qemu-io: Let command functions return error code +Bugzilla: 1519617 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Miroslav Rezanina + +This is basically what everything else in the qemu code base does, so we +can do it here, too. + +Signed-off-by: Max Reitz +Reviewed-by: Eric Blake +Message-id: 20180509194302.21585-3-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit b32d7a39af488d280ce5f02a2ed94871d696f87a) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + include/qemu-io.h | 9 +- + qemu-io-cmds.c | 346 ++++++++++++++++++++++++++++++++---------------------- + qemu-io.c | 34 ++++-- + 3 files changed, 232 insertions(+), 157 deletions(-) + +diff --git a/include/qemu-io.h b/include/qemu-io.h +index 06cdfbf..7433239 100644 +--- a/include/qemu-io.h ++++ b/include/qemu-io.h +@@ -22,7 +22,12 @@ + + #define CMD_FLAG_GLOBAL ((int)0x80000000) /* don't iterate "args" */ + +-typedef void (*cfunc_t)(BlockBackend *blk, int argc, char **argv); ++/* Implement a qemu-io command. ++ * Operate on @blk using @argc/@argv as the command's arguments, and ++ * return 0 on success or negative errno on failure. ++ */ ++typedef int (*cfunc_t)(BlockBackend *blk, int argc, char **argv); ++ + typedef void (*helpfunc_t)(void); + + typedef struct cmdinfo { +@@ -41,7 +46,7 @@ typedef struct cmdinfo { + + extern bool qemuio_misalign; + +-void qemuio_command(BlockBackend *blk, const char *cmd); ++int qemuio_command(BlockBackend *blk, const char *cmd); + + void qemuio_add_command(const cmdinfo_t *ci); + void qemuio_command_usage(const cmdinfo_t *ci); +diff --git a/qemu-io-cmds.c b/qemu-io-cmds.c +index c2fbaae..5bf5f28 100644 +--- a/qemu-io-cmds.c ++++ b/qemu-io-cmds.c +@@ -65,13 +65,13 @@ static int init_check_command(BlockBackend *blk, const cmdinfo_t *ct) + return 1; + } + +-static void command(BlockBackend *blk, const cmdinfo_t *ct, int argc, +- char **argv) ++static int command(BlockBackend *blk, const cmdinfo_t *ct, int argc, ++ char **argv) + { + char *cmd = argv[0]; + + if (!init_check_command(blk, ct)) { +- return; ++ return -EINVAL; + } + + if (argc - 1 < ct->argmin || (ct->argmax != -1 && argc - 1 > ct->argmax)) { +@@ -88,7 +88,7 @@ static void command(BlockBackend *blk, const cmdinfo_t *ct, int argc, + "bad argument count %d to %s, expected between %d and %d arguments\n", + argc-1, cmd, ct->argmin, ct->argmax); + } +- return; ++ return -EINVAL; + } + + /* Request additional permissions if necessary for this command. The caller +@@ -108,13 +108,13 @@ static void command(BlockBackend *blk, const cmdinfo_t *ct, int argc, + ret = blk_set_perm(blk, new_perm, orig_shared_perm, &local_err); + if (ret < 0) { + error_report_err(local_err); +- return; ++ return ret; + } + } + } + + optind = 0; +- ct->cfunc(blk, argc, argv); ++ return ct->cfunc(blk, argc, argv); + } + + static const cmdinfo_t *find_command(const char *cmd) +@@ -633,7 +633,7 @@ static void read_help(void) + "\n"); + } + +-static void read_f(BlockBackend *blk, int argc, char **argv); ++static int read_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t read_cmd = { + .name = "read", +@@ -646,12 +646,12 @@ static const cmdinfo_t read_cmd = { + .help = read_help, + }; + +-static void read_f(BlockBackend *blk, int argc, char **argv) ++static int read_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false, vflag = false; + bool Pflag = false, sflag = false, lflag = false, bflag = false; +- int c, cnt; ++ int c, cnt, ret; + char *buf; + int64_t offset; + int64_t count; +@@ -673,7 +673,7 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + pattern_count = cvtnum(optarg); + if (pattern_count < 0) { + print_cvtnum_err(pattern_count, optarg); +- return; ++ return pattern_count; + } + break; + case 'p': +@@ -683,7 +683,7 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + Pflag = true; + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return; ++ return -EINVAL; + } + break; + case 'q': +@@ -694,7 +694,7 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + pattern_offset = cvtnum(optarg); + if (pattern_offset < 0) { + print_cvtnum_err(pattern_offset, optarg); +- return; ++ return pattern_offset; + } + break; + case 'v': +@@ -702,35 +702,35 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + break; + default: + qemuio_command_usage(&read_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind != argc - 2) { + qemuio_command_usage(&read_cmd); +- return; ++ return -EINVAL; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return; ++ return offset; + } + + optind++; + count = cvtnum(argv[optind]); + if (count < 0) { + print_cvtnum_err(count, argv[optind]); +- return; ++ return count; + } else if (count > BDRV_REQUEST_MAX_BYTES) { + printf("length cannot exceed %" PRIu64 ", given %s\n", + (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]); +- return; ++ return -EINVAL; + } + + if (!Pflag && (lflag || sflag)) { + qemuio_command_usage(&read_cmd); +- return; ++ return -EINVAL; + } + + if (!lflag) { +@@ -739,19 +739,19 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + + if ((pattern_count < 0) || (pattern_count + pattern_offset > count)) { + printf("pattern verification range exceeds end of read data\n"); +- return; ++ return -EINVAL; + } + + if (bflag) { + if (!QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)) { + printf("%" PRId64 " is not a sector-aligned value for 'offset'\n", + offset); +- return; ++ return -EINVAL; + } + if (!QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)) { + printf("%"PRId64" is not a sector-aligned value for 'count'\n", + count); +- return; ++ return -EINVAL; + } + } + +@@ -759,16 +759,19 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + + gettimeofday(&t1, NULL); + if (bflag) { +- cnt = do_load_vmstate(blk, buf, offset, count, &total); ++ ret = do_load_vmstate(blk, buf, offset, count, &total); + } else { +- cnt = do_pread(blk, buf, offset, count, &total); ++ ret = do_pread(blk, buf, offset, count, &total); + } + gettimeofday(&t2, NULL); + +- if (cnt < 0) { +- printf("read failed: %s\n", strerror(-cnt)); ++ if (ret < 0) { ++ printf("read failed: %s\n", strerror(-ret)); + goto out; + } ++ cnt = ret; ++ ++ ret = 0; + + if (Pflag) { + void *cmp_buf = g_malloc(pattern_count); +@@ -777,6 +780,7 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + printf("Pattern verification failed at offset %" + PRId64 ", %"PRId64" bytes\n", + offset + pattern_offset, pattern_count); ++ ret = -EINVAL; + } + g_free(cmp_buf); + } +@@ -795,6 +799,7 @@ static void read_f(BlockBackend *blk, int argc, char **argv) + + out: + qemu_io_free(buf); ++ return ret; + } + + static void readv_help(void) +@@ -816,7 +821,7 @@ static void readv_help(void) + "\n"); + } + +-static void readv_f(BlockBackend *blk, int argc, char **argv); ++static int readv_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t readv_cmd = { + .name = "readv", +@@ -828,11 +833,11 @@ static const cmdinfo_t readv_cmd = { + .help = readv_help, + }; + +-static void readv_f(BlockBackend *blk, int argc, char **argv) ++static int readv_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false, vflag = false; +- int c, cnt; ++ int c, cnt, ret; + char *buf; + int64_t offset; + /* Some compilers get confused and warn if this is not initialized. */ +@@ -851,7 +856,7 @@ static void readv_f(BlockBackend *blk, int argc, char **argv) + Pflag = true; + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return; ++ return -EINVAL; + } + break; + case 'q': +@@ -862,37 +867,40 @@ static void readv_f(BlockBackend *blk, int argc, char **argv) + break; + default: + qemuio_command_usage(&readv_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind > argc - 2) { + qemuio_command_usage(&readv_cmd); +- return; ++ return -EINVAL; + } + + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return; ++ return offset; + } + optind++; + + nr_iov = argc - optind; + buf = create_iovec(blk, &qiov, &argv[optind], nr_iov, 0xab); + if (buf == NULL) { +- return; ++ return -EINVAL; + } + + gettimeofday(&t1, NULL); +- cnt = do_aio_readv(blk, &qiov, offset, &total); ++ ret = do_aio_readv(blk, &qiov, offset, &total); + gettimeofday(&t2, NULL); + +- if (cnt < 0) { +- printf("readv failed: %s\n", strerror(-cnt)); ++ if (ret < 0) { ++ printf("readv failed: %s\n", strerror(-ret)); + goto out; + } ++ cnt = ret; ++ ++ ret = 0; + + if (Pflag) { + void *cmp_buf = g_malloc(qiov.size); +@@ -900,6 +908,7 @@ static void readv_f(BlockBackend *blk, int argc, char **argv) + if (memcmp(buf, cmp_buf, qiov.size)) { + printf("Pattern verification failed at offset %" + PRId64 ", %zd bytes\n", offset, qiov.size); ++ ret = -EINVAL; + } + g_free(cmp_buf); + } +@@ -919,6 +928,7 @@ static void readv_f(BlockBackend *blk, int argc, char **argv) + out: + qemu_iovec_destroy(&qiov); + qemu_io_free(buf); ++ return ret; + } + + static void write_help(void) +@@ -944,7 +954,7 @@ static void write_help(void) + "\n"); + } + +-static void write_f(BlockBackend *blk, int argc, char **argv); ++static int write_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t write_cmd = { + .name = "write", +@@ -958,13 +968,13 @@ static const cmdinfo_t write_cmd = { + .help = write_help, + }; + +-static void write_f(BlockBackend *blk, int argc, char **argv) ++static int write_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false, bflag = false; + bool Pflag = false, zflag = false, cflag = false; + int flags = 0; +- int c, cnt; ++ int c, cnt, ret; + char *buf = NULL; + int64_t offset; + int64_t count; +@@ -993,7 +1003,7 @@ static void write_f(BlockBackend *blk, int argc, char **argv) + Pflag = true; + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return; ++ return -EINVAL; + } + break; + case 'q': +@@ -1007,63 +1017,63 @@ static void write_f(BlockBackend *blk, int argc, char **argv) + break; + default: + qemuio_command_usage(&write_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind != argc - 2) { + qemuio_command_usage(&write_cmd); +- return; ++ return -EINVAL; + } + + if (bflag && zflag) { + printf("-b and -z cannot be specified at the same time\n"); +- return; ++ return -EINVAL; + } + + if ((flags & BDRV_REQ_FUA) && (bflag || cflag)) { + printf("-f and -b or -c cannot be specified at the same time\n"); +- return; ++ return -EINVAL; + } + + if ((flags & BDRV_REQ_MAY_UNMAP) && !zflag) { + printf("-u requires -z to be specified\n"); +- return; ++ return -EINVAL; + } + + if (zflag && Pflag) { + printf("-z and -P cannot be specified at the same time\n"); +- return; ++ return -EINVAL; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return; ++ return offset; + } + + optind++; + count = cvtnum(argv[optind]); + if (count < 0) { + print_cvtnum_err(count, argv[optind]); +- return; ++ return count; + } else if (count > BDRV_REQUEST_MAX_BYTES) { + printf("length cannot exceed %" PRIu64 ", given %s\n", + (uint64_t)BDRV_REQUEST_MAX_BYTES, argv[optind]); +- return; ++ return -EINVAL; + } + + if (bflag || cflag) { + if (!QEMU_IS_ALIGNED(offset, BDRV_SECTOR_SIZE)) { + printf("%" PRId64 " is not a sector-aligned value for 'offset'\n", + offset); +- return; ++ return -EINVAL; + } + + if (!QEMU_IS_ALIGNED(count, BDRV_SECTOR_SIZE)) { + printf("%"PRId64" is not a sector-aligned value for 'count'\n", + count); +- return; ++ return -EINVAL; + } + } + +@@ -1073,20 +1083,23 @@ static void write_f(BlockBackend *blk, int argc, char **argv) + + gettimeofday(&t1, NULL); + if (bflag) { +- cnt = do_save_vmstate(blk, buf, offset, count, &total); ++ ret = do_save_vmstate(blk, buf, offset, count, &total); + } else if (zflag) { +- cnt = do_co_pwrite_zeroes(blk, offset, count, flags, &total); ++ ret = do_co_pwrite_zeroes(blk, offset, count, flags, &total); + } else if (cflag) { +- cnt = do_write_compressed(blk, buf, offset, count, &total); ++ ret = do_write_compressed(blk, buf, offset, count, &total); + } else { +- cnt = do_pwrite(blk, buf, offset, count, flags, &total); ++ ret = do_pwrite(blk, buf, offset, count, flags, &total); + } + gettimeofday(&t2, NULL); + +- if (cnt < 0) { +- printf("write failed: %s\n", strerror(-cnt)); ++ if (ret < 0) { ++ printf("write failed: %s\n", strerror(-ret)); + goto out; + } ++ cnt = ret; ++ ++ ret = 0; + + if (qflag) { + goto out; +@@ -1100,6 +1113,7 @@ out: + if (!zflag) { + qemu_io_free(buf); + } ++ return ret; + } + + static void +@@ -1121,7 +1135,7 @@ writev_help(void) + "\n"); + } + +-static void writev_f(BlockBackend *blk, int argc, char **argv); ++static int writev_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t writev_cmd = { + .name = "writev", +@@ -1134,12 +1148,12 @@ static const cmdinfo_t writev_cmd = { + .help = writev_help, + }; + +-static void writev_f(BlockBackend *blk, int argc, char **argv) ++static int writev_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false; + int flags = 0; +- int c, cnt; ++ int c, cnt, ret; + char *buf; + int64_t offset; + /* Some compilers get confused and warn if this is not initialized. */ +@@ -1162,41 +1176,44 @@ static void writev_f(BlockBackend *blk, int argc, char **argv) + case 'P': + pattern = parse_pattern(optarg); + if (pattern < 0) { +- return; ++ return -EINVAL; + } + break; + default: + qemuio_command_usage(&writev_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind > argc - 2) { + qemuio_command_usage(&writev_cmd); +- return; ++ return -EINVAL; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return; ++ return offset; + } + optind++; + + nr_iov = argc - optind; + buf = create_iovec(blk, &qiov, &argv[optind], nr_iov, pattern); + if (buf == NULL) { +- return; ++ return -EINVAL; + } + + gettimeofday(&t1, NULL); +- cnt = do_aio_writev(blk, &qiov, offset, flags, &total); ++ ret = do_aio_writev(blk, &qiov, offset, flags, &total); + gettimeofday(&t2, NULL); + +- if (cnt < 0) { +- printf("writev failed: %s\n", strerror(-cnt)); ++ if (ret < 0) { ++ printf("writev failed: %s\n", strerror(-ret)); + goto out; + } ++ cnt = ret; ++ ++ ret = 0; + + if (qflag) { + goto out; +@@ -1208,6 +1225,7 @@ static void writev_f(BlockBackend *blk, int argc, char **argv) + out: + qemu_iovec_destroy(&qiov); + qemu_io_free(buf); ++ return ret; + } + + struct aio_ctx { +@@ -1314,6 +1332,9 @@ static void aio_read_help(void) + " standard output stream (with -v option) for subsequent inspection.\n" + " The read is performed asynchronously and the aio_flush command must be\n" + " used to ensure all outstanding aio requests have been completed.\n" ++" Note that due to its asynchronous nature, this command will be\n" ++" considered successful once the request is submitted, independently\n" ++" of potential I/O errors or pattern mismatches.\n" + " -C, -- report statistics in a machine parsable format\n" + " -P, -- use a pattern to verify read data\n" + " -i, -- treat request as invalid, for exercising stats\n" +@@ -1322,7 +1343,7 @@ static void aio_read_help(void) + "\n"); + } + +-static void aio_read_f(BlockBackend *blk, int argc, char **argv); ++static int aio_read_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t aio_read_cmd = { + .name = "aio_read", +@@ -1334,7 +1355,7 @@ static const cmdinfo_t aio_read_cmd = { + .help = aio_read_help, + }; + +-static void aio_read_f(BlockBackend *blk, int argc, char **argv) ++static int aio_read_f(BlockBackend *blk, int argc, char **argv) + { + int nr_iov, c; + struct aio_ctx *ctx = g_new0(struct aio_ctx, 1); +@@ -1350,14 +1371,14 @@ static void aio_read_f(BlockBackend *blk, int argc, char **argv) + ctx->pattern = parse_pattern(optarg); + if (ctx->pattern < 0) { + g_free(ctx); +- return; ++ return -EINVAL; + } + break; + case 'i': + printf("injecting invalid read request\n"); + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_READ); + g_free(ctx); +- return; ++ return 0; + case 'q': + ctx->qflag = true; + break; +@@ -1367,21 +1388,22 @@ static void aio_read_f(BlockBackend *blk, int argc, char **argv) + default: + g_free(ctx); + qemuio_command_usage(&aio_read_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind > argc - 2) { + g_free(ctx); + qemuio_command_usage(&aio_read_cmd); +- return; ++ return -EINVAL; + } + + ctx->offset = cvtnum(argv[optind]); + if (ctx->offset < 0) { +- print_cvtnum_err(ctx->offset, argv[optind]); ++ int ret = ctx->offset; ++ print_cvtnum_err(ret, argv[optind]); + g_free(ctx); +- return; ++ return ret; + } + optind++; + +@@ -1390,13 +1412,14 @@ static void aio_read_f(BlockBackend *blk, int argc, char **argv) + if (ctx->buf == NULL) { + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_READ); + g_free(ctx); +- return; ++ return -EINVAL; + } + + gettimeofday(&ctx->t1, NULL); + block_acct_start(blk_get_stats(blk), &ctx->acct, ctx->qiov.size, + BLOCK_ACCT_READ); + blk_aio_preadv(blk, ctx->offset, &ctx->qiov, 0, aio_read_done, ctx); ++ return 0; + } + + static void aio_write_help(void) +@@ -1413,6 +1436,9 @@ static void aio_write_help(void) + " filled with a set pattern (0xcdcdcdcd).\n" + " The write is performed asynchronously and the aio_flush command must be\n" + " used to ensure all outstanding aio requests have been completed.\n" ++" Note that due to its asynchronous nature, this command will be\n" ++" considered successful once the request is submitted, independently\n" ++" of potential I/O errors or pattern mismatches.\n" + " -P, -- use different pattern to fill file\n" + " -C, -- report statistics in a machine parsable format\n" + " -f, -- use Force Unit Access semantics\n" +@@ -1423,7 +1449,7 @@ static void aio_write_help(void) + "\n"); + } + +-static void aio_write_f(BlockBackend *blk, int argc, char **argv); ++static int aio_write_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t aio_write_cmd = { + .name = "aio_write", +@@ -1436,7 +1462,7 @@ static const cmdinfo_t aio_write_cmd = { + .help = aio_write_help, + }; + +-static void aio_write_f(BlockBackend *blk, int argc, char **argv) ++static int aio_write_f(BlockBackend *blk, int argc, char **argv) + { + int nr_iov, c; + int pattern = 0xcd; +@@ -1462,53 +1488,54 @@ static void aio_write_f(BlockBackend *blk, int argc, char **argv) + pattern = parse_pattern(optarg); + if (pattern < 0) { + g_free(ctx); +- return; ++ return -EINVAL; + } + break; + case 'i': + printf("injecting invalid write request\n"); + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_WRITE); + g_free(ctx); +- return; ++ return 0; + case 'z': + ctx->zflag = true; + break; + default: + g_free(ctx); + qemuio_command_usage(&aio_write_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind > argc - 2) { + g_free(ctx); + qemuio_command_usage(&aio_write_cmd); +- return; ++ return -EINVAL; + } + + if (ctx->zflag && optind != argc - 2) { + printf("-z supports only a single length parameter\n"); + g_free(ctx); +- return; ++ return -EINVAL; + } + + if ((flags & BDRV_REQ_MAY_UNMAP) && !ctx->zflag) { + printf("-u requires -z to be specified\n"); + g_free(ctx); +- return; ++ return -EINVAL; + } + + if (ctx->zflag && ctx->Pflag) { + printf("-z and -P cannot be specified at the same time\n"); + g_free(ctx); +- return; ++ return -EINVAL; + } + + ctx->offset = cvtnum(argv[optind]); + if (ctx->offset < 0) { +- print_cvtnum_err(ctx->offset, argv[optind]); ++ int ret = ctx->offset; ++ print_cvtnum_err(ret, argv[optind]); + g_free(ctx); +- return; ++ return ret; + } + optind++; + +@@ -1517,7 +1544,7 @@ static void aio_write_f(BlockBackend *blk, int argc, char **argv) + if (count < 0) { + print_cvtnum_err(count, argv[optind]); + g_free(ctx); +- return; ++ return count; + } + + ctx->qiov.size = count; +@@ -1530,7 +1557,7 @@ static void aio_write_f(BlockBackend *blk, int argc, char **argv) + if (ctx->buf == NULL) { + block_acct_invalid(blk_get_stats(blk), BLOCK_ACCT_WRITE); + g_free(ctx); +- return; ++ return -EINVAL; + } + + gettimeofday(&ctx->t1, NULL); +@@ -1540,14 +1567,17 @@ static void aio_write_f(BlockBackend *blk, int argc, char **argv) + blk_aio_pwritev(blk, ctx->offset, &ctx->qiov, flags, aio_write_done, + ctx); + } ++ ++ return 0; + } + +-static void aio_flush_f(BlockBackend *blk, int argc, char **argv) ++static int aio_flush_f(BlockBackend *blk, int argc, char **argv) + { + BlockAcctCookie cookie; + block_acct_start(blk_get_stats(blk), &cookie, 0, BLOCK_ACCT_FLUSH); + blk_drain_all(); + block_acct_done(blk_get_stats(blk), &cookie); ++ return 0; + } + + static const cmdinfo_t aio_flush_cmd = { +@@ -1556,9 +1586,9 @@ static const cmdinfo_t aio_flush_cmd = { + .oneline = "completes all outstanding aio requests" + }; + +-static void flush_f(BlockBackend *blk, int argc, char **argv) ++static int flush_f(BlockBackend *blk, int argc, char **argv) + { +- blk_flush(blk); ++ return blk_flush(blk); + } + + static const cmdinfo_t flush_cmd = { +@@ -1568,7 +1598,7 @@ static const cmdinfo_t flush_cmd = { + .oneline = "flush all in-core file state to disk", + }; + +-static void truncate_f(BlockBackend *blk, int argc, char **argv) ++static int truncate_f(BlockBackend *blk, int argc, char **argv) + { + Error *local_err = NULL; + int64_t offset; +@@ -1577,14 +1607,16 @@ static void truncate_f(BlockBackend *blk, int argc, char **argv) + offset = cvtnum(argv[1]); + if (offset < 0) { + print_cvtnum_err(offset, argv[1]); +- return; ++ return offset; + } + + ret = blk_truncate(blk, offset, PREALLOC_MODE_OFF, &local_err); + if (ret < 0) { + error_report_err(local_err); +- return; ++ return ret; + } ++ ++ return 0; + } + + static const cmdinfo_t truncate_cmd = { +@@ -1598,7 +1630,7 @@ static const cmdinfo_t truncate_cmd = { + .oneline = "truncates the current file at the given offset", + }; + +-static void length_f(BlockBackend *blk, int argc, char **argv) ++static int length_f(BlockBackend *blk, int argc, char **argv) + { + int64_t size; + char s1[64]; +@@ -1606,11 +1638,12 @@ static void length_f(BlockBackend *blk, int argc, char **argv) + size = blk_getlength(blk); + if (size < 0) { + printf("getlength: %s\n", strerror(-size)); +- return; ++ return size; + } + + cvtstr(size, s1, sizeof(s1)); + printf("%s\n", s1); ++ return 0; + } + + +@@ -1622,7 +1655,7 @@ static const cmdinfo_t length_cmd = { + }; + + +-static void info_f(BlockBackend *blk, int argc, char **argv) ++static int info_f(BlockBackend *blk, int argc, char **argv) + { + BlockDriverState *bs = blk_bs(blk); + BlockDriverInfo bdi; +@@ -1639,7 +1672,7 @@ static void info_f(BlockBackend *blk, int argc, char **argv) + + ret = bdrv_get_info(bs, &bdi); + if (ret) { +- return; ++ return ret; + } + + cvtstr(bdi.cluster_size, s1, sizeof(s1)); +@@ -1654,6 +1687,8 @@ static void info_f(BlockBackend *blk, int argc, char **argv) + bdrv_image_info_specific_dump(fprintf, stdout, spec_info); + qapi_free_ImageInfoSpecific(spec_info); + } ++ ++ return 0; + } + + +@@ -1680,7 +1715,7 @@ static void discard_help(void) + "\n"); + } + +-static void discard_f(BlockBackend *blk, int argc, char **argv); ++static int discard_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t discard_cmd = { + .name = "discard", +@@ -1694,7 +1729,7 @@ static const cmdinfo_t discard_cmd = { + .help = discard_help, + }; + +-static void discard_f(BlockBackend *blk, int argc, char **argv) ++static int discard_f(BlockBackend *blk, int argc, char **argv) + { + struct timeval t1, t2; + bool Cflag = false, qflag = false; +@@ -1711,31 +1746,31 @@ static void discard_f(BlockBackend *blk, int argc, char **argv) + break; + default: + qemuio_command_usage(&discard_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind != argc - 2) { + qemuio_command_usage(&discard_cmd); +- return; ++ return -EINVAL; + } + + offset = cvtnum(argv[optind]); + if (offset < 0) { + print_cvtnum_err(offset, argv[optind]); +- return; ++ return offset; + } + + optind++; + bytes = cvtnum(argv[optind]); + if (bytes < 0) { + print_cvtnum_err(bytes, argv[optind]); +- return; ++ return bytes; + } else if (bytes >> BDRV_SECTOR_BITS > BDRV_REQUEST_MAX_SECTORS) { + printf("length cannot exceed %"PRIu64", given %s\n", + (uint64_t)BDRV_REQUEST_MAX_SECTORS << BDRV_SECTOR_BITS, + argv[optind]); +- return; ++ return -EINVAL; + } + + gettimeofday(&t1, NULL); +@@ -1744,7 +1779,7 @@ static void discard_f(BlockBackend *blk, int argc, char **argv) + + if (ret < 0) { + printf("discard failed: %s\n", strerror(-ret)); +- return; ++ return ret; + } + + /* Finally, report back -- -C gives a parsable format */ +@@ -1752,9 +1787,11 @@ static void discard_f(BlockBackend *blk, int argc, char **argv) + t2 = tsub(t2, t1); + print_report("discard", &t2, offset, bytes, bytes, 1, Cflag); + } ++ ++ return 0; + } + +-static void alloc_f(BlockBackend *blk, int argc, char **argv) ++static int alloc_f(BlockBackend *blk, int argc, char **argv) + { + BlockDriverState *bs = blk_bs(blk); + int64_t offset, start, remaining, count; +@@ -1765,14 +1802,14 @@ static void alloc_f(BlockBackend *blk, int argc, char **argv) + start = offset = cvtnum(argv[1]); + if (offset < 0) { + print_cvtnum_err(offset, argv[1]); +- return; ++ return offset; + } + + if (argc == 3) { + count = cvtnum(argv[2]); + if (count < 0) { + print_cvtnum_err(count, argv[2]); +- return; ++ return count; + } + } else { + count = BDRV_SECTOR_SIZE; +@@ -1784,7 +1821,7 @@ static void alloc_f(BlockBackend *blk, int argc, char **argv) + ret = bdrv_is_allocated(bs, offset, remaining, &num); + if (ret < 0) { + printf("is_allocated failed: %s\n", strerror(-ret)); +- return; ++ return ret; + } + offset += num; + remaining -= num; +@@ -1801,6 +1838,7 @@ static void alloc_f(BlockBackend *blk, int argc, char **argv) + + printf("%"PRId64"/%"PRId64" bytes allocated at offset %s\n", + sum_alloc, count, s1); ++ return 0; + } + + static const cmdinfo_t alloc_cmd = { +@@ -1846,7 +1884,7 @@ static int map_is_allocated(BlockDriverState *bs, int64_t offset, + return firstret; + } + +-static void map_f(BlockBackend *blk, int argc, char **argv) ++static int map_f(BlockBackend *blk, int argc, char **argv) + { + int64_t offset, bytes; + char s1[64], s2[64]; +@@ -1858,17 +1896,17 @@ static void map_f(BlockBackend *blk, int argc, char **argv) + bytes = blk_getlength(blk); + if (bytes < 0) { + error_report("Failed to query image length: %s", strerror(-bytes)); +- return; ++ return bytes; + } + + while (bytes) { + ret = map_is_allocated(blk_bs(blk), offset, bytes, &num); + if (ret < 0) { + error_report("Failed to get allocation status: %s", strerror(-ret)); +- return; ++ return ret; + } else if (!num) { + error_report("Unexpected end of image"); +- return; ++ return -EIO; + } + + retstr = ret ? " allocated" : "not allocated"; +@@ -1880,6 +1918,8 @@ static void map_f(BlockBackend *blk, int argc, char **argv) + offset += num; + bytes -= num; + } ++ ++ return 0; + } + + static const cmdinfo_t map_cmd = { +@@ -1907,7 +1947,7 @@ static void reopen_help(void) + "\n"); + } + +-static void reopen_f(BlockBackend *blk, int argc, char **argv); ++static int reopen_f(BlockBackend *blk, int argc, char **argv); + + static QemuOptsList reopen_opts = { + .name = "reopen", +@@ -1929,7 +1969,7 @@ static const cmdinfo_t reopen_cmd = { + .help = reopen_help, + }; + +-static void reopen_f(BlockBackend *blk, int argc, char **argv) ++static int reopen_f(BlockBackend *blk, int argc, char **argv) + { + BlockDriverState *bs = blk_bs(blk); + QemuOpts *qopts; +@@ -1947,19 +1987,19 @@ static void reopen_f(BlockBackend *blk, int argc, char **argv) + case 'c': + if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) { + error_report("Invalid cache option: %s", optarg); +- return; ++ return -EINVAL; + } + break; + case 'o': + if (!qemu_opts_parse_noisily(&reopen_opts, optarg, 0)) { + qemu_opts_reset(&reopen_opts); +- return; ++ return -EINVAL; + } + break; + case 'r': + if (has_rw_option) { + error_report("Only one -r/-w option may be given"); +- return; ++ return -EINVAL; + } + flags &= ~BDRV_O_RDWR; + has_rw_option = true; +@@ -1967,7 +2007,7 @@ static void reopen_f(BlockBackend *blk, int argc, char **argv) + case 'w': + if (has_rw_option) { + error_report("Only one -r/-w option may be given"); +- return; ++ return -EINVAL; + } + flags |= BDRV_O_RDWR; + has_rw_option = true; +@@ -1975,14 +2015,14 @@ static void reopen_f(BlockBackend *blk, int argc, char **argv) + default: + qemu_opts_reset(&reopen_opts); + qemuio_command_usage(&reopen_cmd); +- return; ++ return -EINVAL; + } + } + + if (optind != argc) { + qemu_opts_reset(&reopen_opts); + qemuio_command_usage(&reopen_cmd); +- return; ++ return -EINVAL; + } + + if (writethrough != blk_enable_write_cache(blk) && +@@ -1990,7 +2030,7 @@ static void reopen_f(BlockBackend *blk, int argc, char **argv) + { + error_report("Cannot change cache.writeback: Device attached"); + qemu_opts_reset(&reopen_opts); +- return; ++ return -EBUSY; + } + + if (!(flags & BDRV_O_RDWR)) { +@@ -2016,29 +2056,37 @@ static void reopen_f(BlockBackend *blk, int argc, char **argv) + + if (local_err) { + error_report_err(local_err); +- } else { +- blk_set_enable_write_cache(blk, !writethrough); ++ return -EINVAL; + } ++ ++ blk_set_enable_write_cache(blk, !writethrough); ++ return 0; + } + +-static void break_f(BlockBackend *blk, int argc, char **argv) ++static int break_f(BlockBackend *blk, int argc, char **argv) + { + int ret; + + ret = bdrv_debug_breakpoint(blk_bs(blk), argv[1], argv[2]); + if (ret < 0) { + printf("Could not set breakpoint: %s\n", strerror(-ret)); ++ return ret; + } ++ ++ return 0; + } + +-static void remove_break_f(BlockBackend *blk, int argc, char **argv) ++static int remove_break_f(BlockBackend *blk, int argc, char **argv) + { + int ret; + + ret = bdrv_debug_remove_breakpoint(blk_bs(blk), argv[1]); + if (ret < 0) { + printf("Could not remove breakpoint %s: %s\n", argv[1], strerror(-ret)); ++ return ret; + } ++ ++ return 0; + } + + static const cmdinfo_t break_cmd = { +@@ -2060,14 +2108,17 @@ static const cmdinfo_t remove_break_cmd = { + .oneline = "remove a breakpoint by tag", + }; + +-static void resume_f(BlockBackend *blk, int argc, char **argv) ++static int resume_f(BlockBackend *blk, int argc, char **argv) + { + int ret; + + ret = bdrv_debug_resume(blk_bs(blk), argv[1]); + if (ret < 0) { + printf("Could not resume request: %s\n", strerror(-ret)); ++ return ret; + } ++ ++ return 0; + } + + static const cmdinfo_t resume_cmd = { +@@ -2079,11 +2130,12 @@ static const cmdinfo_t resume_cmd = { + .oneline = "resumes the request tagged as tag", + }; + +-static void wait_break_f(BlockBackend *blk, int argc, char **argv) ++static int wait_break_f(BlockBackend *blk, int argc, char **argv) + { + while (!bdrv_debug_is_suspended(blk_bs(blk), argv[1])) { + aio_poll(blk_get_aio_context(blk), true); + } ++ return 0; + } + + static const cmdinfo_t wait_break_cmd = { +@@ -2095,7 +2147,7 @@ static const cmdinfo_t wait_break_cmd = { + .oneline = "waits for the suspension of a request", + }; + +-static void abort_f(BlockBackend *blk, int argc, char **argv) ++static int abort_f(BlockBackend *blk, int argc, char **argv) + { + abort(); + } +@@ -2121,7 +2173,7 @@ static void sigraise_help(void) + "\n", SIGTERM); + } + +-static void sigraise_f(BlockBackend *blk, int argc, char **argv); ++static int sigraise_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t sigraise_cmd = { + .name = "sigraise", +@@ -2134,16 +2186,16 @@ static const cmdinfo_t sigraise_cmd = { + .help = sigraise_help, + }; + +-static void sigraise_f(BlockBackend *blk, int argc, char **argv) ++static int sigraise_f(BlockBackend *blk, int argc, char **argv) + { + int64_t sig = cvtnum(argv[1]); + if (sig < 0) { + print_cvtnum_err(sig, argv[1]); +- return; ++ return sig; + } else if (sig > NSIG) { + printf("signal argument '%s' is too large to be a valid signal\n", + argv[1]); +- return; ++ return -EINVAL; + } + + /* Using raise() to kill this process does not necessarily flush all open +@@ -2153,6 +2205,8 @@ static void sigraise_f(BlockBackend *blk, int argc, char **argv) + fflush(stderr); + + raise(sig); ++ ++ return 0; + } + + static void sleep_cb(void *opaque) +@@ -2161,7 +2215,7 @@ static void sleep_cb(void *opaque) + *expired = true; + } + +-static void sleep_f(BlockBackend *blk, int argc, char **argv) ++static int sleep_f(BlockBackend *blk, int argc, char **argv) + { + char *endptr; + long ms; +@@ -2171,7 +2225,7 @@ static void sleep_f(BlockBackend *blk, int argc, char **argv) + ms = strtol(argv[1], &endptr, 0); + if (ms < 0 || *endptr != '\0') { + printf("%s is not a valid number\n", argv[1]); +- return; ++ return -EINVAL; + } + + timer = timer_new_ns(QEMU_CLOCK_HOST, sleep_cb, &expired); +@@ -2182,6 +2236,7 @@ static void sleep_f(BlockBackend *blk, int argc, char **argv) + } + + timer_free(timer); ++ return 0; + } + + static const cmdinfo_t sleep_cmd = { +@@ -2228,22 +2283,23 @@ static void help_all(void) + printf("\nUse 'help commandname' for extended help.\n"); + } + +-static void help_f(BlockBackend *blk, int argc, char **argv) ++static int help_f(BlockBackend *blk, int argc, char **argv) + { + const cmdinfo_t *ct; + + if (argc == 1) { + help_all(); +- return; ++ return 0; + } + + ct = find_command(argv[1]); + if (ct == NULL) { + printf("command %s not found\n", argv[1]); +- return; ++ return -EINVAL; + } + + help_onecmd(argv[1], ct); ++ return 0; + } + + static const cmdinfo_t help_cmd = { +@@ -2257,13 +2313,14 @@ static const cmdinfo_t help_cmd = { + .oneline = "help for one or all commands", + }; + +-void qemuio_command(BlockBackend *blk, const char *cmd) ++int qemuio_command(BlockBackend *blk, const char *cmd) + { + AioContext *ctx; + char *input; + const cmdinfo_t *ct; + char **v; + int c; ++ int ret = 0; + + input = g_strdup(cmd); + v = breakline(input, &c); +@@ -2272,14 +2329,17 @@ void qemuio_command(BlockBackend *blk, const char *cmd) + if (ct) { + ctx = blk ? blk_get_aio_context(blk) : qemu_get_aio_context(); + aio_context_acquire(ctx); +- command(blk, ct, c, v); ++ ret = command(blk, ct, c, v); + aio_context_release(ctx); + } else { + fprintf(stderr, "command \"%s\" not found\n", v[0]); ++ ret = -EINVAL; + } + } + g_free(input); + g_free(v); ++ ++ return ret; + } + + static void __attribute((constructor)) init_qemuio_commands(void) +diff --git a/qemu-io.c b/qemu-io.c +index 02a67c9..ec66838 100644 +--- a/qemu-io.c ++++ b/qemu-io.c +@@ -66,10 +66,11 @@ static int get_eof_char(void) + #endif + } + +-static void close_f(BlockBackend *blk, int argc, char **argv) ++static int close_f(BlockBackend *blk, int argc, char **argv) + { + blk_unref(qemuio_blk); + qemuio_blk = NULL; ++ return 0; + } + + static const cmdinfo_t close_cmd = { +@@ -136,7 +137,7 @@ static void open_help(void) + "\n"); + } + +-static void open_f(BlockBackend *blk, int argc, char **argv); ++static int open_f(BlockBackend *blk, int argc, char **argv); + + static const cmdinfo_t open_cmd = { + .name = "open", +@@ -160,12 +161,13 @@ static QemuOptsList empty_opts = { + }, + }; + +-static void open_f(BlockBackend *blk, int argc, char **argv) ++static int open_f(BlockBackend *blk, int argc, char **argv) + { + int flags = BDRV_O_UNMAP; + int readonly = 0; + bool writethrough = true; + int c; ++ int ret; + QemuOpts *qopts; + QDict *opts; + bool force_share = false; +@@ -192,25 +194,25 @@ static void open_f(BlockBackend *blk, int argc, char **argv) + if (bdrv_parse_cache_mode(optarg, &flags, &writethrough) < 0) { + error_report("Invalid cache option: %s", optarg); + qemu_opts_reset(&empty_opts); +- return; ++ return -EINVAL; + } + break; + case 'd': + if (bdrv_parse_discard_flags(optarg, &flags) < 0) { + error_report("Invalid discard option: %s", optarg); + qemu_opts_reset(&empty_opts); +- return; ++ return -EINVAL; + } + break; + case 'o': + if (imageOpts) { + printf("--image-opts and 'open -o' are mutually exclusive\n"); + qemu_opts_reset(&empty_opts); +- return; ++ return -EINVAL; + } + if (!qemu_opts_parse_noisily(&empty_opts, optarg, false)) { + qemu_opts_reset(&empty_opts); +- return; ++ return -EINVAL; + } + break; + case 'U': +@@ -219,7 +221,7 @@ static void open_f(BlockBackend *blk, int argc, char **argv) + default: + qemu_opts_reset(&empty_opts); + qemuio_command_usage(&open_cmd); +- return; ++ return -EINVAL; + } + } + +@@ -230,7 +232,7 @@ static void open_f(BlockBackend *blk, int argc, char **argv) + if (imageOpts && (optind == argc - 1)) { + if (!qemu_opts_parse_noisily(&empty_opts, argv[optind], false)) { + qemu_opts_reset(&empty_opts); +- return; ++ return -EINVAL; + } + optind++; + } +@@ -240,18 +242,26 @@ static void open_f(BlockBackend *blk, int argc, char **argv) + qemu_opts_reset(&empty_opts); + + if (optind == argc - 1) { +- openfile(argv[optind], flags, writethrough, force_share, opts); ++ ret = openfile(argv[optind], flags, writethrough, force_share, opts); + } else if (optind == argc) { +- openfile(NULL, flags, writethrough, force_share, opts); ++ ret = openfile(NULL, flags, writethrough, force_share, opts); + } else { + qobject_unref(opts); + qemuio_command_usage(&open_cmd); ++ return -EINVAL; ++ } ++ ++ if (ret) { ++ return -EINVAL; + } ++ ++ return 0; + } + +-static void quit_f(BlockBackend *blk, int argc, char **argv) ++static int quit_f(BlockBackend *blk, int argc, char **argv) + { + quit_qemu_io = true; ++ return 0; + } + + static const cmdinfo_t quit_cmd = { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-io-Use-purely-string-blockdev-options.patch b/SOURCES/kvm-qemu-io-Use-purely-string-blockdev-options.patch new file mode 100644 index 0000000..6f1d513 --- /dev/null +++ b/SOURCES/kvm-qemu-io-Use-purely-string-blockdev-options.patch @@ -0,0 +1,65 @@ +From 54deefe8c7efcf53da58d1cc546ca34c27a975b8 Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 25 Jun 2018 13:04:46 +0200 +Subject: [PATCH 046/268] qemu-io: Use purely string blockdev options + +RH-Author: Max Reitz +Message-id: <20180618163106.23010-2-mreitz@redhat.com> +Patchwork-id: 80775 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/3] qemu-io: Use purely string blockdev options +Bugzilla: 1576598 +RH-Acked-by: Kevin Wolf +RH-Acked-by: Fam Zheng +RH-Acked-by: Miroslav Rezanina + +Currently, qemu-io only uses string-valued blockdev options (as all are +converted directly from QemuOpts) -- with one exception: -U adds the +force-share option as a boolean. This in itself is already a bit +questionable, but a real issue is that it also assumes the value already +existing in the options QDict would be a boolean, which is wrong. + +That has the following effect: + +$ ./qemu-io -r -U --image-opts \ + driver=file,filename=/dev/null,force-share=off +[1] 15200 segmentation fault (core dumped) ./qemu-io -r -U +--image-opts driver=file,filename=/dev/null,force-share=off + +Since @opts is converted from QemuOpts, the value must be a string, and +we have to compare it as such. Consequently, it makes sense to also set +it as a string instead of a boolean. + +Cc: qemu-stable@nongnu.org +Signed-off-by: Max Reitz +Message-id: 20180502202051.15493-2-mreitz@redhat.com +Reviewed-by: Eric Blake +Signed-off-by: Max Reitz +(cherry picked from commit 2a01c01f9ecb43af4c0a85fe6adc429ffc9c31b5) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-io.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/qemu-io.c b/qemu-io.c +index 72fee0d..73c638f 100644 +--- a/qemu-io.c ++++ b/qemu-io.c +@@ -95,12 +95,12 @@ static int openfile(char *name, int flags, bool writethrough, bool force_share, + opts = qdict_new(); + } + if (qdict_haskey(opts, BDRV_OPT_FORCE_SHARE) +- && !qdict_get_bool(opts, BDRV_OPT_FORCE_SHARE)) { ++ && strcmp(qdict_get_str(opts, BDRV_OPT_FORCE_SHARE), "on")) { + error_report("-U conflicts with image options"); + qobject_unref(opts); + return 1; + } +- qdict_put_bool(opts, BDRV_OPT_FORCE_SHARE, true); ++ qdict_put_str(opts, BDRV_OPT_FORCE_SHARE, "on"); + } + qemuio_blk = blk_new_open(name, NULL, opts, flags, &local_err); + if (!qemuio_blk) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Add-VM.get_qmp_events_filtered.patch b/SOURCES/kvm-qemu-iotests-Add-VM.get_qmp_events_filtered.patch new file mode 100644 index 0000000..2d2f2ec --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Add-VM.get_qmp_events_filtered.patch @@ -0,0 +1,46 @@ +From c491cc3ca87111382c337eda74a1d7cf74891df0 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:41 +0200 +Subject: [PATCH 133/268] qemu-iotests: Add VM.get_qmp_events_filtered() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-59-kwolf@redhat.com> +Patchwork-id: 81122 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 58/73] qemu-iotests: Add VM.get_qmp_events_filtered() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a helper function that returns a list of QMP events that are +already filtered through filter_qmp_event(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 5ad1dbf76a97b6f07d685585175832e990fe9a92) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 5 +++++ + 1 file changed, 5 insertions(+) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index d190ce7..8c08d05 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -393,6 +393,11 @@ class VM(qtest.QEMUQtestMachine): + output_list += [key + '=' + obj[key]] + return ','.join(output_list) + ++ def get_qmp_events_filtered(self, wait=True): ++ result = [] ++ for ev in self.get_qmp_events(wait=wait): ++ result.append(filter_qmp_event(ev)) ++ return result + + + index_re = re.compile(r'([^\[]+)\[([^\]]+)\]') +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Add-VM.qmp_log.patch b/SOURCES/kvm-qemu-iotests-Add-VM.qmp_log.patch new file mode 100644 index 0000000..f367be6 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Add-VM.qmp_log.patch @@ -0,0 +1,59 @@ +From 0c5bae9e2abd50b5fbe71dbb614b586c675d77d3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:42 +0200 +Subject: [PATCH 134/268] qemu-iotests: Add VM.qmp_log() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-60-kwolf@redhat.com> +Patchwork-id: 81067 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 59/73] qemu-iotests: Add VM.qmp_log() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a helper function that logs both the QMP request and the +received response before returning it. + +Signed-off-by: Kevin Wolf +Reviewed-by: Jeff Cody +Reviewed-by: Max Reitz +(cherry picked from commit e234398a8e142fd0cfe571f7efb0e6a2f34fe73d) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 11 +++++++++++ + 1 file changed, 11 insertions(+) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 8c08d05..e3de6b0 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -216,6 +216,10 @@ def filter_qmp_event(event): + event['timestamp']['microseconds'] = 'USECS' + return event + ++def filter_testfiles(msg): ++ prefix = os.path.join(test_dir, "%s-" % (os.getpid())) ++ return msg.replace(prefix, 'TEST_DIR/PID-') ++ + def log(msg, filters=[]): + for flt in filters: + msg = flt(msg) +@@ -399,6 +403,13 @@ class VM(qtest.QEMUQtestMachine): + result.append(filter_qmp_event(ev)) + return result + ++ def qmp_log(self, cmd, filters=[filter_testfiles], **kwargs): ++ logmsg = "{'execute': '%s', 'arguments': %s}" % (cmd, kwargs) ++ log(logmsg, filters) ++ result = self.qmp(cmd, **kwargs) ++ log(str(result), filters) ++ return result ++ + + index_re = re.compile(r'([^\[]+)\[([^\]]+)\]') + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Add-VM.run_job.patch b/SOURCES/kvm-qemu-iotests-Add-VM.run_job.patch new file mode 100644 index 0000000..52d272f --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Add-VM.run_job.patch @@ -0,0 +1,61 @@ +From e589d367bbab2f9e30ea0896cf7728b8a296f407 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:44 +0200 +Subject: [PATCH 136/268] qemu-iotests: Add VM.run_job() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-62-kwolf@redhat.com> +Patchwork-id: 81080 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 61/73] qemu-iotests: Add VM.run_job() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Add an iotests.py function that runs a job and only returns when it is +destroyed. An error is logged when the job failed and job-finalize and +job-dismiss commands are issued if necessary. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit fc47d8513b45b1968b0ae32d4bb2f96d3aca066a) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 19 +++++++++++++++++++ + 1 file changed, 19 insertions(+) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index db4206f..2b47062 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -428,6 +428,25 @@ class VM(qtest.QEMUQtestMachine): + log(str(result), filters) + return result + ++ def run_job(self, job, auto_finalize=True, auto_dismiss=False): ++ while True: ++ for ev in self.get_qmp_events_filtered(wait=True): ++ if ev['event'] == 'JOB_STATUS_CHANGE': ++ status = ev['data']['status'] ++ if status == 'aborting': ++ result = self.qmp('query-jobs') ++ for j in result['return']: ++ if j['id'] == job: ++ log('Job failed: %s' % (j['error'])) ++ elif status == 'pending' and not auto_finalize: ++ self.qmp_log('job-finalize', id=job) ++ elif status == 'concluded' and not auto_dismiss: ++ self.qmp_log('job-dismiss', id=job) ++ elif status == 'null': ++ return ++ else: ++ iotests.log(ev) ++ + + index_re = re.compile(r'([^\[]+)\[([^\]]+)\]') + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Add-iotests.img_info_log.patch b/SOURCES/kvm-qemu-iotests-Add-iotests.img_info_log.patch new file mode 100644 index 0000000..fc3cd48 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Add-iotests.img_info_log.patch @@ -0,0 +1,67 @@ +From 9bc557fe1998a5e7f77fe3f39ceaf09690b3b7dc Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:43 +0200 +Subject: [PATCH 135/268] qemu-iotests: Add iotests.img_info_log() + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-61-kwolf@redhat.com> +Patchwork-id: 81082 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 60/73] qemu-iotests: Add iotests.img_info_log() +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a filter function to postprocess 'qemu-img info' input +(similar to what _img_info does), and an img_info_log() function that +calls 'qemu-img info' and logs the filtered output. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 6b605adec4d7491488d9cfb50bc256e667d8caf1) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 18 ++++++++++++++++++ + 1 file changed, 18 insertions(+) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index e3de6b0..db4206f 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -110,6 +110,12 @@ def qemu_img_pipe(*args): + sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + return subp.communicate()[0] + ++def img_info_log(filename, filter_path=None): ++ output = qemu_img_pipe('info', '-f', imgfmt, filename) ++ if not filter_path: ++ filter_path = filename ++ log(filter_img_info(output, filter_path)) ++ + def qemu_io(*args): + '''Run qemu-io and return the stdout data''' + args = qemu_io_args + list(args) +@@ -220,6 +226,18 @@ def filter_testfiles(msg): + prefix = os.path.join(test_dir, "%s-" % (os.getpid())) + return msg.replace(prefix, 'TEST_DIR/PID-') + ++def filter_img_info(output, filename): ++ lines = [] ++ for line in output.split('\n'): ++ if 'disk size' in line or 'actual-size' in line: ++ continue ++ line = line.replace(filename, 'TEST_IMG') \ ++ .replace(imgfmt, 'IMGFMT') ++ line = re.sub('iters: [0-9]+', 'iters: XXX', line) ++ line = re.sub('uuid: [-a-f0-9]+', 'uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX', line) ++ lines.append(line) ++ return '\n'.join(lines) ++ + def log(msg, filters=[]): + for flt in filters: + msg = flt(msg) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Rewrite-206-for-blockdev-create-job.patch b/SOURCES/kvm-qemu-iotests-Rewrite-206-for-blockdev-create-job.patch new file mode 100644 index 0000000..2d0f7f3 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Rewrite-206-for-blockdev-create-job.patch @@ -0,0 +1,1080 @@ +From 2f079d67036b546bd79093182814c66f246fe2cc Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:46 +0200 +Subject: [PATCH 138/268] qemu-iotests: Rewrite 206 for blockdev-create job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-64-kwolf@redhat.com> +Patchwork-id: 81077 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 63/73] qemu-iotests: Rewrite 206 for blockdev-create job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This rewrites the test case 206 to work with the new x-blockdev-create +job rather than the old synchronous version of the command. + +All of the test cases stay the same as before, but in order to be able +to implement proper job handling, the test case is rewritten in Python. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 4de110f8fd2811171013254fd7ed2384f1de87b4) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/206 | 680 ++++++++++++++++++--------------------------- + tests/qemu-iotests/206.out | 253 ++++++++++------- + tests/qemu-iotests/group | 2 +- + 3 files changed, 414 insertions(+), 521 deletions(-) + +diff --git a/tests/qemu-iotests/206 b/tests/qemu-iotests/206 +index 0a18b2b..b8cf2e7 100755 +--- a/tests/qemu-iotests/206 ++++ b/tests/qemu-iotests/206 +@@ -1,9 +1,11 @@ +-#!/bin/bash ++#!/usr/bin/env python + # + # Test qcow2 and file image creation + # + # Copyright (C) 2018 Red Hat, Inc. + # ++# Creator/Owner: Kevin Wolf ++# + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or +@@ -18,419 +20,263 @@ + # along with this program. If not, see . + # + +-# creator +-owner=kwolf@redhat.com +- +-seq=`basename $0` +-echo "QA output created by $seq" +- +-here=`pwd` +-status=1 # failure is the default! +- +-# get standard environment, filters and checks +-. ./common.rc +-. ./common.filter +- +-_supported_fmt qcow2 +-_supported_proto file +-_supported_os Linux +- +-function do_run_qemu() +-{ +- echo Testing: "$@" +- $QEMU -nographic -qmp stdio -serial none "$@" +- echo +-} +- +-function run_qemu() +-{ +- do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ +- | _filter_qemu | _filter_imgfmt \ +- | _filter_actual_image_size +-} +- +-echo +-echo "=== Successful image creation (defaults) ===" +-echo +- +-size=$((128 * 1024 * 1024)) +- +-run_qemu < +Date: Tue, 26 Jun 2018 09:48:47 +0200 +Subject: [PATCH 139/268] qemu-iotests: Rewrite 207 for blockdev-create job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-65-kwolf@redhat.com> +Patchwork-id: 81083 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 64/73] qemu-iotests: Rewrite 207 for blockdev-create job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This rewrites the test case 207 to work with the new x-blockdev-create +job rather than the old synchronous version of the command. + +Most of the test cases stay the same as before (the exception being some +improved 'size' options that allow distinguishing which command created +the image), but in order to be able to implement proper job handling, +the test case is rewritten in Python. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 00af19359e8d77e53a09de9a5d3ed6f6e149e0d2) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/207 | 440 ++++++++++++++++++++------------------------- + tests/qemu-iotests/207.out | 107 +++++------ + tests/qemu-iotests/group | 6 +- + 3 files changed, 257 insertions(+), 296 deletions(-) + +diff --git a/tests/qemu-iotests/207 b/tests/qemu-iotests/207 +index f5c7785..b595c92 100755 +--- a/tests/qemu-iotests/207 ++++ b/tests/qemu-iotests/207 +@@ -1,9 +1,11 @@ +-#!/bin/bash ++#!/usr/bin/env python + # + # Test ssh image creation + # + # Copyright (C) 2018 Red Hat, Inc. + # ++# Creator/Owner: Kevin Wolf ++# + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or +@@ -18,244 +20,198 @@ + # along with this program. If not, see . + # + +-# creator +-owner=kwolf@redhat.com +- +-seq=`basename $0` +-echo "QA output created by $seq" +- +-here=`pwd` +-status=1 # failure is the default! +- +-# get standard environment, filters and checks +-. ./common.rc +-. ./common.filter +- +-_supported_fmt raw +-_supported_proto ssh +-_supported_os Linux +- +-function do_run_qemu() +-{ +- echo Testing: "$@" +- $QEMU -nographic -qmp stdio -serial none "$@" +- echo +-} +- +-function run_qemu() +-{ +- do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ +- | _filter_qemu | _filter_imgfmt \ +- | _filter_actual_image_size +-} +- +-echo +-echo "=== Successful image creation (defaults) ===" +-echo +- +-run_qemu </dev/null | grep -v "\\^#" | +- cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1) +- +-run_qemu </dev/null | grep -v "\\^#" | +- cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1) +- +-run_qemu </dev/null | grep -v "\\^#" | ' + ++ 'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1', ++ shell=True).rstrip() ++ ++ vm.launch() ++ blockdev_create(vm, { 'driver': 'ssh', ++ 'location': { ++ 'path': disk_path, ++ 'server': { ++ 'host': '127.0.0.1', ++ 'port': '22' ++ }, ++ 'host-key-check': { ++ 'mode': 'hash', ++ 'type': 'md5', ++ 'hash': 'wrong', ++ } ++ }, ++ 'size': 2097152 }) ++ blockdev_create(vm, { 'driver': 'ssh', ++ 'location': { ++ 'path': disk_path, ++ 'server': { ++ 'host': '127.0.0.1', ++ 'port': '22' ++ }, ++ 'host-key-check': { ++ 'mode': 'hash', ++ 'type': 'md5', ++ 'hash': md5_key, ++ } ++ }, ++ 'size': 8388608 }) ++ vm.shutdown() ++ ++ iotests.img_info_log(remote_path, filter_path=disk_path) ++ ++ sha1_key = subprocess.check_output( ++ 'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' + ++ 'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1', ++ shell=True).rstrip() ++ ++ vm.launch() ++ blockdev_create(vm, { 'driver': 'ssh', ++ 'location': { ++ 'path': disk_path, ++ 'server': { ++ 'host': '127.0.0.1', ++ 'port': '22' ++ }, ++ 'host-key-check': { ++ 'mode': 'hash', ++ 'type': 'sha1', ++ 'hash': 'wrong', ++ } ++ }, ++ 'size': 2097152 }) ++ blockdev_create(vm, { 'driver': 'ssh', ++ 'location': { ++ 'path': disk_path, ++ 'server': { ++ 'host': '127.0.0.1', ++ 'port': '22' ++ }, ++ 'host-key-check': { ++ 'mode': 'hash', ++ 'type': 'sha1', ++ 'hash': sha1_key, ++ } ++ }, ++ 'size': 4194304 }) ++ vm.shutdown() ++ ++ iotests.img_info_log(remote_path, filter_path=disk_path) ++ ++ # ++ # Invalid path and user ++ # ++ iotests.log("=== Invalid path and user ===") ++ iotests.log("") ++ ++ vm.launch() ++ blockdev_create(vm, { 'driver': 'ssh', ++ 'location': { ++ 'path': '/this/is/not/an/existing/path', ++ 'server': { ++ 'host': '127.0.0.1', ++ 'port': '22' ++ }, ++ 'host-key-check': { ++ 'mode': 'none' ++ } ++ }, ++ 'size': 4194304 }) ++ blockdev_create(vm, { 'driver': 'ssh', ++ 'location': { ++ 'path': disk_path, ++ 'user': 'invalid user', ++ 'server': { ++ 'host': '127.0.0.1', ++ 'port': '22' ++ }, ++ 'host-key-check': { ++ 'mode': 'none' ++ } ++ }, ++ 'size': 4194304 }) ++ vm.shutdown() +diff --git a/tests/qemu-iotests/207.out b/tests/qemu-iotests/207.out +index 417deee..5eee17b 100644 +--- a/tests/qemu-iotests/207.out ++++ b/tests/qemu-iotests/207.out +@@ -1,75 +1,80 @@ +-QA output created by 207 +- + === Successful image creation (defaults) === + +-Testing: +-QMP_VERSION +-{"return": {}} +-{"return": {}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{u'return': {}} ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} + +-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}} ++image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} + file format: IMGFMT + virtual size: 4.0M (4194304 bytes) + +-image: TEST_DIR/t.IMGFMT ++ ++image: TEST_IMG + file format: IMGFMT + virtual size: 4.0M (4194304 bytes) + + === Test host-key-check options === + +-Testing: +-QMP_VERSION +-{"return": {}} +-{"return": {}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} ++{u'return': {}} ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} + +-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}} ++image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} + file format: IMGFMT + virtual size: 8.0M (8388608 bytes) +-Testing: +-QMP_VERSION +-{"return": {}} +-{"return": {}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +- +-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}} ++ ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'known_hosts'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{u'return': {}} ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} ++ ++image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} + file format: IMGFMT + virtual size: 4.0M (4194304 bytes) +-Testing: +-QMP_VERSION +-{"return": {}} +-{"error": {"class": "GenericError", "desc": "remote host key does not match host_key_check 'wrong'"}} +-{"return": {}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +- +-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}} ++ ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} ++{u'return': {}} ++Job failed: remote host key does not match host_key_check 'wrong' ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} ++ ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'md5', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 8388608}}} ++{u'return': {}} ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} ++ ++image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} + file format: IMGFMT + virtual size: 8.0M (8388608 bytes) +-Testing: +-QMP_VERSION +-{"return": {}} +-{"error": {"class": "GenericError", "desc": "remote host key does not match host_key_check 'wrong'"}} +-{"return": {}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +- +-image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_DIR/t.IMGFMT"}} ++ ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': 'wrong', 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 2097152}}} ++{u'return': {}} ++Job failed: remote host key does not match host_key_check 'wrong' ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} ++ ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'hash': HASH, 'type': 'sha1', 'mode': 'hash'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{u'return': {}} ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} ++ ++image: json:{"driver": "IMGFMT", "file": {"server.host": "127.0.0.1", "server.port": "22", "driver": "ssh", "path": "TEST_IMG"}} + file format: IMGFMT + virtual size: 4.0M (4194304 bytes) + + === Invalid path and user === + +-Testing: +-QMP_VERSION +-{"return": {}} +-{"error": {"class": "GenericError", "desc": "failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31)"}} +-{"error": {"class": "GenericError", "desc": "failed to authenticate using publickey authentication and the identities held by your ssh-agent"}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': '/this/is/not/an/existing/path', 'host-key-check': {'mode': 'none'}, 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{u'return': {}} ++Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31) ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} ++ ++{'execute': 'x-blockdev-create', 'arguments': {'job_id': 'job0', 'options': {'driver': 'ssh', 'location': {'path': 'TEST_DIR/PID-t.img', 'host-key-check': {'mode': 'none'}, 'user': 'invalid user', 'server': {'host': '127.0.0.1', 'port': '22'}}, 'size': 4194304}}} ++{u'return': {}} ++Job failed: failed to authenticate using publickey authentication and the identities held by your ssh-agent ++{'execute': 'job-dismiss', 'arguments': {'id': 'job0'}} ++{u'return': {}} + +-*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 3d6ae02..649291a 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -205,11 +205,11 @@ + 204 rw auto quick + 205 rw auto quick + 206 rw auto +-# TODO The following commented out tests need to be reworked to work +-# with the x-blockdev-create job +-#207 rw auto ++207 rw auto + 208 rw auto quick + 209 rw auto quick ++# TODO The following commented out tests need to be reworked to work ++# with the x-blockdev-create job + #210 rw auto + #211 rw auto quick + #212 rw auto quick +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Rewrite-210-for-blockdev-create-job.patch b/SOURCES/kvm-qemu-iotests-Rewrite-210-for-blockdev-create-job.patch new file mode 100644 index 0000000..8a13470 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Rewrite-210-for-blockdev-create-job.patch @@ -0,0 +1,745 @@ +From 1687b0798a047ac15c4d93ad32daa0378975be0e Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:48 +0200 +Subject: [PATCH 140/268] qemu-iotests: Rewrite 210 for blockdev-create job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-66-kwolf@redhat.com> +Patchwork-id: 81090 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 65/73] qemu-iotests: Rewrite 210 for blockdev-create job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This rewrites the test case 210 to work with the new x-blockdev-create +job rather than the old synchronous version of the command. + +All of the test cases stay the same as before, but in order to be able +to implement proper job handling, the test case is rewritten in Python. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 5ba141dc6f17ca0f250f107aace2df19558c8bc4) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/210 | 393 ++++++++++++++++++------------------------ + tests/qemu-iotests/210.out | 197 ++++++++++++++------- + tests/qemu-iotests/group | 2 +- + tests/qemu-iotests/iotests.py | 12 +- + 4 files changed, 314 insertions(+), 290 deletions(-) + +diff --git a/tests/qemu-iotests/210 b/tests/qemu-iotests/210 +index e607c0d..ff4fdde 100755 +--- a/tests/qemu-iotests/210 ++++ b/tests/qemu-iotests/210 +@@ -1,9 +1,11 @@ +-#!/bin/bash ++#!/usr/bin/env python + # + # Test luks and file image creation + # + # Copyright (C) 2018 Red Hat, Inc. + # ++# Creator/Owner: Kevin Wolf ++# + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or +@@ -18,230 +20,165 @@ + # along with this program. If not, see . + # + +-# creator +-owner=kwolf@redhat.com +- +-seq=`basename $0` +-echo "QA output created by $seq" +- +-here=`pwd` +-status=1 # failure is the default! +- +-# get standard environment, filters and checks +-. ./common.rc +-. ./common.filter +- +-_supported_fmt luks +-_supported_proto file +-_supported_os Linux +- +-function do_run_qemu() +-{ +- echo Testing: "$@" +- $QEMU -nographic -qmp stdio -serial none "$@" +- echo +-} +- +-function run_qemu() +-{ +- do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ +- | _filter_qemu | _filter_imgfmt \ +- | _filter_actual_image_size +-} +- +-echo +-echo "=== Successful image creation (defaults) ===" +-echo +- +-size=$((128 * 1024 * 1024)) +- +-run_qemu -object secret,id=keysec0,data="foo" <0 size"}} +-{"return": {}} +-{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "SHUTDOWN", "data": {"guest": false}} +- +-image: json:{"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "key-secret": "keysec0"} ++{'execute': 'block_resize', 'arguments': {'size': 9223372036854775296, 'node_name': 'node1'}} ++{u'error': {u'class': u'GenericError', u'desc': u'The requested file size is too large'}} ++{'execute': 'block_resize', 'arguments': {'size': 9223372036854775808L, 'node_name': 'node1'}} ++{u'error': {u'class': u'GenericError', u'desc': u"Invalid parameter type for 'size', expected: integer"}} ++{'execute': 'block_resize', 'arguments': {'size': 18446744073709551104L, 'node_name': 'node1'}} ++{u'error': {u'class': u'GenericError', u'desc': u"Invalid parameter type for 'size', expected: integer"}} ++{'execute': 'block_resize', 'arguments': {'size': -9223372036854775808, 'node_name': 'node1'}} ++{u'error': {u'class': u'GenericError', u'desc': u"Parameter 'size' expects a >0 size"}} ++image: json:{"driver": "IMGFMT", "file": {"driver": "file", "filename": "TEST_IMG"}, "key-secret": "keysec0"} + file format: IMGFMT + virtual size: 0 (0 bytes) +-*** done ++encrypted: yes ++Format specific information: ++ ivgen alg: plain64 ++ hash alg: sha256 ++ cipher alg: aes-256 ++ uuid: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX ++ cipher mode: xts ++ slots: ++ [0]: ++ active: true ++ iters: XXX ++ key offset: 4096 ++ stripes: 4000 ++ [1]: ++ active: false ++ key offset: 262144 ++ [2]: ++ active: false ++ key offset: 520192 ++ [3]: ++ active: false ++ key offset: 778240 ++ [4]: ++ active: false ++ key offset: 1036288 ++ [5]: ++ active: false ++ key offset: 1294336 ++ [6]: ++ active: false ++ key offset: 1552384 ++ [7]: ++ active: false ++ key offset: 1810432 ++ payload offset: 2068480 ++ master key iters: XXX ++ +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index 649291a..3ecdafa 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -208,9 +208,9 @@ + 207 rw auto + 208 rw auto quick + 209 rw auto quick ++210 rw auto + # TODO The following commented out tests need to be reworked to work + # with the x-blockdev-create job +-#210 rw auto + #211 rw auto quick + #212 rw auto quick + #213 rw auto quick +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 4461e1d..0dbfbfd 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -110,8 +110,16 @@ def qemu_img_pipe(*args): + sys.stderr.write('qemu-img received signal %i: %s\n' % (-exitcode, ' '.join(qemu_img_args + list(args)))) + return subp.communicate()[0] + +-def img_info_log(filename, filter_path=None): +- output = qemu_img_pipe('info', '-f', imgfmt, filename) ++def img_info_log(filename, filter_path=None, imgopts=False, extra_args=[]): ++ args = [ 'info' ] ++ if imgopts: ++ args.append('--image-opts') ++ else: ++ args += [ '-f', imgfmt ] ++ args += extra_args ++ args.append(filename) ++ ++ output = qemu_img_pipe(*args) + if not filter_path: + filter_path = filename + log(filter_img_info(output, filter_path)) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Rewrite-211-for-blockdev-create-job.patch b/SOURCES/kvm-qemu-iotests-Rewrite-211-for-blockdev-create-job.patch new file mode 100644 index 0000000..7251af7 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Rewrite-211-for-blockdev-create-job.patch @@ -0,0 +1,623 @@ +From db1ffc2d647b6e09bc93e751f744e56bcee8d835 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:49 +0200 +Subject: [PATCH 141/268] qemu-iotests: Rewrite 211 for blockdev-create job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-67-kwolf@redhat.com> +Patchwork-id: 81096 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 66/73] qemu-iotests: Rewrite 211 for blockdev-create job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This rewrites the test case 211 to work with the new x-blockdev-create +job rather than the old synchronous version of the command. + +All of the test cases stay the same as before, but in order to be able +to implement proper job handling, the test case is rewritten in Python. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit abbab72cad2eafcaf3b0f4e970add813b4264e5f) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/211 | 381 ++++++++++++++++++--------------------------- + tests/qemu-iotests/211.out | 133 +++++++++------- + tests/qemu-iotests/group | 2 +- + 3 files changed, 229 insertions(+), 287 deletions(-) + +diff --git a/tests/qemu-iotests/211 b/tests/qemu-iotests/211 +index 1edec26..b45f886 100755 +--- a/tests/qemu-iotests/211 ++++ b/tests/qemu-iotests/211 +@@ -1,9 +1,11 @@ +-#!/bin/bash ++#!/usr/bin/env python + # + # Test VDI and file image creation + # + # Copyright (C) 2018 Red Hat, Inc. + # ++# Creator/Owner: Kevin Wolf ++# + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or +@@ -18,229 +20,154 @@ + # along with this program. If not, see . + # + +-# creator +-owner=kwolf@redhat.com +- +-seq=`basename $0` +-echo "QA output created by $seq" +- +-here=`pwd` +-status=1 # failure is the default! +- +-# get standard environment, filters and checks +-. ./common.rc +-. ./common.filter +- +-_supported_fmt vdi +-_supported_proto file +-_supported_os Linux +- +-function do_run_qemu() +-{ +- echo Testing: "$@" +- $QEMU -nographic -qmp stdio -serial none "$@" +- echo +-} +- +-function run_qemu() +-{ +- do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ +- | _filter_qemu | _filter_imgfmt \ +- | _filter_actual_image_size +-} +- +-echo +-echo "=== Successful image creation (defaults) ===" +-echo +- +-size=$((128 * 1024 * 1024)) +- +-run_qemu < +Date: Tue, 26 Jun 2018 09:48:50 +0200 +Subject: [PATCH 142/268] qemu-iotests: Rewrite 212 for blockdev-create job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-68-kwolf@redhat.com> +Patchwork-id: 81100 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 67/73] qemu-iotests: Rewrite 212 for blockdev-create job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This rewrites the test case 212 to work with the new x-blockdev-create +job rather than the old synchronous version of the command. + +All of the test cases stay the same as before, but in order to be able +to implement proper job handling, the test case is rewritten in Python. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 2d7abfbeb75fbe624a8b3a2ff253dbf5674ccb65) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/212 | 483 +++++++++++++++++---------------------------- + tests/qemu-iotests/212.out | 191 +++++++++++------- + tests/qemu-iotests/group | 2 +- + 3 files changed, 295 insertions(+), 381 deletions(-) + +diff --git a/tests/qemu-iotests/212 b/tests/qemu-iotests/212 +index e5a1ba7..03cf41d 100755 +--- a/tests/qemu-iotests/212 ++++ b/tests/qemu-iotests/212 +@@ -1,9 +1,11 @@ +-#!/bin/bash ++#!/usr/bin/env python + # + # Test parallels and file image creation + # + # Copyright (C) 2018 Red Hat, Inc. + # ++# Creator/Owner: Kevin Wolf ++# + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or +@@ -18,309 +20,176 @@ + # along with this program. If not, see . + # + +-# creator +-owner=kwolf@redhat.com +- +-seq=`basename $0` +-echo "QA output created by $seq" +- +-here=`pwd` +-status=1 # failure is the default! +- +-# get standard environment, filters and checks +-. ./common.rc +-. ./common.filter +- +-_supported_fmt parallels +-_supported_proto file +-_supported_os Linux +- +-function do_run_qemu() +-{ +- echo Testing: "$@" +- $QEMU -nographic -qmp stdio -serial none "$@" +- echo +-} +- +-function run_qemu() +-{ +- do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ +- | _filter_qemu | _filter_imgfmt \ +- | _filter_actual_image_size +-} +- +-echo +-echo "=== Successful image creation (defaults) ===" +-echo +- +-size=$((128 * 1024 * 1024)) +- +-run_qemu < +Date: Tue, 26 Jun 2018 09:48:51 +0200 +Subject: [PATCH 143/268] qemu-iotests: Rewrite 213 for blockdev-create job + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-69-kwolf@redhat.com> +Patchwork-id: 81109 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 68/73] qemu-iotests: Rewrite 213 for blockdev-create job +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This rewrites the test case 213 to work with the new x-blockdev-create +job rather than the old synchronous version of the command. + +All of the test cases stay the same as before, but in order to be able +to implement proper job handling, the test case is rewritten in Python. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 0c46a69a5eb8a061134719a0a85898854eb8e533) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/213 | 520 +++++++++++++++++---------------------------- + tests/qemu-iotests/213.out | 208 +++++++++++------- + tests/qemu-iotests/group | 4 +- + 3 files changed, 319 insertions(+), 413 deletions(-) + +diff --git a/tests/qemu-iotests/213 b/tests/qemu-iotests/213 +index 3a00a0f..29d25bc 100755 +--- a/tests/qemu-iotests/213 ++++ b/tests/qemu-iotests/213 +@@ -1,9 +1,11 @@ +-#!/bin/bash ++#!/usr/bin/env python + # + # Test vhdx and file image creation + # + # Copyright (C) 2018 Red Hat, Inc. + # ++# Creator/Owner: Kevin Wolf ++# + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or +@@ -18,332 +20,190 @@ + # along with this program. If not, see . + # + +-# creator +-owner=kwolf@redhat.com +- +-seq=`basename $0` +-echo "QA output created by $seq" +- +-here=`pwd` +-status=1 # failure is the default! +- +-# get standard environment, filters and checks +-. ./common.rc +-. ./common.filter +- +-_supported_fmt vhdx +-_supported_proto file +-_supported_os Linux +- +-function do_run_qemu() +-{ +- echo Testing: "$@" +- $QEMU -nographic -qmp stdio -serial none "$@" +- echo +-} +- +-function run_qemu() +-{ +- do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qmp \ +- | _filter_qemu | _filter_imgfmt \ +- | _filter_actual_image_size +-} +- +-echo +-echo "=== Successful image creation (defaults) ===" +-echo +- +-size=$((128 * 1024 * 1024)) +- +-run_qemu < +Date: Thu, 10 Jan 2019 12:44:41 +0000 +Subject: [PATCH 11/14] qemu-iotests: Test auto-read-only with -drive and + -blockdev + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-12-kwolf@redhat.com> +Patchwork-id: 83961 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 11/12] qemu-iotests: Test auto-read-only with -drive and -blockdev +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Eric Blake + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 36f808fa15f85a894c2f6cce9df46d27e8f0f129) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/232 | 147 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/232.out | 59 ++++++++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 207 insertions(+) + create mode 100755 tests/qemu-iotests/232 + create mode 100644 tests/qemu-iotests/232.out + +diff --git a/tests/qemu-iotests/232 b/tests/qemu-iotests/232 +new file mode 100755 +index 0000000..bc2972d +--- /dev/null ++++ b/tests/qemu-iotests/232 +@@ -0,0 +1,147 @@ ++#!/bin/bash ++# ++# Test for auto-read-only ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++ ++# creator ++owner=kwolf@redhat.com ++ ++seq=`basename $0` ++echo "QA output created by $seq" ++ ++here=`pwd` ++status=1 # failure is the default! ++ ++_cleanup() ++{ ++ _cleanup_test_img ++ rm -f $TEST_IMG.snap ++} ++trap "_cleanup; exit \$status" 0 1 2 3 15 ++ ++# get standard environment, filters and checks ++. ./common.rc ++. ./common.filter ++ ++_supported_fmt generic ++_supported_proto file ++_supported_os Linux ++ ++function do_run_qemu() ++{ ++ echo Testing: "$@" ++ ( ++ if ! test -t 0; then ++ while read cmd; do ++ echo $cmd ++ done ++ fi ++ echo quit ++ ) | $QEMU -nographic -monitor stdio -nodefaults "$@" ++ echo ++} ++ ++function run_qemu() ++{ ++ do_run_qemu "$@" 2>&1 | _filter_testdir | _filter_qemu | _filter_hmp | ++ _filter_generated_node_ids | _filter_imgfmt ++} ++ ++function run_qemu_info_block() ++{ ++ echo "info block -n" | run_qemu "$@" | grep -e "(file" -e "QEMU_PROG" ++} ++ ++size=128M ++ ++_make_test_img $size ++ ++echo ++echo "=== -drive with read-write image: read-only/auto-read-only combinations ===" ++echo ++ ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=on,auto-read-only=off ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=on,auto-read-only=on ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=on ++echo ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=off,auto-read-only=off ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=off,auto-read-only=on ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=off ++echo ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,auto-read-only=off ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,auto-read-only=on ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none ++ ++echo ++echo "=== -drive with read-only image: read-only/auto-read-only combinations ===" ++echo ++ ++chmod a-w $TEST_IMG ++ ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=on,auto-read-only=off ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=on,auto-read-only=on ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=on ++echo ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=off,auto-read-only=off ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=off,auto-read-only=on ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,read-only=off ++echo ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,auto-read-only=off ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none,auto-read-only=on ++run_qemu_info_block -drive driver=file,file="$TEST_IMG",if=none ++ ++echo ++echo "=== -blockdev with read-write image: read-only/auto-read-only combinations ===" ++echo ++ ++chmod a+w $TEST_IMG ++ ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=on,auto-read-only=off ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=on,auto-read-only=on ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=on ++echo ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=off,auto-read-only=off ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=off,auto-read-only=on ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=off ++echo ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,auto-read-only=off ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,auto-read-only=on ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0 ++ ++echo ++echo "=== -blockdev with read-only image: read-only/auto-read-only combinations ===" ++echo ++ ++chmod a-w $TEST_IMG ++ ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=on,auto-read-only=off ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=on,auto-read-only=on ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=on ++echo ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=off,auto-read-only=off ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=off,auto-read-only=on ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,read-only=off ++echo ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,auto-read-only=off ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0,auto-read-only=on ++run_qemu_info_block -blockdev driver=file,filename="$TEST_IMG",node-name=node0 ++ ++# success, all done ++echo "*** done" ++rm -f $seq.full ++status=0 +diff --git a/tests/qemu-iotests/232.out b/tests/qemu-iotests/232.out +new file mode 100644 +index 0000000..dcb683a +--- /dev/null ++++ b/tests/qemu-iotests/232.out +@@ -0,0 +1,59 @@ ++QA output created by 232 ++Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 ++ ++=== -drive with read-write image: read-only/auto-read-only combinations === ++ ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++ ++NODE_NAME: TEST_DIR/t.IMGFMT (file) ++NODE_NAME: TEST_DIR/t.IMGFMT (file) ++NODE_NAME: TEST_DIR/t.IMGFMT (file) ++ ++NODE_NAME: TEST_DIR/t.IMGFMT (file) ++NODE_NAME: TEST_DIR/t.IMGFMT (file) ++NODE_NAME: TEST_DIR/t.IMGFMT (file) ++ ++=== -drive with read-only image: read-only/auto-read-only combinations === ++ ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++ ++QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,read-only=off,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++ ++QEMU_PROG: -drive driver=file,file=TEST_DIR/t.IMGFMT,if=none,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++NODE_NAME: TEST_DIR/t.IMGFMT (file, read-only) ++ ++=== -blockdev with read-write image: read-only/auto-read-only combinations === ++ ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++ ++node0: TEST_DIR/t.IMGFMT (file) ++node0: TEST_DIR/t.IMGFMT (file) ++node0: TEST_DIR/t.IMGFMT (file) ++ ++node0: TEST_DIR/t.IMGFMT (file) ++node0: TEST_DIR/t.IMGFMT (file) ++node0: TEST_DIR/t.IMGFMT (file) ++ ++=== -blockdev with read-only image: read-only/auto-read-only combinations === ++ ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++ ++QEMU_PROG: -blockdev driver=file,filename=TEST_DIR/t.IMGFMT,node-name=node0,read-only=off,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++QEMU_PROG: -blockdev driver=file,filename=TEST_DIR/t.IMGFMT,node-name=node0,read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied ++ ++QEMU_PROG: -blockdev driver=file,filename=TEST_DIR/t.IMGFMT,node-name=node0,auto-read-only=off: Could not open 'TEST_DIR/t.IMGFMT': Permission denied ++node0: TEST_DIR/t.IMGFMT (file, read-only) ++QEMU_PROG: -blockdev driver=file,filename=TEST_DIR/t.IMGFMT,node-name=node0: Could not open 'TEST_DIR/t.IMGFMT': Permission denied ++*** done +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index b30689e..61c0f9c 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -224,4 +224,5 @@ + 226 auto quick + 229 auto quick + 231 auto quick ++232 auto quick + 234 auto quick migration +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Test-commit-with-top-node-base-node.patch b/SOURCES/kvm-qemu-iotests-Test-commit-with-top-node-base-node.patch new file mode 100644 index 0000000..958d827 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Test-commit-with-top-node-base-node.patch @@ -0,0 +1,127 @@ +From 8e6b5a2696e10066c65f047624c32e40ecd8ff8c Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 13:50:55 +0100 +Subject: [PATCH 5/5] qemu-iotests: Test commit with top-node/base-node + +RH-Author: Kevin Wolf +Message-id: <20181010135055.3874-3-kwolf@redhat.com> +Patchwork-id: 82568 +O-Subject: [RHEL-8 qemu-kvm PATCH 2/2] qemu-iotests: Test commit with top-node/base-node +Bugzilla: 1637970 +RH-Acked-by: John Snow +RH-Acked-by: Fam Zheng +RH-Acked-by: Stefan Hajnoczi + +This adds some tests for block-commit with the new options top-node and +base-node (taking node names) instead of top and base (taking file +names). + +Signed-off-by: Kevin Wolf +(cherry picked from commit d57177a48fc604e5427921bf20b22ee0e6d578b3) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/qemu-iotests/040 | 52 ++++++++++++++++++++++++++++++++++++++++++++-- + tests/qemu-iotests/040.out | 4 ++-- + 2 files changed, 52 insertions(+), 4 deletions(-) + +diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040 +index 1beb5e6..1cb1cee 100755 +--- a/tests/qemu-iotests/040 ++++ b/tests/qemu-iotests/040 +@@ -57,9 +57,12 @@ class ImageCommitTestCase(iotests.QMPTestCase): + self.assert_no_active_block_jobs() + self.vm.shutdown() + +- def run_commit_test(self, top, base, need_ready=False): ++ def run_commit_test(self, top, base, need_ready=False, node_names=False): + self.assert_no_active_block_jobs() +- result = self.vm.qmp('block-commit', device='drive0', top=top, base=base) ++ if node_names: ++ result = self.vm.qmp('block-commit', device='drive0', top_node=top, base_node=base) ++ else: ++ result = self.vm.qmp('block-commit', device='drive0', top=top, base=base) + self.assert_qmp(result, 'return', {}) + self.wait_for_complete(need_ready) + +@@ -101,6 +104,11 @@ class TestSingleDrive(ImageCommitTestCase): + self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xab 0 524288', backing_img).find("verification failed")) + self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xef 524288 524288', backing_img).find("verification failed")) + ++ def test_commit_node(self): ++ self.run_commit_test("mid", "base", node_names=True) ++ self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xab 0 524288', backing_img).find("verification failed")) ++ self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xef 524288 524288', backing_img).find("verification failed")) ++ + def test_device_not_found(self): + result = self.vm.qmp('block-commit', device='nonexistent', top='%s' % mid_img) + self.assert_qmp(result, 'error/class', 'DeviceNotFound') +@@ -123,6 +131,30 @@ class TestSingleDrive(ImageCommitTestCase): + self.assert_qmp(result, 'error/class', 'GenericError') + self.assert_qmp(result, 'error/desc', 'Base \'badfile\' not found') + ++ def test_top_node_invalid(self): ++ self.assert_no_active_block_jobs() ++ result = self.vm.qmp('block-commit', device='drive0', top_node='badfile', base_node='base') ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ self.assert_qmp(result, 'error/desc', "Cannot find device= nor node_name=badfile") ++ ++ def test_base_node_invalid(self): ++ self.assert_no_active_block_jobs() ++ result = self.vm.qmp('block-commit', device='drive0', top_node='mid', base_node='badfile') ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ self.assert_qmp(result, 'error/desc', "Cannot find device= nor node_name=badfile") ++ ++ def test_top_path_and_node(self): ++ self.assert_no_active_block_jobs() ++ result = self.vm.qmp('block-commit', device='drive0', top_node='mid', base_node='base', top='%s' % mid_img) ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ self.assert_qmp(result, 'error/desc', "'top-node' and 'top' are mutually exclusive") ++ ++ def test_base_path_and_node(self): ++ self.assert_no_active_block_jobs() ++ result = self.vm.qmp('block-commit', device='drive0', top_node='mid', base_node='base', base='%s' % backing_img) ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ self.assert_qmp(result, 'error/desc', "'base-node' and 'base' are mutually exclusive") ++ + def test_top_is_active(self): + self.run_commit_test(test_img, backing_img, need_ready=True) + self.assertEqual(-1, qemu_io('-f', 'raw', '-c', 'read -P 0xab 0 524288', backing_img).find("verification failed")) +@@ -139,6 +171,22 @@ class TestSingleDrive(ImageCommitTestCase): + self.assert_qmp(result, 'error/class', 'GenericError') + self.assert_qmp(result, 'error/desc', 'Base \'%s\' not found' % mid_img) + ++ def test_top_and_base_node_reversed(self): ++ self.assert_no_active_block_jobs() ++ result = self.vm.qmp('block-commit', device='drive0', top_node='base', base_node='top') ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ self.assert_qmp(result, 'error/desc', "'top' is not in this backing file chain") ++ ++ def test_top_node_in_wrong_chain(self): ++ self.assert_no_active_block_jobs() ++ ++ result = self.vm.qmp('blockdev-add', driver='null-co', node_name='null') ++ self.assert_qmp(result, 'return', {}) ++ ++ result = self.vm.qmp('block-commit', device='drive0', top_node='null', base_node='base') ++ self.assert_qmp(result, 'error/class', 'GenericError') ++ self.assert_qmp(result, 'error/desc', "'null' is not in this backing file chain") ++ + # When the job is running on a BB that is automatically deleted on hot + # unplug, the job is cancelled when the device disappears + def test_hot_unplug(self): +diff --git a/tests/qemu-iotests/040.out b/tests/qemu-iotests/040.out +index e20a75c..802ffaa 100644 +--- a/tests/qemu-iotests/040.out ++++ b/tests/qemu-iotests/040.out +@@ -1,5 +1,5 @@ +-............................. ++........................................... + ---------------------------------------------------------------------- +-Ran 29 tests ++Ran 43 tests + + OK +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Test-job-with-block-jobs.patch b/SOURCES/kvm-qemu-iotests-Test-job-with-block-jobs.patch new file mode 100644 index 0000000..39ec49d --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Test-job-with-block-jobs.patch @@ -0,0 +1,589 @@ +From afd502677f82814b106b0095104b350f982c067f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:36 +0200 +Subject: [PATCH 128/268] qemu-iotests: Test job-* with block jobs + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-54-kwolf@redhat.com> +Patchwork-id: 81101 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 53/73] qemu-iotests: Test job-* with block jobs +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds a test case that tests the new job-* QMP commands with +mirror and backup block jobs. + +Signed-off-by: Kevin Wolf +(cherry picked from commit bdebdc712b06ba82e103d617c335830682cde242) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/219 | 209 +++++++++++++++++++++++++++++ + tests/qemu-iotests/219.out | 327 +++++++++++++++++++++++++++++++++++++++++++++ + tests/qemu-iotests/group | 1 + + 3 files changed, 537 insertions(+) + create mode 100755 tests/qemu-iotests/219 + create mode 100644 tests/qemu-iotests/219.out + +diff --git a/tests/qemu-iotests/219 b/tests/qemu-iotests/219 +new file mode 100755 +index 0000000..898a26e +--- /dev/null ++++ b/tests/qemu-iotests/219 +@@ -0,0 +1,209 @@ ++#!/usr/bin/env python ++# ++# Copyright (C) 2018 Red Hat, Inc. ++# ++# This program is free software; you can redistribute it and/or modify ++# it under the terms of the GNU General Public License as published by ++# the Free Software Foundation; either version 2 of the License, or ++# (at your option) any later version. ++# ++# This program is distributed in the hope that it will be useful, ++# but WITHOUT ANY WARRANTY; without even the implied warranty of ++# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++# GNU General Public License for more details. ++# ++# You should have received a copy of the GNU General Public License ++# along with this program. If not, see . ++# ++# Creator/Owner: Kevin Wolf ++# ++# Check using the job-* QMP commands with block jobs ++ ++import iotests ++ ++iotests.verify_image_format(supported_fmts=['qcow2']) ++ ++def pause_wait(vm, job_id): ++ with iotests.Timeout(3, "Timeout waiting for job to pause"): ++ while True: ++ result = vm.qmp('query-jobs') ++ for job in result['return']: ++ if job['id'] == job_id and job['status'] in ['paused', 'standby']: ++ return job ++ ++# Test that block-job-pause/resume and job-pause/resume can be mixed ++def test_pause_resume(vm): ++ for pause_cmd, pause_arg in [('block-job-pause', 'device'), ++ ('job-pause', 'id')]: ++ for resume_cmd, resume_arg in [('block-job-resume', 'device'), ++ ('job-resume', 'id')]: ++ iotests.log('=== Testing %s/%s ===' % (pause_cmd, resume_cmd)) ++ ++ iotests.log(vm.qmp(pause_cmd, **{pause_arg: 'job0'})) ++ pause_wait(vm, 'job0') ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ iotests.log(vm.qmp('query-jobs')) ++ ++ iotests.log(vm.qmp(resume_cmd, **{resume_arg: 'job0'})) ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ iotests.log(vm.qmp('query-jobs')) ++ ++def test_job_lifecycle(vm, job, job_args, has_ready=False): ++ iotests.log('') ++ iotests.log('') ++ iotests.log('Starting block job: %s (auto-finalize: %s; auto-dismiss: %s)' % ++ (job, ++ job_args.get('auto-finalize', True), ++ job_args.get('auto-dismiss', True))) ++ iotests.log(vm.qmp(job, job_id='job0', **job_args)) ++ ++ # Depending on the storage, the first request may or may not have completed ++ # yet, so filter out the progress. Later query-job calls don't need the ++ # filtering because the progress is made deterministic by the block job ++ # speed ++ result = vm.qmp('query-jobs') ++ for j in result['return']: ++ del j['current-progress'] ++ iotests.log(result) ++ ++ # undefined -> created -> running ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ ++ # RUNNING state: ++ # pause/resume should work, complete/finalize/dismiss should error out ++ iotests.log('') ++ iotests.log('Pause/resume in RUNNING') ++ test_pause_resume(vm) ++ ++ iotests.log(vm.qmp('job-complete', id='job0')) ++ iotests.log(vm.qmp('job-finalize', id='job0')) ++ iotests.log(vm.qmp('job-dismiss', id='job0')) ++ ++ iotests.log(vm.qmp('block-job-complete', device='job0')) ++ iotests.log(vm.qmp('block-job-finalize', id='job0')) ++ iotests.log(vm.qmp('block-job-dismiss', id='job0')) ++ ++ # Let the job complete (or transition to READY if it supports that) ++ iotests.log(vm.qmp('block-job-set-speed', device='job0', speed=0)) ++ if has_ready: ++ iotests.log('') ++ iotests.log('Waiting for READY state...') ++ vm.event_wait('BLOCK_JOB_READY') ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ iotests.log(vm.qmp('query-jobs')) ++ ++ # READY state: ++ # pause/resume/complete should work, finalize/dismiss should error out ++ iotests.log('') ++ iotests.log('Pause/resume in READY') ++ test_pause_resume(vm) ++ ++ iotests.log(vm.qmp('job-finalize', id='job0')) ++ iotests.log(vm.qmp('job-dismiss', id='job0')) ++ ++ iotests.log(vm.qmp('block-job-finalize', id='job0')) ++ iotests.log(vm.qmp('block-job-dismiss', id='job0')) ++ ++ # Transition to WAITING ++ iotests.log(vm.qmp('job-complete', id='job0')) ++ ++ # Move to WAITING and PENDING state ++ iotests.log('') ++ iotests.log('Waiting for PENDING state...') ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ ++ if not job_args.get('auto-finalize', True): ++ # PENDING state: ++ # finalize should work, pause/complete/dismiss should error out ++ iotests.log(vm.qmp('query-jobs')) ++ ++ iotests.log(vm.qmp('job-pause', id='job0')) ++ iotests.log(vm.qmp('job-complete', id='job0')) ++ iotests.log(vm.qmp('job-dismiss', id='job0')) ++ ++ iotests.log(vm.qmp('block-job-pause', device='job0')) ++ iotests.log(vm.qmp('block-job-complete', device='job0')) ++ iotests.log(vm.qmp('block-job-dismiss', id='job0')) ++ ++ # Transition to CONCLUDED ++ iotests.log(vm.qmp('job-finalize', id='job0')) ++ ++ ++ # Move to CONCLUDED state ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ ++ if not job_args.get('auto-dismiss', True): ++ # CONCLUDED state: ++ # dismiss should work, pause/complete/finalize should error out ++ iotests.log(vm.qmp('query-jobs')) ++ ++ iotests.log(vm.qmp('job-pause', id='job0')) ++ iotests.log(vm.qmp('job-complete', id='job0')) ++ iotests.log(vm.qmp('job-finalize', id='job0')) ++ ++ iotests.log(vm.qmp('block-job-pause', device='job0')) ++ iotests.log(vm.qmp('block-job-complete', device='job0')) ++ iotests.log(vm.qmp('block-job-finalize', id='job0')) ++ ++ # Transition to NULL ++ iotests.log(vm.qmp('job-dismiss', id='job0')) ++ ++ # Move to NULL state ++ iotests.log(iotests.filter_qmp_event(vm.event_wait('JOB_STATUS_CHANGE'))) ++ iotests.log(vm.qmp('query-jobs')) ++ ++ ++with iotests.FilePath('disk.img') as disk_path, \ ++ iotests.FilePath('copy.img') as copy_path, \ ++ iotests.VM() as vm: ++ ++ img_size = '4M' ++ iotests.qemu_img_create('-f', iotests.imgfmt, disk_path, img_size) ++ iotests.qemu_io('-c', 'write 0 %s' % (img_size), ++ '-f', iotests.imgfmt, disk_path) ++ ++ iotests.log('Launching VM...') ++ vm.add_blockdev(vm.qmp_to_opts({ ++ 'driver': iotests.imgfmt, ++ 'node-name': 'drive0-node', ++ 'file': { ++ 'driver': 'file', ++ 'filename': disk_path, ++ }, ++ })) ++ vm.launch() ++ ++ # In order to keep things deterministic (especially progress in query-job, ++ # but related to this also automatic state transitions like job ++ # completion), but still get pause points often enough to avoid making this ++ # test very slow, it's important to have the right ratio between speed and ++ # buf_size. ++ # ++ # For backup, buf_size is hard-coded to the source image cluster size (64k), ++ # so we'll pick the same for mirror. The slice time, i.e. the granularity ++ # of the rate limiting is 100ms. With a speed of 256k per second, we can ++ # get four pause points per second. This gives us 250ms per iteration, ++ # which should be enough to stay deterministic. ++ ++ test_job_lifecycle(vm, 'drive-mirror', has_ready=True, job_args={ ++ 'device': 'drive0-node', ++ 'target': copy_path, ++ 'sync': 'full', ++ 'speed': 262144, ++ 'buf_size': 65536, ++ }) ++ ++ for auto_finalize in [True, False]: ++ for auto_dismiss in [True, False]: ++ test_job_lifecycle(vm, 'drive-backup', job_args={ ++ 'device': 'drive0-node', ++ 'target': copy_path, ++ 'sync': 'full', ++ 'speed': 262144, ++ 'auto-finalize': auto_finalize, ++ 'auto-dismiss': auto_dismiss, ++ }) ++ ++ vm.shutdown() +diff --git a/tests/qemu-iotests/219.out b/tests/qemu-iotests/219.out +new file mode 100644 +index 0000000..346801b +--- /dev/null ++++ b/tests/qemu-iotests/219.out +@@ -0,0 +1,327 @@ ++Launching VM... ++ ++ ++Starting block job: drive-mirror (auto-finalize: True; auto-dismiss: True) ++{u'return': {}} ++{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++ ++Pause/resume in RUNNING ++=== Testing block-job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++=== Testing block-job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++=== Testing job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++=== Testing job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++ ++Waiting for READY state... ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++ ++Pause/resume in READY ++=== Testing block-job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++=== Testing block-job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++=== Testing job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++=== Testing job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'standby', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'standby', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'ready', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'ready', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'mirror'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'ready' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++ ++Waiting for PENDING state... ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': []} ++ ++ ++Starting block job: drive-backup (auto-finalize: True; auto-dismiss: True) ++{u'return': {}} ++{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++ ++Pause/resume in RUNNING ++=== Testing block-job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing block-job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++ ++Waiting for PENDING state... ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': []} ++ ++ ++Starting block job: drive-backup (auto-finalize: True; auto-dismiss: False) ++{u'return': {}} ++{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++ ++Pause/resume in RUNNING ++=== Testing block-job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing block-job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++ ++Waiting for PENDING state... ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'concluded', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': []} ++ ++ ++Starting block job: drive-backup (auto-finalize: False; auto-dismiss: True) ++{u'return': {}} ++{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++ ++Pause/resume in RUNNING ++=== Testing block-job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing block-job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++ ++Waiting for PENDING state... ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'pending', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': []} ++ ++ ++Starting block job: drive-backup (auto-finalize: False; auto-dismiss: False) ++{u'return': {}} ++{u'return': [{u'status': u'running', u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'created', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++ ++Pause/resume in RUNNING ++=== Testing block-job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 65536, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing block-job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 131072, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/block-job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 196608, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++=== Testing job-pause/job-resume === ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'paused', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'paused', u'current-progress': 262144, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'running', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'running', u'current-progress': 327680, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'running' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++ ++Waiting for PENDING state... ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'waiting', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'pending', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'pending', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'pending' cannot accept command verb 'dismiss'"}} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'concluded', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': [{u'status': u'concluded', u'current-progress': 4194304, u'total-progress': 4194304, u'id': u'job0', u'type': u'backup'}]} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'pause'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'complete'"}} ++{u'error': {u'class': u'GenericError', u'desc': u"Job 'job0' in state 'concluded' cannot accept command verb 'finalize'"}} ++{u'return': {}} ++{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'status': u'null', u'id': u'job0'}, u'event': u'JOB_STATUS_CHANGE'} ++{u'return': []} +diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group +index cd7bc29..5c55adc 100644 +--- a/tests/qemu-iotests/group ++++ b/tests/qemu-iotests/group +@@ -217,3 +217,4 @@ + 216 rw auto quick + 217 rw auto quick + 218 rw auto quick ++219 rw auto +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-Test-qcow2-not-leaking-clusters-on-writ.patch b/SOURCES/kvm-qemu-iotests-Test-qcow2-not-leaking-clusters-on-writ.patch new file mode 100644 index 0000000..1d68fd4 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-Test-qcow2-not-leaking-clusters-on-writ.patch @@ -0,0 +1,95 @@ +From 59a0b65ebe7148f5b2b46d9436df93f6fc52db1a Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Mon, 2 Jul 2018 15:40:08 +0200 +Subject: [PATCH 184/268] qemu-iotests: Test qcow2 not leaking clusters on + write error + +RH-Author: Kevin Wolf +Message-id: <20180702154008.15533-4-kwolf@redhat.com> +Patchwork-id: 81187 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/3] qemu-iotests: Test qcow2 not leaking clusters on write error +Bugzilla: 1528541 +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng +RH-Acked-by: Stefan Hajnoczi + +This adds a test for a temporary write failure, which simulates the +situation after werror=stop/enospc has stopped the VM. We shouldn't +leave leaked clusters behind in such cases. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit ae376c6255d0eee4b3c4d60acc4679aa99c0d2c8) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/026 | 17 +++++++++++++++++ + tests/qemu-iotests/026.out | 8 ++++++++ + tests/qemu-iotests/026.out.nocache | 8 ++++++++ + 3 files changed, 33 insertions(+) + +diff --git a/tests/qemu-iotests/026 b/tests/qemu-iotests/026 +index 7fadfba..582d254 100755 +--- a/tests/qemu-iotests/026 ++++ b/tests/qemu-iotests/026 +@@ -200,6 +200,23 @@ done + done + done + ++echo ++echo === Avoid cluster leaks after temporary failure === ++echo ++ ++cat > "$TEST_DIR/blkdebug.conf" < +Date: Mon, 2 Jul 2018 15:40:06 +0200 +Subject: [PATCH 182/268] qemu-iotests: Update 026.out.nocache reference output + +RH-Author: Kevin Wolf +Message-id: <20180702154008.15533-2-kwolf@redhat.com> +Patchwork-id: 81186 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/3] qemu-iotests: Update 026.out.nocache reference output +Bugzilla: 1528541 +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng +RH-Acked-by: Stefan Hajnoczi + +Commit abf754fe406 updated 026.out, but forgot to also update +026.out.nocache. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit 93a3642efcadf4ad6045ccea38a05ff5297dfe26) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/026.out.nocache | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +diff --git a/tests/qemu-iotests/026.out.nocache b/tests/qemu-iotests/026.out.nocache +index ea2e166..650ccd8 100644 +--- a/tests/qemu-iotests/026.out.nocache ++++ b/tests/qemu-iotests/026.out.nocache +@@ -541,7 +541,7 @@ Failed to flush the L2 table cache: No space left on device + Failed to flush the refcount block cache: No space left on device + write failed: No space left on device + +-11 leaked clusters were found on the image. ++10 leaked clusters were found on the image. + This means waste of disk space, but no harm to data. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 + +@@ -569,7 +569,7 @@ Failed to flush the L2 table cache: No space left on device + Failed to flush the refcount block cache: No space left on device + write failed: No space left on device + +-11 leaked clusters were found on the image. ++10 leaked clusters were found on the image. + This means waste of disk space, but no harm to data. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 + +@@ -597,7 +597,7 @@ Failed to flush the L2 table cache: No space left on device + Failed to flush the refcount block cache: No space left on device + write failed: No space left on device + +-11 leaked clusters were found on the image. ++10 leaked clusters were found on the image. + This means waste of disk space, but no harm to data. + Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-iotests.py-helper-for-non-file-protocol.patch b/SOURCES/kvm-qemu-iotests-iotests.py-helper-for-non-file-protocol.patch new file mode 100644 index 0000000..eca8582 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-iotests.py-helper-for-non-file-protocol.patch @@ -0,0 +1,66 @@ +From ff9bd6b6b80a3d97091f95e9df634e583bdada71 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:45 +0200 +Subject: [PATCH 137/268] qemu-iotests: iotests.py helper for non-file + protocols + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-63-kwolf@redhat.com> +Patchwork-id: 81081 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 62/73] qemu-iotests: iotests.py helper for non-file protocols +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +This adds two helper functions that are useful for test cases that make +use of a non-file protocol (specifically ssh). + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 5a259e868b308564de9f2bfaef0606074a4f9028) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/iotests.py | 17 +++++++++++++++++ + 1 file changed, 17 insertions(+) + +diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py +index 2b47062..4461e1d 100644 +--- a/tests/qemu-iotests/iotests.py ++++ b/tests/qemu-iotests/iotests.py +@@ -313,6 +313,13 @@ def file_path(*names): + + return paths[0] if len(paths) == 1 else paths + ++def remote_filename(path): ++ if imgproto == 'file': ++ return path ++ elif imgproto == 'ssh': ++ return "ssh://127.0.0.1%s" % (path) ++ else: ++ raise Exception("Protocol %s not supported" % (imgproto)) + + class VM(qtest.QEMUQtestMachine): + '''A QEMU VM''' +@@ -611,6 +618,16 @@ def verify_image_format(supported_fmts=[], unsupported_fmts=[]): + if not_sup or (imgfmt in unsupported_fmts): + notrun('not suitable for this image format: %s' % imgfmt) + ++def verify_protocol(supported=[], unsupported=[]): ++ assert not (supported and unsupported) ++ ++ if 'generic' in supported: ++ return ++ ++ not_sup = supported and (imgproto not in supported) ++ if not_sup or (imgproto in unsupported): ++ notrun('not suitable for this protocol: %s' % imgproto) ++ + def verify_platform(supported_oses=['linux']): + if True not in [sys.platform.startswith(x) for x in supported_oses]: + notrun('not suitable for this OS: %s' % sys.platform) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-iotests-reduce-chance-of-races-in-185.patch b/SOURCES/kvm-qemu-iotests-reduce-chance-of-races-in-185.patch new file mode 100644 index 0000000..8378a51 --- /dev/null +++ b/SOURCES/kvm-qemu-iotests-reduce-chance-of-races-in-185.patch @@ -0,0 +1,90 @@ +From 882087e49dc3c3286db14e7c98f3dc557a7af235 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:45 +0200 +Subject: [PATCH 077/268] qemu-iotests: reduce chance of races in 185 + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-3-kwolf@redhat.com> +Patchwork-id: 81061 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 02/73] qemu-iotests: reduce chance of races in 185 +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +From: Stefan Hajnoczi + +Commit 8565c3ab537e78f3e69977ec2c609dc9417a806e ("qemu-iotests: fix +185") identified a race condition in a sub-test. + +Similar issues also affect the other sub-tests. If disk I/O completes +quickly, it races with the QMP 'quit' command. This causes spurious +test failures because QMP events are emitted in an unpredictable order. + +This test relies on QEMU internals and there is no QMP API for getting +deterministic behavior needed to make this test 100% reliable. At the +same time, the test is useful and it would be a shame to remove it. + +Add sleep 0.5 to reduce the chance of races. This is not a real fix but +appears to reduce spurious failures in practice. + +Cc: Vladimir Sementsov-Ogievskiy +Signed-off-by: Stefan Hajnoczi +Reviewed-by: Eric Blake +Message-id: 20180508135436.30140-2-stefanha@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit ddf2d98a94c8a98a661a217fb629cfd15f4dcec7) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + tests/qemu-iotests/185 | 12 ++++++++++++ + 1 file changed, 12 insertions(+) + +diff --git a/tests/qemu-iotests/185 b/tests/qemu-iotests/185 +index 298d88d..975404c 100755 +--- a/tests/qemu-iotests/185 ++++ b/tests/qemu-iotests/185 +@@ -118,6 +118,9 @@ _send_qemu_cmd $h \ + 'speed': 65536 } }" \ + "return" + ++# If we don't sleep here 'quit' command races with disk I/O ++sleep 0.5 ++ + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" + wait=1 _cleanup_qemu + +@@ -137,6 +140,9 @@ _send_qemu_cmd $h \ + 'speed': 65536 } }" \ + "return" + ++# If we don't sleep here 'quit' command races with disk I/O ++sleep 0.5 ++ + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" + wait=1 _cleanup_qemu + +@@ -183,6 +189,9 @@ _send_qemu_cmd $h \ + 'speed': 65536 } }" \ + "return" + ++# If we don't sleep here 'quit' command races with disk I/O ++sleep 0.5 ++ + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" + wait=1 _cleanup_qemu + +@@ -201,6 +210,9 @@ _send_qemu_cmd $h \ + 'speed': 65536 } }" \ + "return" + ++# If we don't sleep here 'quit' command races with disk I/O ++sleep 0.5 ++ + _send_qemu_cmd $h "{ 'execute': 'quit' }" "return" + wait=1 _cleanup_qemu + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-option-Pull-out-Supported-options-print.patch b/SOURCES/kvm-qemu-option-Pull-out-Supported-options-print.patch new file mode 100644 index 0000000..0a7318d --- /dev/null +++ b/SOURCES/kvm-qemu-option-Pull-out-Supported-options-print.patch @@ -0,0 +1,56 @@ +From b8370cda9a44259acaba5f1760c0c8e58d52faaa Mon Sep 17 00:00:00 2001 +From: Max Reitz +Date: Mon, 18 Jun 2018 14:59:39 +0200 +Subject: [PATCH 066/268] qemu-option: Pull out "Supported options" print + +RH-Author: Max Reitz +Message-id: <20180618145943.4489-4-mreitz@redhat.com> +Patchwork-id: 80756 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 3/7] qemu-option: Pull out "Supported options" print +Bugzilla: 1537956 +RH-Acked-by: John Snow +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +It really is up to the caller to decide what this list of options means. + +Signed-off-by: Max Reitz +Reviewed-by: John Snow +Reviewed-by: Eric Blake +Message-id: 20180509210023.20283-4-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 7f3fb00136aaec74df8132492d1f48b8e9840258) +Signed-off-by: Max Reitz +Signed-off-by: Miroslav Rezanina +--- + qemu-img.c | 1 + + util/qemu-option.c | 1 - + 2 files changed, 1 insertion(+), 1 deletion(-) + +diff --git a/qemu-img.c b/qemu-img.c +index e40d6ff..cdeb01b 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -261,6 +261,7 @@ static int print_block_option_help(const char *filename, const char *fmt) + create_opts = qemu_opts_append(create_opts, proto_drv->create_opts); + } + ++ printf("Supported options:\n"); + qemu_opts_print_help(create_opts); + qemu_opts_free(create_opts); + return 0; +diff --git a/util/qemu-option.c b/util/qemu-option.c +index d0756fd..95e6cf4 100644 +--- a/util/qemu-option.c ++++ b/util/qemu-option.c +@@ -218,7 +218,6 @@ void qemu_opts_print_help(QemuOptsList *list) + + assert(list); + desc = list->desc; +- printf("Supported options:\n"); + while (desc && desc->name) { + printf("%-16s %s\n", desc->name, + desc->help ? desc->help : "No description available"); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qemu-options-Add-missing-newline-to-accel-help-text.patch b/SOURCES/kvm-qemu-options-Add-missing-newline-to-accel-help-text.patch new file mode 100644 index 0000000..687054f --- /dev/null +++ b/SOURCES/kvm-qemu-options-Add-missing-newline-to-accel-help-text.patch @@ -0,0 +1,54 @@ +From b1b31310c45e28770c779969e2a3692daa3eb28c Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Tue, 3 Jul 2018 13:27:24 +0200 +Subject: [PATCH 185/268] qemu-options: Add missing newline to -accel help text + +RH-Author: Eduardo Habkost +Message-id: <20180703132724.21964-2-ehabkost@redhat.com> +Patchwork-id: 81208 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] qemu-options: Add missing newline to -accel help text +Bugzilla: 1586313 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Igor Mammedov + +The newline was removed by commit c97d6d2c, and broke -help output: + +Before this patch: + + $ qemu-system-x86_64 -help | grep smp + thread=single|multi (enable multi-threaded TCG)-smp [...] + +After this patch: + + $ qemu-system-x86_64 -help | grep smp + -smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,sockets=sockets] + +Fixes: c97d6d2cdf97edb4aebe832fdba65d701ad7bcb6 +Cc: Sergio Andres Gomez Del Real +Signed-off-by: Eduardo Habkost +Message-Id: <20180611195607.3015-1-ehabkost@redhat.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit 0b3c5c81bf0a9e32fd08c532acde3caa446b3712) +Signed-off-by: Eduardo Habkost +Signed-off-by: Miroslav Rezanina +--- + qemu-options.hx | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/qemu-options.hx b/qemu-options.hx +index 43f10b1..4271cd3 100644 +--- a/qemu-options.hx ++++ b/qemu-options.hx +@@ -130,7 +130,7 @@ ETEXI + DEF("accel", HAS_ARG, QEMU_OPTION_accel, + "-accel [accel=]accelerator[,thread=single|multi]\n" + " select accelerator (kvm, xen, hax, hvf, whpx or tcg; use 'help' for a list)\n" +- " thread=single|multi (enable multi-threaded TCG)", QEMU_ARCH_ALL) ++ " thread=single|multi (enable multi-threaded TCG)\n", QEMU_ARCH_ALL) + STEXI + @item -accel @var{name}[,prop=@var{value}[,...]] + @findex -accel +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qga-fix-driver-leak-in-guest-get-fsinfo.patch b/SOURCES/kvm-qga-fix-driver-leak-in-guest-get-fsinfo.patch new file mode 100644 index 0000000..dc04994 --- /dev/null +++ b/SOURCES/kvm-qga-fix-driver-leak-in-guest-get-fsinfo.patch @@ -0,0 +1,48 @@ +From bc451d7850fab973418fa083527b59f7d4fe1779 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Tue, 5 Feb 2019 11:25:51 +0000 +Subject: [PATCH 2/2] qga: fix 'driver' leak in guest-get-fsinfo +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20190205112551.14763-3-marcandre.lureau@redhat.com> +Patchwork-id: 84242 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/2] qga: fix 'driver' leak in guest-get-fsinfo +Bugzilla: 1666952 +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Danilo de Paula + +'driver' is leaked when the loop is not broken. + +Leak introduced by commit 743c71d03c20d64f2bae5fba6f26cdf5e4b1bda6, +spotted by ASAN. + +Signed-off-by: Marc-André Lureau +Reviewed-by: Laszlo Ersek +Signed-off-by: Michael Roth + +(cherry picked from commit bb23a7362a7942739f080990a53e44afc319e36c) +Signed-off-by: Marc-André Lureau +Signed-off-by: Danilo C. L. de Paula +--- + qga/commands-posix.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/qga/commands-posix.c b/qga/commands-posix.c +index 71cb644..28b1c4c 100644 +--- a/qga/commands-posix.c ++++ b/qga/commands-posix.c +@@ -897,6 +897,7 @@ static void build_guest_fsinfo_for_real_device(char const *syspath, + break; + } + ++ g_free(driver); + if (sscanf(p, "/%x:%x:%x.%x%n", + pci, pci + 1, pci + 2, pci + 3, &pcilen) == 4) { + p += pcilen; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qga-linux-report-disk-serial-number.patch b/SOURCES/kvm-qga-linux-report-disk-serial-number.patch new file mode 100644 index 0000000..6d5b50b --- /dev/null +++ b/SOURCES/kvm-qga-linux-report-disk-serial-number.patch @@ -0,0 +1,154 @@ +From dab127e2324fdfbaf5d595144ccf0d0bfcf73074 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Sun, 4 Nov 2018 15:45:27 +0000 +Subject: [PATCH 02/35] qga: linux: report disk serial number +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20181104154528.19241-3-marcandre.lureau@redhat.com> +Patchwork-id: 82927 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/3] qga: linux: report disk serial number +Bugzilla: 1636185 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Markus Armbruster + +From: Tomáš Golembiovský + +Add reporting of disk serial number on Linux guests. The feature depends +on libudev. + +Example: + + { + "name": "dm-2", + "mountpoint": "/", + ... + "disk": [ + { + "serial": "SAMSUNG_MZ7LN512HCHP-000L1_S1ZKNXAG822493", + ... + } + ], + } + +Signed-off-by: Tomáš Golembiovský +Signed-off-by: Michael Roth + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1636185 + +(cherry picked from commit b616105a904bc350a409e12c39af0ae210900124) + +Signed-off-by: Marc-André Lureau +Signed-off-by: Danilo C. L. de Paula +--- + qga/Makefile.objs | 1 + + qga/commands-posix.c | 32 ++++++++++++++++++++++++++++++-- + qga/qapi-schema.json | 4 +++- + 3 files changed, 34 insertions(+), 3 deletions(-) + +diff --git a/qga/Makefile.objs b/qga/Makefile.objs +index ed08c59..80e6bb3 100644 +--- a/qga/Makefile.objs ++++ b/qga/Makefile.objs +@@ -1,3 +1,4 @@ ++commands-posix.o-libs := $(LIBUDEV_LIBS) + qga-obj-y = commands.o guest-agent-command-state.o main.o + qga-obj-$(CONFIG_POSIX) += commands-posix.o channel-posix.o + qga-obj-$(CONFIG_WIN32) += commands-win32.o channel-win32.o service-win32.o +diff --git a/qga/commands-posix.c b/qga/commands-posix.c +index 0dc219d..c151891 100644 +--- a/qga/commands-posix.c ++++ b/qga/commands-posix.c +@@ -47,6 +47,10 @@ extern char **environ; + #include + #include + ++#ifdef CONFIG_LIBUDEV ++#include ++#endif ++ + #ifdef FIFREEZE + #define CONFIG_FSFREEZE + #endif +@@ -871,6 +875,10 @@ static void build_guest_fsinfo_for_real_device(char const *syspath, + GuestDiskAddressList *list = NULL; + bool has_ata = false, has_host = false, has_tgt = false; + char *p, *q, *driver = NULL; ++#ifdef CONFIG_LIBUDEV ++ struct udev *udev = NULL; ++ struct udev_device *udevice = NULL; ++#endif + + p = strstr(syspath, "/devices/pci"); + if (!p || sscanf(p + 12, "%*x:%*x/%x:%x:%x.%x%n", +@@ -919,6 +927,21 @@ static void build_guest_fsinfo_for_real_device(char const *syspath, + list = g_malloc0(sizeof(*list)); + list->value = disk; + ++#ifdef CONFIG_LIBUDEV ++ udev = udev_new(); ++ udevice = udev_device_new_from_syspath(udev, syspath); ++ if (udev == NULL || udevice == NULL) { ++ g_debug("failed to query udev"); ++ } else { ++ const char *serial; ++ serial = udev_device_get_property_value(udevice, "ID_SERIAL"); ++ if (serial != NULL && *serial != 0) { ++ disk->serial = g_strdup(serial); ++ disk->has_serial = true; ++ } ++ } ++#endif ++ + if (strcmp(driver, "ata_piix") == 0) { + /* a host per ide bus, target*:0::0 */ + if (!has_host || !has_tgt) { +@@ -978,14 +1001,19 @@ static void build_guest_fsinfo_for_real_device(char const *syspath, + + list->next = fs->disk; + fs->disk = list; +- g_free(driver); +- return; ++ goto out; + + cleanup: + if (list) { + qapi_free_GuestDiskAddressList(list); + } ++out: + g_free(driver); ++#ifdef CONFIG_LIBUDEV ++ udev_unref(udev); ++ udev_device_unref(udevice); ++#endif ++ return; + } + + static void build_guest_fsinfo_for_device(char const *devpath, +diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json +index 17884c7..426528c 100644 +--- a/qga/qapi-schema.json ++++ b/qga/qapi-schema.json +@@ -832,13 +832,15 @@ + # @bus: bus id + # @target: target id + # @unit: unit id ++# @serial: serial number (since: 3.1) + # + # Since: 2.2 + ## + { 'struct': 'GuestDiskAddress', + 'data': {'pci-controller': 'GuestPCIAddress', + 'bus-type': 'GuestDiskBusType', +- 'bus': 'int', 'target': 'int', 'unit': 'int'} } ++ 'bus': 'int', 'target': 'int', 'unit': 'int', ++ '*serial': 'str'} } + + ## + # @GuestFilesystemInfo: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qga-linux-return-disk-device-in-guest-get-fsinfo.patch b/SOURCES/kvm-qga-linux-return-disk-device-in-guest-get-fsinfo.patch new file mode 100644 index 0000000..fcb2aab --- /dev/null +++ b/SOURCES/kvm-qga-linux-return-disk-device-in-guest-get-fsinfo.patch @@ -0,0 +1,78 @@ +From 5f01e33acc56e6b82bee691470d5cb19d59bec5c Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Sun, 4 Nov 2018 15:45:28 +0000 +Subject: [PATCH 03/35] qga: linux: return disk device in guest-get-fsinfo +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20181104154528.19241-4-marcandre.lureau@redhat.com> +Patchwork-id: 82926 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 3/3] qga: linux: return disk device in guest-get-fsinfo +Bugzilla: 1636185 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Markus Armbruster + +From: Tomáš Golembiovský + +Report device node of the disk on Linux (e.g. "/dev/sda2"). +Requirs libudev. + +Signed-off-by: Tomáš Golembiovský +Signed-off-by: Michael Roth + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1636185 + +(cherry picked from commit 6589ce35734e7e71463485650e5fb6e4bbe64729) + +Signed-off-by: Marc-André Lureau +Signed-off-by: Danilo C. L. de Paula +--- + qga/commands-posix.c | 7 ++++++- + qga/qapi-schema.json | 3 ++- + 2 files changed, 8 insertions(+), 2 deletions(-) + +diff --git a/qga/commands-posix.c b/qga/commands-posix.c +index c151891..82cb120 100644 +--- a/qga/commands-posix.c ++++ b/qga/commands-posix.c +@@ -933,7 +933,12 @@ static void build_guest_fsinfo_for_real_device(char const *syspath, + if (udev == NULL || udevice == NULL) { + g_debug("failed to query udev"); + } else { +- const char *serial; ++ const char *devnode, *serial; ++ devnode = udev_device_get_devnode(udevice); ++ if (devnode != NULL) { ++ disk->dev = g_strdup(devnode); ++ disk->has_dev = true; ++ } + serial = udev_device_get_property_value(udevice, "ID_SERIAL"); + if (serial != NULL && *serial != 0) { + disk->serial = g_strdup(serial); +diff --git a/qga/qapi-schema.json b/qga/qapi-schema.json +index 426528c..ca022ad 100644 +--- a/qga/qapi-schema.json ++++ b/qga/qapi-schema.json +@@ -833,6 +833,7 @@ + # @target: target id + # @unit: unit id + # @serial: serial number (since: 3.1) ++# @dev: device node (POSIX) or device UNC (Windows) (since: 3.1) + # + # Since: 2.2 + ## +@@ -840,7 +841,7 @@ + 'data': {'pci-controller': 'GuestPCIAddress', + 'bus-type': 'GuestDiskBusType', + 'bus': 'int', 'target': 'int', 'unit': 'int', +- '*serial': 'str'} } ++ '*serial': 'str', '*dev': 'str'} } + + ## + # @GuestFilesystemInfo: +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qmp-transaction-support-for-x-block-dirty-bitmap-ena.patch b/SOURCES/kvm-qmp-transaction-support-for-x-block-dirty-bitmap-ena.patch new file mode 100644 index 0000000..2531d21 --- /dev/null +++ b/SOURCES/kvm-qmp-transaction-support-for-x-block-dirty-bitmap-ena.patch @@ -0,0 +1,163 @@ +From f694daf74067ada6720659a7133224f727f98ed3 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Wed, 18 Jul 2018 22:54:39 +0200 +Subject: [PATCH 221/268] qmp: transaction support for + x-block-dirty-bitmap-enable/disable + +RH-Author: John Snow +Message-id: <20180718225511.14878-4-jsnow@redhat.com> +Patchwork-id: 81410 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 03/35] qmp: transaction support for x-block-dirty-bitmap-enable/disable +Bugzilla: 1207657 +RH-Acked-by: Eric Blake +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Fam Zheng + +From: Vladimir Sementsov-Ogievskiy + +Signed-off-by: Vladimir Sementsov-Ogievskiy +Signed-off-by: John Snow +Reviewed-by: Jeff Cody +Message-id: 20180606182449.1607-4-jsnow@redhat.com +[Added x- prefix. --js] +Signed-off-by: John Snow +(cherry picked from commit c6490447402f574bbe45f45c318e0bdd19121baf) +Signed-off-by: John Snow + +Signed-off-by: Miroslav Rezanina +--- + blockdev.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++- + qapi/transaction.json | 4 +++ + 2 files changed, 84 insertions(+), 1 deletion(-) + +diff --git a/blockdev.c b/blockdev.c +index 9e435f4..b74b7d4 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -2053,6 +2053,7 @@ typedef struct BlockDirtyBitmapState { + BlockDriverState *bs; + HBitmap *backup; + bool prepared; ++ bool was_enabled; + } BlockDirtyBitmapState; + + static void block_dirty_bitmap_add_prepare(BlkActionState *common, +@@ -2152,6 +2153,74 @@ static void block_dirty_bitmap_clear_commit(BlkActionState *common) + hbitmap_free(state->backup); + } + ++static void block_dirty_bitmap_enable_prepare(BlkActionState *common, ++ Error **errp) ++{ ++ BlockDirtyBitmap *action; ++ BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, ++ common, common); ++ ++ if (action_check_completion_mode(common, errp) < 0) { ++ return; ++ } ++ ++ action = common->action->u.x_block_dirty_bitmap_enable.data; ++ state->bitmap = block_dirty_bitmap_lookup(action->node, ++ action->name, ++ NULL, ++ errp); ++ if (!state->bitmap) { ++ return; ++ } ++ ++ state->was_enabled = bdrv_dirty_bitmap_enabled(state->bitmap); ++ bdrv_enable_dirty_bitmap(state->bitmap); ++} ++ ++static void block_dirty_bitmap_enable_abort(BlkActionState *common) ++{ ++ BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, ++ common, common); ++ ++ if (!state->was_enabled) { ++ bdrv_disable_dirty_bitmap(state->bitmap); ++ } ++} ++ ++static void block_dirty_bitmap_disable_prepare(BlkActionState *common, ++ Error **errp) ++{ ++ BlockDirtyBitmap *action; ++ BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, ++ common, common); ++ ++ if (action_check_completion_mode(common, errp) < 0) { ++ return; ++ } ++ ++ action = common->action->u.x_block_dirty_bitmap_disable.data; ++ state->bitmap = block_dirty_bitmap_lookup(action->node, ++ action->name, ++ NULL, ++ errp); ++ if (!state->bitmap) { ++ return; ++ } ++ ++ state->was_enabled = bdrv_dirty_bitmap_enabled(state->bitmap); ++ bdrv_disable_dirty_bitmap(state->bitmap); ++} ++ ++static void block_dirty_bitmap_disable_abort(BlkActionState *common) ++{ ++ BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, ++ common, common); ++ ++ if (state->was_enabled) { ++ bdrv_enable_dirty_bitmap(state->bitmap); ++ } ++} ++ + static void abort_prepare(BlkActionState *common, Error **errp) + { + error_setg(errp, "Transaction aborted using Abort action"); +@@ -2212,7 +2281,17 @@ static const BlkActionOps actions[] = { + .prepare = block_dirty_bitmap_clear_prepare, + .commit = block_dirty_bitmap_clear_commit, + .abort = block_dirty_bitmap_clear_abort, +- } ++ }, ++ [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_ENABLE] = { ++ .instance_size = sizeof(BlockDirtyBitmapState), ++ .prepare = block_dirty_bitmap_enable_prepare, ++ .abort = block_dirty_bitmap_enable_abort, ++ }, ++ [TRANSACTION_ACTION_KIND_X_BLOCK_DIRTY_BITMAP_DISABLE] = { ++ .instance_size = sizeof(BlockDirtyBitmapState), ++ .prepare = block_dirty_bitmap_disable_prepare, ++ .abort = block_dirty_bitmap_disable_abort, ++ } + }; + + /** +diff --git a/qapi/transaction.json b/qapi/transaction.json +index bd31279..d7e4274 100644 +--- a/qapi/transaction.json ++++ b/qapi/transaction.json +@@ -46,6 +46,8 @@ + # - @abort: since 1.6 + # - @block-dirty-bitmap-add: since 2.5 + # - @block-dirty-bitmap-clear: since 2.5 ++# - @x-block-dirty-bitmap-enable: since 3.0 ++# - @x-block-dirty-bitmap-disable: since 3.0 + # - @blockdev-backup: since 2.3 + # - @blockdev-snapshot: since 2.5 + # - @blockdev-snapshot-internal-sync: since 1.7 +@@ -59,6 +61,8 @@ + 'abort': 'Abort', + 'block-dirty-bitmap-add': 'BlockDirtyBitmapAdd', + 'block-dirty-bitmap-clear': 'BlockDirtyBitmap', ++ 'x-block-dirty-bitmap-enable': 'BlockDirtyBitmap', ++ 'x-block-dirty-bitmap-disable': 'BlockDirtyBitmap', + 'blockdev-backup': 'BlockdevBackup', + 'blockdev-snapshot': 'BlockdevSnapshot', + 'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal', +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qobject-Ensure-base-is-at-offset-0.patch b/SOURCES/kvm-qobject-Ensure-base-is-at-offset-0.patch new file mode 100644 index 0000000..62785e1 --- /dev/null +++ b/SOURCES/kvm-qobject-Ensure-base-is-at-offset-0.patch @@ -0,0 +1,76 @@ +From d177d54c7ca08f3fff5824d69d4e66ca05a4eaf8 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:09 +0200 +Subject: [PATCH 011/268] qobject: Ensure base is at offset 0 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-3-armbru@redhat.com> +Patchwork-id: 80724 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 02/23] qobject: Ensure base is at offset 0 +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +From: Marc-André Lureau + +All QObject types have the base QObject as their first field. This +allows the simplification of qobject_to(). + +Signed-off-by: Marc-André Lureau +Reviewed-by: Eric Blake +Message-Id: <20180419150145.24795-2-marcandre.lureau@redhat.com> +Reviewed-by: Markus Armbruster +[Commit message paragraph on type casts dropped, to avoid giving the +impression type casting would be okay] +Signed-off-by: Markus Armbruster +(cherry picked from commit 7ee9edfdb117da47c86c9764d90f0be11a648666) + +Signed-off-by: Miroslav Rezanina +--- + include/qapi/qmp/qobject.h | 5 ++--- + qobject/qobject.c | 9 +++++++++ + 2 files changed, 11 insertions(+), 3 deletions(-) + +diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h +index e022707..5206ff9 100644 +--- a/include/qapi/qmp/qobject.h ++++ b/include/qapi/qmp/qobject.h +@@ -61,9 +61,8 @@ struct QObject { + QEMU_BUILD_BUG_MSG(QTYPE__MAX != 7, + "The QTYPE_CAST_TO_* list needs to be extended"); + +-#define qobject_to(type, obj) ({ \ +- QObject *_tmp = qobject_check_type(obj, glue(QTYPE_CAST_TO_, type)); \ +- _tmp ? container_of(_tmp, type, base) : (type *)NULL; }) ++#define qobject_to(type, obj) \ ++ ((type *)qobject_check_type(obj, glue(QTYPE_CAST_TO_, type))) + + /* Initialize an object to default values */ + static inline void qobject_init(QObject *obj, QType type) +diff --git a/qobject/qobject.c b/qobject/qobject.c +index 23600aa..87649c5 100644 +--- a/qobject/qobject.c ++++ b/qobject/qobject.c +@@ -16,6 +16,15 @@ + #include "qapi/qmp/qlist.h" + #include "qapi/qmp/qstring.h" + ++QEMU_BUILD_BUG_MSG( ++ offsetof(QNull, base) != 0 || ++ offsetof(QNum, base) != 0 || ++ offsetof(QString, base) != 0 || ++ offsetof(QDict, base) != 0 || ++ offsetof(QList, base) != 0 || ++ offsetof(QBool, base) != 0, ++ "base qobject must be at offset 0"); ++ + static void (*qdestroy[QTYPE__MAX])(QObject *) = { + [QTYPE_NONE] = NULL, /* No such object exists */ + [QTYPE_QNULL] = NULL, /* qnull_ is indestructible */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qobject-Modify-qobject_ref-to-return-obj.patch b/SOURCES/kvm-qobject-Modify-qobject_ref-to-return-obj.patch new file mode 100644 index 0000000..7efd478 --- /dev/null +++ b/SOURCES/kvm-qobject-Modify-qobject_ref-to-return-obj.patch @@ -0,0 +1,437 @@ +From 6dc6db930a9906faa26dc8c4e89d4b356dbaaa58 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:12 +0200 +Subject: [PATCH 014/268] qobject: Modify qobject_ref() to return obj +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-6-armbru@redhat.com> +Patchwork-id: 80738 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 05/23] qobject: Modify qobject_ref() to return obj +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +From: Marc-André Lureau + +For convenience and clarity, make it possible to call qobject_ref() at +the time when the reference is associated with a variable, or +argument, by making qobject_ref() return the same pointer as given. +Use that to simplify the callers. + +Signed-off-by: Marc-André Lureau +Reviewed-by: Eric Blake +Message-Id: <20180419150145.24795-5-marcandre.lureau@redhat.com> +Reviewed-by: Markus Armbruster +[Useless change to qobject_ref_impl() dropped, commit message improved +slightly] +Signed-off-by: Markus Armbruster +(cherry picked from commit f5a74a5a50387c6f980b2e2f94f062487a1826da) + +Signed-off-by: Miroslav Rezanina +--- + block.c | 8 ++++---- + block/blkdebug.c | 7 +++---- + block/blkverify.c | 8 ++++---- + block/null.c | 3 +-- + block/nvme.c | 3 +-- + block/quorum.c | 4 ++-- + include/qapi/qmp/qnull.h | 3 +-- + include/qapi/qmp/qobject.h | 9 ++++++++- + monitor.c | 20 +++++++------------- + qapi/qobject-input-visitor.c | 6 ++---- + qapi/qobject-output-visitor.c | 7 +++---- + qobject/qdict.c | 33 +++++++++++---------------------- + 12 files changed, 47 insertions(+), 64 deletions(-) + +diff --git a/block.c b/block.c +index 55a7984..676e57f 100644 +--- a/block.c ++++ b/block.c +@@ -5134,8 +5134,8 @@ static bool append_open_options(QDict *d, BlockDriverState *bs) + continue; + } + +- qobject_ref(qdict_entry_value(entry)); +- qdict_put_obj(d, qdict_entry_key(entry), qdict_entry_value(entry)); ++ qdict_put_obj(d, qdict_entry_key(entry), ++ qobject_ref(qdict_entry_value(entry))); + found_any = true; + } + +@@ -5207,8 +5207,8 @@ void bdrv_refresh_filename(BlockDriverState *bs) + * suffices without querying the (exact_)filename of this BDS. */ + if (bs->file->bs->full_open_options) { + qdict_put_str(opts, "driver", drv->format_name); +- qobject_ref(bs->file->bs->full_open_options); +- qdict_put(opts, "file", bs->file->bs->full_open_options); ++ qdict_put(opts, "file", ++ qobject_ref(bs->file->bs->full_open_options)); + + bs->full_open_options = opts; + } else { +diff --git a/block/blkdebug.c b/block/blkdebug.c +index 689703d..053372c 100644 +--- a/block/blkdebug.c ++++ b/block/blkdebug.c +@@ -845,13 +845,12 @@ static void blkdebug_refresh_filename(BlockDriverState *bs, QDict *options) + opts = qdict_new(); + qdict_put_str(opts, "driver", "blkdebug"); + +- qobject_ref(bs->file->bs->full_open_options); +- qdict_put(opts, "image", bs->file->bs->full_open_options); ++ qdict_put(opts, "image", qobject_ref(bs->file->bs->full_open_options)); + + for (e = qdict_first(options); e; e = qdict_next(options, e)) { + if (strcmp(qdict_entry_key(e), "x-image")) { +- qobject_ref(qdict_entry_value(e)); +- qdict_put_obj(opts, qdict_entry_key(e), qdict_entry_value(e)); ++ qdict_put_obj(opts, qdict_entry_key(e), ++ qobject_ref(qdict_entry_value(e))); + } + } + +diff --git a/block/blkverify.c b/block/blkverify.c +index 3cffcb1..754cc9e 100644 +--- a/block/blkverify.c ++++ b/block/blkverify.c +@@ -291,10 +291,10 @@ static void blkverify_refresh_filename(BlockDriverState *bs, QDict *options) + QDict *opts = qdict_new(); + qdict_put_str(opts, "driver", "blkverify"); + +- qobject_ref(bs->file->bs->full_open_options); +- qdict_put(opts, "raw", bs->file->bs->full_open_options); +- qobject_ref(s->test_file->bs->full_open_options); +- qdict_put(opts, "test", s->test_file->bs->full_open_options); ++ qdict_put(opts, "raw", ++ qobject_ref(bs->file->bs->full_open_options)); ++ qdict_put(opts, "test", ++ qobject_ref(s->test_file->bs->full_open_options)); + + bs->full_open_options = opts; + } +diff --git a/block/null.c b/block/null.c +index 700a2d0..3944550 100644 +--- a/block/null.c ++++ b/block/null.c +@@ -244,7 +244,6 @@ static int coroutine_fn null_co_block_status(BlockDriverState *bs, + + static void null_refresh_filename(BlockDriverState *bs, QDict *opts) + { +- qobject_ref(opts); + qdict_del(opts, "filename"); + + if (!qdict_size(opts)) { +@@ -253,7 +252,7 @@ static void null_refresh_filename(BlockDriverState *bs, QDict *opts) + } + + qdict_put_str(opts, "driver", bs->drv->format_name); +- bs->full_open_options = opts; ++ bs->full_open_options = qobject_ref(opts); + } + + static BlockDriver bdrv_null_co = { +diff --git a/block/nvme.c b/block/nvme.c +index e192da9..6f71122 100644 +--- a/block/nvme.c ++++ b/block/nvme.c +@@ -1073,7 +1073,6 @@ static int nvme_reopen_prepare(BDRVReopenState *reopen_state, + + static void nvme_refresh_filename(BlockDriverState *bs, QDict *opts) + { +- qobject_ref(opts); + qdict_del(opts, "filename"); + + if (!qdict_size(opts)) { +@@ -1082,7 +1081,7 @@ static void nvme_refresh_filename(BlockDriverState *bs, QDict *opts) + } + + qdict_put_str(opts, "driver", bs->drv->format_name); +- bs->full_open_options = opts; ++ bs->full_open_options = qobject_ref(opts); + } + + static void nvme_refresh_limits(BlockDriverState *bs, Error **errp) +diff --git a/block/quorum.c b/block/quorum.c +index 862cea3..a5051da 100644 +--- a/block/quorum.c ++++ b/block/quorum.c +@@ -1082,8 +1082,8 @@ static void quorum_refresh_filename(BlockDriverState *bs, QDict *options) + + children = qlist_new(); + for (i = 0; i < s->num_children; i++) { +- qobject_ref(s->children[i]->bs->full_open_options); +- qlist_append(children, s->children[i]->bs->full_open_options); ++ qlist_append(children, ++ qobject_ref(s->children[i]->bs->full_open_options)); + } + + opts = qdict_new(); +diff --git a/include/qapi/qmp/qnull.h b/include/qapi/qmp/qnull.h +index 75b29c6..c142688 100644 +--- a/include/qapi/qmp/qnull.h ++++ b/include/qapi/qmp/qnull.h +@@ -23,8 +23,7 @@ extern QNull qnull_; + + static inline QNull *qnull(void) + { +- qobject_ref(&qnull_); +- return &qnull_; ++ return qobject_ref(&qnull_); + } + + bool qnull_is_equal(const QObject *x, const QObject *y); +diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h +index e20006f..fcfd549 100644 +--- a/include/qapi/qmp/qobject.h ++++ b/include/qapi/qmp/qobject.h +@@ -103,8 +103,15 @@ static inline void qobject_unref_impl(QObject *obj) + + /** + * qobject_ref(): Increment QObject's reference count ++ * ++ * Returns: the same @obj. The type of @obj will be propagated to the ++ * return type. + */ +-#define qobject_ref(obj) qobject_ref_impl(QOBJECT(obj)) ++#define qobject_ref(obj) ({ \ ++ typeof(obj) _o = (obj); \ ++ qobject_ref_impl(QOBJECT(_o)); \ ++ _o; \ ++}) + + /** + * qobject_unref(): Decrement QObject's reference count, deallocate +diff --git a/monitor.c b/monitor.c +index 4f43eee..46814af 100644 +--- a/monitor.c ++++ b/monitor.c +@@ -494,9 +494,8 @@ static void monitor_json_emitter(Monitor *mon, QObject *data) + * caller won't free the data (which will be finally freed in + * responder thread). + */ +- qobject_ref(data); + qemu_mutex_lock(&mon->qmp.qmp_queue_lock); +- g_queue_push_tail(mon->qmp.qmp_responses, data); ++ g_queue_push_tail(mon->qmp.qmp_responses, qobject_ref(data)); + qemu_mutex_unlock(&mon->qmp.qmp_queue_lock); + qemu_bh_schedule(mon_global.qmp_respond_bh); + } else { +@@ -614,8 +613,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, Error **errp) + * replacing a prior stored event if any. + */ + qobject_unref(evstate->qdict); +- evstate->qdict = qdict; +- qobject_ref(evstate->qdict); ++ evstate->qdict = qobject_ref(qdict); + } else { + /* + * Last send was (at least) evconf->rate ns ago. +@@ -629,8 +627,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, Error **errp) + + evstate = g_new(MonitorQAPIEventState, 1); + evstate->event = event; +- evstate->data = data; +- qobject_ref(evstate->data); ++ evstate->data = qobject_ref(data); + evstate->qdict = NULL; + evstate->timer = timer_new_ns(event_clock_type, + monitor_qapi_event_handler, +@@ -4048,9 +4045,7 @@ static void monitor_qmp_respond(Monitor *mon, QObject *rsp, + + if (rsp) { + if (id) { +- /* This is for the qdict below. */ +- qobject_ref(id); +- qdict_put_obj(qobject_to(QDict, rsp), "id", id); ++ qdict_put_obj(qobject_to(QDict, rsp), "id", qobject_ref(id)); + } + + monitor_json_emitter(mon, rsp); +@@ -4190,15 +4185,14 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens) + goto err; + } + +- qobject_ref(id); +- qdict_del(qdict, "id"); +- + req_obj = g_new0(QMPRequest, 1); + req_obj->mon = mon; +- req_obj->id = id; ++ req_obj->id = qobject_ref(id); + req_obj->req = req; + req_obj->need_resume = false; + ++ qdict_del(qdict, "id"); ++ + if (qmp_is_oob(qdict)) { + /* Out-Of-Band (OOB) requests are executed directly in parser. */ + trace_monitor_qmp_cmd_out_of_band(qobject_get_try_str(req_obj->id) +diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c +index 7a290c4..da57f4c 100644 +--- a/qapi/qobject-input-visitor.c ++++ b/qapi/qobject-input-visitor.c +@@ -588,8 +588,7 @@ static void qobject_input_type_any(Visitor *v, const char *name, QObject **obj, + return; + } + +- qobject_ref(qobj); +- *obj = qobj; ++ *obj = qobject_ref(qobj); + } + + static void qobject_input_type_null(Visitor *v, const char *name, +@@ -677,8 +676,7 @@ static QObjectInputVisitor *qobject_input_visitor_base_new(QObject *obj) + v->visitor.optional = qobject_input_optional; + v->visitor.free = qobject_input_free; + +- v->root = obj; +- qobject_ref(obj); ++ v->root = qobject_ref(obj); + + return v; + } +diff --git a/qapi/qobject-output-visitor.c b/qapi/qobject-output-visitor.c +index 3a933b4..89ffd8a 100644 +--- a/qapi/qobject-output-visitor.c ++++ b/qapi/qobject-output-visitor.c +@@ -188,8 +188,8 @@ static void qobject_output_type_any(Visitor *v, const char *name, + QObject **obj, Error **errp) + { + QObjectOutputVisitor *qov = to_qov(v); +- qobject_ref(*obj); +- qobject_output_add_obj(qov, name, *obj); ++ ++ qobject_output_add_obj(qov, name, qobject_ref(*obj)); + } + + static void qobject_output_type_null(Visitor *v, const char *name, +@@ -210,8 +210,7 @@ static void qobject_output_complete(Visitor *v, void *opaque) + assert(qov->root && QSLIST_EMPTY(&qov->stack)); + assert(opaque == qov->result); + +- qobject_ref(qov->root); +- *qov->result = qov->root; ++ *qov->result = qobject_ref(qov->root); + qov->result = NULL; + } + +diff --git a/qobject/qdict.c b/qobject/qdict.c +index 2e9bd53..22800ee 100644 +--- a/qobject/qdict.c ++++ b/qobject/qdict.c +@@ -373,8 +373,7 @@ QDict *qdict_clone_shallow(const QDict *src) + + for (i = 0; i < QDICT_BUCKET_MAX; i++) { + QLIST_FOREACH(entry, &src->table[i], next) { +- qobject_ref(entry->value); +- qdict_put_obj(dest, entry->key, entry->value); ++ qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); + } + } + +@@ -480,8 +479,7 @@ void qdict_copy_default(QDict *dst, QDict *src, const char *key) + + val = qdict_get(src, key); + if (val) { +- qobject_ref(val); +- qdict_put_obj(dst, key, val); ++ qdict_put_obj(dst, key, qobject_ref(val)); + } + } + +@@ -526,8 +524,7 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) + qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + } else { + /* All other types are moved to the target unchanged. */ +- qobject_ref(value); +- qdict_put_obj(target, new_key, value); ++ qdict_put_obj(target, new_key, qobject_ref(value)); + } + + g_free(new_key); +@@ -566,8 +563,7 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + delete = true; + } else if (prefix) { + /* All other objects are moved to the target unchanged. */ +- qobject_ref(value); +- qdict_put_obj(target, new_key, value); ++ qdict_put_obj(target, new_key, qobject_ref(value)); + delete = true; + } + +@@ -610,8 +606,7 @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) + while (entry != NULL) { + next = qdict_next(src, entry); + if (strstart(entry->key, start, &p)) { +- qobject_ref(entry->value); +- qdict_put_obj(*dst, p, entry->value); ++ qdict_put_obj(*dst, p, qobject_ref(entry->value)); + qdict_del(src, entry->key); + } + entry = next; +@@ -894,16 +889,14 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); + } + +- qobject_ref(ent->value); +- qdict_put_obj(child_dict, suffix, ent->value); ++ qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); + } else { + if (child) { + error_setg(errp, "Key %s prefix is already set as a dict", + prefix); + goto error; + } +- qobject_ref(ent->value); +- qdict_put_obj(two_level, prefix, ent->value); ++ qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); + } + + g_free(prefix); +@@ -924,8 +917,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + + qdict_put_obj(multi_level, ent->key, child); + } else { +- qobject_ref(ent->value); +- qdict_put_obj(multi_level, ent->key, ent->value); ++ qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); + } + } + qobject_unref(two_level); +@@ -951,8 +943,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + goto error; + } + +- qobject_ref(child); +- qlist_append_obj(qobject_to(QList, dst), child); ++ qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); + } + qobject_unref(multi_level); + multi_level = NULL; +@@ -1055,8 +1046,7 @@ void qdict_join(QDict *dest, QDict *src, bool overwrite) + next = qdict_next(src, entry); + + if (overwrite || !qdict_haskey(dest, entry->key)) { +- qobject_ref(entry->value); +- qdict_put_obj(dest, entry->key, entry->value); ++ qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); + qdict_del(src, entry->key); + } + +@@ -1088,8 +1078,7 @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) + } + + qobj = qdict_get(qdict, renames->from); +- qobject_ref(qobj); +- qdict_put_obj(qdict, renames->to, qobj); ++ qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); + qdict_del(qdict, renames->from); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qobject-Move-block-specific-qdict-code-to-block-qdic.patch b/SOURCES/kvm-qobject-Move-block-specific-qdict-code-to-block-qdic.patch new file mode 100644 index 0000000..4c6e246 --- /dev/null +++ b/SOURCES/kvm-qobject-Move-block-specific-qdict-code-to-block-qdic.patch @@ -0,0 +1,2728 @@ +From c5a822e1af01299f91faac2193b918392146b189 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:16 +0200 +Subject: [PATCH 018/268] qobject: Move block-specific qdict code to + block-qdict.c + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-10-armbru@redhat.com> +Patchwork-id: 80722 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 09/23] qobject: Move block-specific qdict code to block-qdict.c +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Pure code motion, except for two brace placements and a comment +tweaked to appease checkpatch. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit 0bcc8e5bd8d6fd6e5cb6462054f7cfa45b282f9a) +Signed-off-by: Miroslav Rezanina +--- + MAINTAINERS | 2 + + qobject/Makefile.objs | 1 + + qobject/block-qdict.c | 640 ++++++++++++++++++++++++++++++++++++++++++++ + qobject/qdict.c | 629 -------------------------------------------- + tests/Makefile.include | 4 + + tests/check-block-qdict.c | 655 ++++++++++++++++++++++++++++++++++++++++++++++ + tests/check-qdict.c | 642 --------------------------------------------- + 7 files changed, 1302 insertions(+), 1271 deletions(-) + create mode 100644 qobject/block-qdict.c + create mode 100644 tests/check-block-qdict.c + +diff --git a/MAINTAINERS b/MAINTAINERS +index 24b7016..ad5edc8 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1339,6 +1339,8 @@ F: qemu-img* + F: qemu-io* + F: tests/qemu-iotests/ + F: util/qemu-progress.c ++F: qobject/block-qdict.c ++F: test/check-block-qdict.c + T: git git://repo.or.cz/qemu/kevin.git block + + Block I/O path +diff --git a/qobject/Makefile.objs b/qobject/Makefile.objs +index 002d258..7b12c9c 100644 +--- a/qobject/Makefile.objs ++++ b/qobject/Makefile.objs +@@ -1,2 +1,3 @@ + util-obj-y = qnull.o qnum.o qstring.o qdict.o qlist.o qbool.o qlit.o + util-obj-y += qjson.o qobject.o json-lexer.o json-streamer.o json-parser.o ++util-obj-y += block-qdict.o +diff --git a/qobject/block-qdict.c b/qobject/block-qdict.c +new file mode 100644 +index 0000000..fb92010 +--- /dev/null ++++ b/qobject/block-qdict.c +@@ -0,0 +1,640 @@ ++/* ++ * Special QDict functions used by the block layer ++ * ++ * Copyright (c) 2013-2018 Red Hat, Inc. ++ * ++ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. ++ * See the COPYING.LIB file in the top-level directory. ++ */ ++ ++#include "qemu/osdep.h" ++#include "block/qdict.h" ++#include "qapi/qmp/qlist.h" ++#include "qemu/cutils.h" ++#include "qapi/error.h" ++ ++/** ++ * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the ++ * value of 'key' in 'src' is copied there (and the refcount increased ++ * accordingly). ++ */ ++void qdict_copy_default(QDict *dst, QDict *src, const char *key) ++{ ++ QObject *val; ++ ++ if (qdict_haskey(dst, key)) { ++ return; ++ } ++ ++ val = qdict_get(src, key); ++ if (val) { ++ qdict_put_obj(dst, key, qobject_ref(val)); ++ } ++} ++ ++/** ++ * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a ++ * new QString initialised by 'val' is put there. ++ */ ++void qdict_set_default_str(QDict *dst, const char *key, const char *val) ++{ ++ if (qdict_haskey(dst, key)) { ++ return; ++ } ++ ++ qdict_put_str(dst, key, val); ++} ++ ++static void qdict_flatten_qdict(QDict *qdict, QDict *target, ++ const char *prefix); ++ ++static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) ++{ ++ QObject *value; ++ const QListEntry *entry; ++ char *new_key; ++ int i; ++ ++ /* This function is never called with prefix == NULL, i.e., it is always ++ * called from within qdict_flatten_q(list|dict)(). Therefore, it does not ++ * need to remove list entries during the iteration (the whole list will be ++ * deleted eventually anyway from qdict_flatten_qdict()). */ ++ assert(prefix); ++ ++ entry = qlist_first(qlist); ++ ++ for (i = 0; entry; entry = qlist_next(entry), i++) { ++ value = qlist_entry_obj(entry); ++ new_key = g_strdup_printf("%s.%i", prefix, i); ++ ++ if (qobject_type(value) == QTYPE_QDICT) { ++ qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); ++ } else if (qobject_type(value) == QTYPE_QLIST) { ++ qdict_flatten_qlist(qobject_to(QList, value), target, new_key); ++ } else { ++ /* All other types are moved to the target unchanged. */ ++ qdict_put_obj(target, new_key, qobject_ref(value)); ++ } ++ ++ g_free(new_key); ++ } ++} ++ ++static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) ++{ ++ QObject *value; ++ const QDictEntry *entry, *next; ++ char *new_key; ++ bool delete; ++ ++ entry = qdict_first(qdict); ++ ++ while (entry != NULL) { ++ ++ next = qdict_next(qdict, entry); ++ value = qdict_entry_value(entry); ++ new_key = NULL; ++ delete = false; ++ ++ if (prefix) { ++ new_key = g_strdup_printf("%s.%s", prefix, entry->key); ++ } ++ ++ if (qobject_type(value) == QTYPE_QDICT) { ++ /* Entries of QDicts are processed recursively, the QDict object ++ * itself disappears. */ ++ qdict_flatten_qdict(qobject_to(QDict, value), target, ++ new_key ? new_key : entry->key); ++ delete = true; ++ } else if (qobject_type(value) == QTYPE_QLIST) { ++ qdict_flatten_qlist(qobject_to(QList, value), target, ++ new_key ? new_key : entry->key); ++ delete = true; ++ } else if (prefix) { ++ /* All other objects are moved to the target unchanged. */ ++ qdict_put_obj(target, new_key, qobject_ref(value)); ++ delete = true; ++ } ++ ++ g_free(new_key); ++ ++ if (delete) { ++ qdict_del(qdict, entry->key); ++ ++ /* Restart loop after modifying the iterated QDict */ ++ entry = qdict_first(qdict); ++ continue; ++ } ++ ++ entry = next; ++ } ++} ++ ++/** ++ * qdict_flatten(): For each nested QDict with key x, all fields with key y ++ * are moved to this QDict and their key is renamed to "x.y". For each nested ++ * QList with key x, the field at index y is moved to this QDict with the key ++ * "x.y" (i.e., the reverse of what qdict_array_split() does). ++ * This operation is applied recursively for nested QDicts and QLists. ++ */ ++void qdict_flatten(QDict *qdict) ++{ ++ qdict_flatten_qdict(qdict, qdict, NULL); ++} ++ ++/* extract all the src QDict entries starting by start into dst */ ++void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) ++ ++{ ++ const QDictEntry *entry, *next; ++ const char *p; ++ ++ *dst = qdict_new(); ++ entry = qdict_first(src); ++ ++ while (entry != NULL) { ++ next = qdict_next(src, entry); ++ if (strstart(entry->key, start, &p)) { ++ qdict_put_obj(*dst, p, qobject_ref(entry->value)); ++ qdict_del(src, entry->key); ++ } ++ entry = next; ++ } ++} ++ ++static int qdict_count_prefixed_entries(const QDict *src, const char *start) ++{ ++ const QDictEntry *entry; ++ int count = 0; ++ ++ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { ++ if (strstart(entry->key, start, NULL)) { ++ if (count == INT_MAX) { ++ return -ERANGE; ++ } ++ count++; ++ } ++ } ++ ++ return count; ++} ++ ++/** ++ * qdict_array_split(): This function moves array-like elements of a QDict into ++ * a new QList. Every entry in the original QDict with a key "%u" or one ++ * prefixed "%u.", where %u designates an unsigned integer starting at 0 and ++ * incrementally counting up, will be moved to a new QDict at index %u in the ++ * output QList with the key prefix removed, if that prefix is "%u.". If the ++ * whole key is just "%u", the whole QObject will be moved unchanged without ++ * creating a new QDict. The function terminates when there is no entry in the ++ * QDict with a prefix directly (incrementally) following the last one; it also ++ * returns if there are both entries with "%u" and "%u." for the same index %u. ++ * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66} ++ * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66}) ++ * => [{"a": 42, "b": 23}, {"x": 0}, 66] ++ * and {"4.y": 1, "o.o": 7} (remainder of the old QDict) ++ */ ++void qdict_array_split(QDict *src, QList **dst) ++{ ++ unsigned i; ++ ++ *dst = qlist_new(); ++ ++ for (i = 0; i < UINT_MAX; i++) { ++ QObject *subqobj; ++ bool is_subqdict; ++ QDict *subqdict; ++ char indexstr[32], prefix[32]; ++ size_t snprintf_ret; ++ ++ snprintf_ret = snprintf(indexstr, 32, "%u", i); ++ assert(snprintf_ret < 32); ++ ++ subqobj = qdict_get(src, indexstr); ++ ++ snprintf_ret = snprintf(prefix, 32, "%u.", i); ++ assert(snprintf_ret < 32); ++ ++ /* Overflow is the same as positive non-zero results */ ++ is_subqdict = qdict_count_prefixed_entries(src, prefix); ++ ++ /* ++ * There may be either a single subordinate object (named ++ * "%u") or multiple objects (each with a key prefixed "%u."), ++ * but not both. ++ */ ++ if (!subqobj == !is_subqdict) { ++ break; ++ } ++ ++ if (is_subqdict) { ++ qdict_extract_subqdict(src, &subqdict, prefix); ++ assert(qdict_size(subqdict) > 0); ++ } else { ++ qobject_ref(subqobj); ++ qdict_del(src, indexstr); ++ } ++ ++ qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict)); ++ } ++} ++ ++/** ++ * qdict_split_flat_key: ++ * @key: the key string to split ++ * @prefix: non-NULL pointer to hold extracted prefix ++ * @suffix: non-NULL pointer to remaining suffix ++ * ++ * Given a flattened key such as 'foo.0.bar', split it into two parts ++ * at the first '.' separator. Allows double dot ('..') to escape the ++ * normal separator. ++ * ++ * e.g. ++ * 'foo.0.bar' -> prefix='foo' and suffix='0.bar' ++ * 'foo..0.bar' -> prefix='foo.0' and suffix='bar' ++ * ++ * The '..' sequence will be unescaped in the returned 'prefix' ++ * string. The 'suffix' string will be left in escaped format, so it ++ * can be fed back into the qdict_split_flat_key() key as the input ++ * later. ++ * ++ * The caller is responsible for freeing the string returned in @prefix ++ * using g_free(). ++ */ ++static void qdict_split_flat_key(const char *key, char **prefix, ++ const char **suffix) ++{ ++ const char *separator; ++ size_t i, j; ++ ++ /* Find first '.' separator, but if there is a pair '..' ++ * that acts as an escape, so skip over '..' */ ++ separator = NULL; ++ do { ++ if (separator) { ++ separator += 2; ++ } else { ++ separator = key; ++ } ++ separator = strchr(separator, '.'); ++ } while (separator && separator[1] == '.'); ++ ++ if (separator) { ++ *prefix = g_strndup(key, separator - key); ++ *suffix = separator + 1; ++ } else { ++ *prefix = g_strdup(key); ++ *suffix = NULL; ++ } ++ ++ /* Unescape the '..' sequence into '.' */ ++ for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) { ++ if ((*prefix)[i] == '.') { ++ assert((*prefix)[i + 1] == '.'); ++ i++; ++ } ++ (*prefix)[j] = (*prefix)[i]; ++ } ++ (*prefix)[j] = '\0'; ++} ++ ++/** ++ * qdict_is_list: ++ * @maybe_list: dict to check if keys represent list elements. ++ * ++ * Determine whether all keys in @maybe_list are valid list elements. ++ * If @maybe_list is non-zero in length and all the keys look like ++ * valid list indexes, this will return 1. If @maybe_list is zero ++ * length or all keys are non-numeric then it will return 0 to indicate ++ * it is a normal qdict. If there is a mix of numeric and non-numeric ++ * keys, or the list indexes are non-contiguous, an error is reported. ++ * ++ * Returns: 1 if a valid list, 0 if a dict, -1 on error ++ */ ++static int qdict_is_list(QDict *maybe_list, Error **errp) ++{ ++ const QDictEntry *ent; ++ ssize_t len = 0; ++ ssize_t max = -1; ++ int is_list = -1; ++ int64_t val; ++ ++ for (ent = qdict_first(maybe_list); ent != NULL; ++ ent = qdict_next(maybe_list, ent)) { ++ ++ if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { ++ if (is_list == -1) { ++ is_list = 1; ++ } else if (!is_list) { ++ error_setg(errp, ++ "Cannot mix list and non-list keys"); ++ return -1; ++ } ++ len++; ++ if (val > max) { ++ max = val; ++ } ++ } else { ++ if (is_list == -1) { ++ is_list = 0; ++ } else if (is_list) { ++ error_setg(errp, ++ "Cannot mix list and non-list keys"); ++ return -1; ++ } ++ } ++ } ++ ++ if (is_list == -1) { ++ assert(!qdict_size(maybe_list)); ++ is_list = 0; ++ } ++ ++ /* NB this isn't a perfect check - e.g. it won't catch ++ * a list containing '1', '+1', '01', '3', but that ++ * does not matter - we've still proved that the ++ * input is a list. It is up the caller to do a ++ * stricter check if desired */ ++ if (len != (max + 1)) { ++ error_setg(errp, "List indices are not contiguous, " ++ "saw %zd elements but %zd largest index", ++ len, max); ++ return -1; ++ } ++ ++ return is_list; ++} ++ ++/** ++ * qdict_crumple: ++ * @src: the original flat dictionary (only scalar values) to crumple ++ * ++ * Takes a flat dictionary whose keys use '.' separator to indicate ++ * nesting, and values are scalars, and crumples it into a nested ++ * structure. ++ * ++ * To include a literal '.' in a key name, it must be escaped as '..' ++ * ++ * For example, an input of: ++ * ++ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1', ++ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' } ++ * ++ * will result in an output of: ++ * ++ * { ++ * 'foo': [ ++ * { 'bar': 'one', 'wizz': '1' }, ++ * { 'bar': 'two', 'wizz': '2' } ++ * ], ++ * } ++ * ++ * The following scenarios in the input dict will result in an ++ * error being returned: ++ * ++ * - Any values in @src are non-scalar types ++ * - If keys in @src imply that a particular level is both a ++ * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar". ++ * - If keys in @src imply that a particular level is a list, ++ * but the indices are non-contiguous. e.g. "foo.0.bar" and ++ * "foo.2.bar" without any "foo.1.bar" present. ++ * - If keys in @src represent list indexes, but are not in ++ * the "%zu" format. e.g. "foo.+0.bar" ++ * ++ * Returns: either a QDict or QList for the nested data structure, or NULL ++ * on error ++ */ ++QObject *qdict_crumple(const QDict *src, Error **errp) ++{ ++ const QDictEntry *ent; ++ QDict *two_level, *multi_level = NULL; ++ QObject *dst = NULL, *child; ++ size_t i; ++ char *prefix = NULL; ++ const char *suffix = NULL; ++ int is_list; ++ ++ two_level = qdict_new(); ++ ++ /* Step 1: split our totally flat dict into a two level dict */ ++ for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { ++ if (qobject_type(ent->value) == QTYPE_QDICT || ++ qobject_type(ent->value) == QTYPE_QLIST) { ++ error_setg(errp, "Value %s is not a scalar", ++ ent->key); ++ goto error; ++ } ++ ++ qdict_split_flat_key(ent->key, &prefix, &suffix); ++ ++ child = qdict_get(two_level, prefix); ++ if (suffix) { ++ QDict *child_dict = qobject_to(QDict, child); ++ if (!child_dict) { ++ if (child) { ++ error_setg(errp, "Key %s prefix is already set as a scalar", ++ prefix); ++ goto error; ++ } ++ ++ child_dict = qdict_new(); ++ qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); ++ } ++ ++ qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); ++ } else { ++ if (child) { ++ error_setg(errp, "Key %s prefix is already set as a dict", ++ prefix); ++ goto error; ++ } ++ qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); ++ } ++ ++ g_free(prefix); ++ prefix = NULL; ++ } ++ ++ /* Step 2: optionally process the two level dict recursively ++ * into a multi-level dict */ ++ multi_level = qdict_new(); ++ for (ent = qdict_first(two_level); ent != NULL; ++ ent = qdict_next(two_level, ent)) { ++ QDict *dict = qobject_to(QDict, ent->value); ++ if (dict) { ++ child = qdict_crumple(dict, errp); ++ if (!child) { ++ goto error; ++ } ++ ++ qdict_put_obj(multi_level, ent->key, child); ++ } else { ++ qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); ++ } ++ } ++ qobject_unref(two_level); ++ two_level = NULL; ++ ++ /* Step 3: detect if we need to turn our dict into list */ ++ is_list = qdict_is_list(multi_level, errp); ++ if (is_list < 0) { ++ goto error; ++ } ++ ++ if (is_list) { ++ dst = QOBJECT(qlist_new()); ++ ++ for (i = 0; i < qdict_size(multi_level); i++) { ++ char *key = g_strdup_printf("%zu", i); ++ ++ child = qdict_get(multi_level, key); ++ g_free(key); ++ ++ if (!child) { ++ error_setg(errp, "Missing list index %zu", i); ++ goto error; ++ } ++ ++ qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); ++ } ++ qobject_unref(multi_level); ++ multi_level = NULL; ++ } else { ++ dst = QOBJECT(multi_level); ++ } ++ ++ return dst; ++ ++ error: ++ g_free(prefix); ++ qobject_unref(multi_level); ++ qobject_unref(two_level); ++ qobject_unref(dst); ++ return NULL; ++} ++ ++/** ++ * qdict_array_entries(): Returns the number of direct array entries if the ++ * sub-QDict of src specified by the prefix in subqdict (or src itself for ++ * prefix == "") is valid as an array, i.e. the length of the created list if ++ * the sub-QDict would become empty after calling qdict_array_split() on it. If ++ * the array is not valid, -EINVAL is returned. ++ */ ++int qdict_array_entries(QDict *src, const char *subqdict) ++{ ++ const QDictEntry *entry; ++ unsigned i; ++ unsigned entries = 0; ++ size_t subqdict_len = strlen(subqdict); ++ ++ assert(!subqdict_len || subqdict[subqdict_len - 1] == '.'); ++ ++ /* qdict_array_split() loops until UINT_MAX, but as we want to return ++ * negative errors, we only have a signed return value here. Any additional ++ * entries will lead to -EINVAL. */ ++ for (i = 0; i < INT_MAX; i++) { ++ QObject *subqobj; ++ int subqdict_entries; ++ char *prefix = g_strdup_printf("%s%u.", subqdict, i); ++ ++ subqdict_entries = qdict_count_prefixed_entries(src, prefix); ++ ++ /* Remove ending "." */ ++ prefix[strlen(prefix) - 1] = 0; ++ subqobj = qdict_get(src, prefix); ++ ++ g_free(prefix); ++ ++ if (subqdict_entries < 0) { ++ return subqdict_entries; ++ } ++ ++ /* There may be either a single subordinate object (named "%u") or ++ * multiple objects (each with a key prefixed "%u."), but not both. */ ++ if (subqobj && subqdict_entries) { ++ return -EINVAL; ++ } else if (!subqobj && !subqdict_entries) { ++ break; ++ } ++ ++ entries += subqdict_entries ? subqdict_entries : 1; ++ } ++ ++ /* Consider everything handled that isn't part of the given sub-QDict */ ++ for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { ++ if (!strstart(qdict_entry_key(entry), subqdict, NULL)) { ++ entries++; ++ } ++ } ++ ++ /* Anything left in the sub-QDict that wasn't handled? */ ++ if (qdict_size(src) != entries) { ++ return -EINVAL; ++ } ++ ++ return i; ++} ++ ++/** ++ * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all ++ * elements from src to dest. ++ * ++ * If an element from src has a key already present in dest, it will not be ++ * moved unless overwrite is true. ++ * ++ * If overwrite is true, the conflicting values in dest will be discarded and ++ * replaced by the corresponding values from src. ++ * ++ * Therefore, with overwrite being true, the src QDict will always be empty when ++ * this function returns. If overwrite is false, the src QDict will be empty ++ * iff there were no conflicts. ++ */ ++void qdict_join(QDict *dest, QDict *src, bool overwrite) ++{ ++ const QDictEntry *entry, *next; ++ ++ entry = qdict_first(src); ++ while (entry) { ++ next = qdict_next(src, entry); ++ ++ if (overwrite || !qdict_haskey(dest, entry->key)) { ++ qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); ++ qdict_del(src, entry->key); ++ } ++ ++ entry = next; ++ } ++} ++ ++/** ++ * qdict_rename_keys(): Rename keys in qdict according to the replacements ++ * specified in the array renames. The array must be terminated by an entry ++ * with from = NULL. ++ * ++ * The renames are performed individually in the order of the array, so entries ++ * may be renamed multiple times and may or may not conflict depending on the ++ * order of the renames array. ++ * ++ * Returns true for success, false in error cases. ++ */ ++bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) ++{ ++ QObject *qobj; ++ ++ while (renames->from) { ++ if (qdict_haskey(qdict, renames->from)) { ++ if (qdict_haskey(qdict, renames->to)) { ++ error_setg(errp, "'%s' and its alias '%s' can't be used at the " ++ "same time", renames->to, renames->from); ++ return false; ++ } ++ ++ qobj = qdict_get(qdict, renames->from); ++ qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); ++ qdict_del(qdict, renames->from); ++ } ++ ++ renames++; ++ } ++ return true; ++} +diff --git a/qobject/qdict.c b/qobject/qdict.c +index 0554c64..3d8c2f7 100644 +--- a/qobject/qdict.c ++++ b/qobject/qdict.c +@@ -11,17 +11,11 @@ + */ + + #include "qemu/osdep.h" +-#include "block/qdict.h" + #include "qapi/qmp/qnum.h" + #include "qapi/qmp/qdict.h" + #include "qapi/qmp/qbool.h" +-#include "qapi/qmp/qlist.h" + #include "qapi/qmp/qnull.h" + #include "qapi/qmp/qstring.h" +-#include "qapi/error.h" +-#include "qemu/queue.h" +-#include "qemu-common.h" +-#include "qemu/cutils.h" + + /** + * qdict_new(): Create a new QDict +@@ -464,626 +458,3 @@ void qdict_destroy_obj(QObject *obj) + + g_free(qdict); + } +- +-/** +- * qdict_copy_default(): If no entry mapped by 'key' exists in 'dst' yet, the +- * value of 'key' in 'src' is copied there (and the refcount increased +- * accordingly). +- */ +-void qdict_copy_default(QDict *dst, QDict *src, const char *key) +-{ +- QObject *val; +- +- if (qdict_haskey(dst, key)) { +- return; +- } +- +- val = qdict_get(src, key); +- if (val) { +- qdict_put_obj(dst, key, qobject_ref(val)); +- } +-} +- +-/** +- * qdict_set_default_str(): If no entry mapped by 'key' exists in 'dst' yet, a +- * new QString initialised by 'val' is put there. +- */ +-void qdict_set_default_str(QDict *dst, const char *key, const char *val) +-{ +- if (qdict_haskey(dst, key)) { +- return; +- } +- +- qdict_put_str(dst, key, val); +-} +- +-static void qdict_flatten_qdict(QDict *qdict, QDict *target, +- const char *prefix); +- +-static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) +-{ +- QObject *value; +- const QListEntry *entry; +- char *new_key; +- int i; +- +- /* This function is never called with prefix == NULL, i.e., it is always +- * called from within qdict_flatten_q(list|dict)(). Therefore, it does not +- * need to remove list entries during the iteration (the whole list will be +- * deleted eventually anyway from qdict_flatten_qdict()). */ +- assert(prefix); +- +- entry = qlist_first(qlist); +- +- for (i = 0; entry; entry = qlist_next(entry), i++) { +- value = qlist_entry_obj(entry); +- new_key = g_strdup_printf("%s.%i", prefix, i); +- +- if (qobject_type(value) == QTYPE_QDICT) { +- qdict_flatten_qdict(qobject_to(QDict, value), target, new_key); +- } else if (qobject_type(value) == QTYPE_QLIST) { +- qdict_flatten_qlist(qobject_to(QList, value), target, new_key); +- } else { +- /* All other types are moved to the target unchanged. */ +- qdict_put_obj(target, new_key, qobject_ref(value)); +- } +- +- g_free(new_key); +- } +-} +- +-static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) +-{ +- QObject *value; +- const QDictEntry *entry, *next; +- char *new_key; +- bool delete; +- +- entry = qdict_first(qdict); +- +- while (entry != NULL) { +- +- next = qdict_next(qdict, entry); +- value = qdict_entry_value(entry); +- new_key = NULL; +- delete = false; +- +- if (prefix) { +- new_key = g_strdup_printf("%s.%s", prefix, entry->key); +- } +- +- if (qobject_type(value) == QTYPE_QDICT) { +- /* Entries of QDicts are processed recursively, the QDict object +- * itself disappears. */ +- qdict_flatten_qdict(qobject_to(QDict, value), target, +- new_key ? new_key : entry->key); +- delete = true; +- } else if (qobject_type(value) == QTYPE_QLIST) { +- qdict_flatten_qlist(qobject_to(QList, value), target, +- new_key ? new_key : entry->key); +- delete = true; +- } else if (prefix) { +- /* All other objects are moved to the target unchanged. */ +- qdict_put_obj(target, new_key, qobject_ref(value)); +- delete = true; +- } +- +- g_free(new_key); +- +- if (delete) { +- qdict_del(qdict, entry->key); +- +- /* Restart loop after modifying the iterated QDict */ +- entry = qdict_first(qdict); +- continue; +- } +- +- entry = next; +- } +-} +- +-/** +- * qdict_flatten(): For each nested QDict with key x, all fields with key y +- * are moved to this QDict and their key is renamed to "x.y". For each nested +- * QList with key x, the field at index y is moved to this QDict with the key +- * "x.y" (i.e., the reverse of what qdict_array_split() does). +- * This operation is applied recursively for nested QDicts and QLists. +- */ +-void qdict_flatten(QDict *qdict) +-{ +- qdict_flatten_qdict(qdict, qdict, NULL); +-} +- +-/* extract all the src QDict entries starting by start into dst */ +-void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) +- +-{ +- const QDictEntry *entry, *next; +- const char *p; +- +- *dst = qdict_new(); +- entry = qdict_first(src); +- +- while (entry != NULL) { +- next = qdict_next(src, entry); +- if (strstart(entry->key, start, &p)) { +- qdict_put_obj(*dst, p, qobject_ref(entry->value)); +- qdict_del(src, entry->key); +- } +- entry = next; +- } +-} +- +-static int qdict_count_prefixed_entries(const QDict *src, const char *start) +-{ +- const QDictEntry *entry; +- int count = 0; +- +- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { +- if (strstart(entry->key, start, NULL)) { +- if (count == INT_MAX) { +- return -ERANGE; +- } +- count++; +- } +- } +- +- return count; +-} +- +-/** +- * qdict_array_split(): This function moves array-like elements of a QDict into +- * a new QList. Every entry in the original QDict with a key "%u" or one +- * prefixed "%u.", where %u designates an unsigned integer starting at 0 and +- * incrementally counting up, will be moved to a new QDict at index %u in the +- * output QList with the key prefix removed, if that prefix is "%u.". If the +- * whole key is just "%u", the whole QObject will be moved unchanged without +- * creating a new QDict. The function terminates when there is no entry in the +- * QDict with a prefix directly (incrementally) following the last one; it also +- * returns if there are both entries with "%u" and "%u." for the same index %u. +- * Example: {"0.a": 42, "0.b": 23, "1.x": 0, "4.y": 1, "o.o": 7, "2": 66} +- * (or {"1.x": 0, "4.y": 1, "0.a": 42, "o.o": 7, "0.b": 23, "2": 66}) +- * => [{"a": 42, "b": 23}, {"x": 0}, 66] +- * and {"4.y": 1, "o.o": 7} (remainder of the old QDict) +- */ +-void qdict_array_split(QDict *src, QList **dst) +-{ +- unsigned i; +- +- *dst = qlist_new(); +- +- for (i = 0; i < UINT_MAX; i++) { +- QObject *subqobj; +- bool is_subqdict; +- QDict *subqdict; +- char indexstr[32], prefix[32]; +- size_t snprintf_ret; +- +- snprintf_ret = snprintf(indexstr, 32, "%u", i); +- assert(snprintf_ret < 32); +- +- subqobj = qdict_get(src, indexstr); +- +- snprintf_ret = snprintf(prefix, 32, "%u.", i); +- assert(snprintf_ret < 32); +- +- /* Overflow is the same as positive non-zero results */ +- is_subqdict = qdict_count_prefixed_entries(src, prefix); +- +- // There may be either a single subordinate object (named "%u") or +- // multiple objects (each with a key prefixed "%u."), but not both. +- if (!subqobj == !is_subqdict) { +- break; +- } +- +- if (is_subqdict) { +- qdict_extract_subqdict(src, &subqdict, prefix); +- assert(qdict_size(subqdict) > 0); +- } else { +- qobject_ref(subqobj); +- qdict_del(src, indexstr); +- } +- +- qlist_append_obj(*dst, subqobj ?: QOBJECT(subqdict)); +- } +-} +- +-/** +- * qdict_split_flat_key: +- * @key: the key string to split +- * @prefix: non-NULL pointer to hold extracted prefix +- * @suffix: non-NULL pointer to remaining suffix +- * +- * Given a flattened key such as 'foo.0.bar', split it into two parts +- * at the first '.' separator. Allows double dot ('..') to escape the +- * normal separator. +- * +- * e.g. +- * 'foo.0.bar' -> prefix='foo' and suffix='0.bar' +- * 'foo..0.bar' -> prefix='foo.0' and suffix='bar' +- * +- * The '..' sequence will be unescaped in the returned 'prefix' +- * string. The 'suffix' string will be left in escaped format, so it +- * can be fed back into the qdict_split_flat_key() key as the input +- * later. +- * +- * The caller is responsible for freeing the string returned in @prefix +- * using g_free(). +- */ +-static void qdict_split_flat_key(const char *key, char **prefix, +- const char **suffix) +-{ +- const char *separator; +- size_t i, j; +- +- /* Find first '.' separator, but if there is a pair '..' +- * that acts as an escape, so skip over '..' */ +- separator = NULL; +- do { +- if (separator) { +- separator += 2; +- } else { +- separator = key; +- } +- separator = strchr(separator, '.'); +- } while (separator && separator[1] == '.'); +- +- if (separator) { +- *prefix = g_strndup(key, separator - key); +- *suffix = separator + 1; +- } else { +- *prefix = g_strdup(key); +- *suffix = NULL; +- } +- +- /* Unescape the '..' sequence into '.' */ +- for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) { +- if ((*prefix)[i] == '.') { +- assert((*prefix)[i + 1] == '.'); +- i++; +- } +- (*prefix)[j] = (*prefix)[i]; +- } +- (*prefix)[j] = '\0'; +-} +- +-/** +- * qdict_is_list: +- * @maybe_list: dict to check if keys represent list elements. +- * +- * Determine whether all keys in @maybe_list are valid list elements. +- * If @maybe_list is non-zero in length and all the keys look like +- * valid list indexes, this will return 1. If @maybe_list is zero +- * length or all keys are non-numeric then it will return 0 to indicate +- * it is a normal qdict. If there is a mix of numeric and non-numeric +- * keys, or the list indexes are non-contiguous, an error is reported. +- * +- * Returns: 1 if a valid list, 0 if a dict, -1 on error +- */ +-static int qdict_is_list(QDict *maybe_list, Error **errp) +-{ +- const QDictEntry *ent; +- ssize_t len = 0; +- ssize_t max = -1; +- int is_list = -1; +- int64_t val; +- +- for (ent = qdict_first(maybe_list); ent != NULL; +- ent = qdict_next(maybe_list, ent)) { +- +- if (qemu_strtoi64(ent->key, NULL, 10, &val) == 0) { +- if (is_list == -1) { +- is_list = 1; +- } else if (!is_list) { +- error_setg(errp, +- "Cannot mix list and non-list keys"); +- return -1; +- } +- len++; +- if (val > max) { +- max = val; +- } +- } else { +- if (is_list == -1) { +- is_list = 0; +- } else if (is_list) { +- error_setg(errp, +- "Cannot mix list and non-list keys"); +- return -1; +- } +- } +- } +- +- if (is_list == -1) { +- assert(!qdict_size(maybe_list)); +- is_list = 0; +- } +- +- /* NB this isn't a perfect check - e.g. it won't catch +- * a list containing '1', '+1', '01', '3', but that +- * does not matter - we've still proved that the +- * input is a list. It is up the caller to do a +- * stricter check if desired */ +- if (len != (max + 1)) { +- error_setg(errp, "List indices are not contiguous, " +- "saw %zd elements but %zd largest index", +- len, max); +- return -1; +- } +- +- return is_list; +-} +- +-/** +- * qdict_crumple: +- * @src: the original flat dictionary (only scalar values) to crumple +- * +- * Takes a flat dictionary whose keys use '.' separator to indicate +- * nesting, and values are scalars, and crumples it into a nested +- * structure. +- * +- * To include a literal '.' in a key name, it must be escaped as '..' +- * +- * For example, an input of: +- * +- * { 'foo.0.bar': 'one', 'foo.0.wizz': '1', +- * 'foo.1.bar': 'two', 'foo.1.wizz': '2' } +- * +- * will result in an output of: +- * +- * { +- * 'foo': [ +- * { 'bar': 'one', 'wizz': '1' }, +- * { 'bar': 'two', 'wizz': '2' } +- * ], +- * } +- * +- * The following scenarios in the input dict will result in an +- * error being returned: +- * +- * - Any values in @src are non-scalar types +- * - If keys in @src imply that a particular level is both a +- * list and a dict. e.g., "foo.0.bar" and "foo.eek.bar". +- * - If keys in @src imply that a particular level is a list, +- * but the indices are non-contiguous. e.g. "foo.0.bar" and +- * "foo.2.bar" without any "foo.1.bar" present. +- * - If keys in @src represent list indexes, but are not in +- * the "%zu" format. e.g. "foo.+0.bar" +- * +- * Returns: either a QDict or QList for the nested data structure, or NULL +- * on error +- */ +-QObject *qdict_crumple(const QDict *src, Error **errp) +-{ +- const QDictEntry *ent; +- QDict *two_level, *multi_level = NULL; +- QObject *dst = NULL, *child; +- size_t i; +- char *prefix = NULL; +- const char *suffix = NULL; +- int is_list; +- +- two_level = qdict_new(); +- +- /* Step 1: split our totally flat dict into a two level dict */ +- for (ent = qdict_first(src); ent != NULL; ent = qdict_next(src, ent)) { +- if (qobject_type(ent->value) == QTYPE_QDICT || +- qobject_type(ent->value) == QTYPE_QLIST) { +- error_setg(errp, "Value %s is not a scalar", +- ent->key); +- goto error; +- } +- +- qdict_split_flat_key(ent->key, &prefix, &suffix); +- +- child = qdict_get(two_level, prefix); +- if (suffix) { +- QDict *child_dict = qobject_to(QDict, child); +- if (!child_dict) { +- if (child) { +- error_setg(errp, "Key %s prefix is already set as a scalar", +- prefix); +- goto error; +- } +- +- child_dict = qdict_new(); +- qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); +- } +- +- qdict_put_obj(child_dict, suffix, qobject_ref(ent->value)); +- } else { +- if (child) { +- error_setg(errp, "Key %s prefix is already set as a dict", +- prefix); +- goto error; +- } +- qdict_put_obj(two_level, prefix, qobject_ref(ent->value)); +- } +- +- g_free(prefix); +- prefix = NULL; +- } +- +- /* Step 2: optionally process the two level dict recursively +- * into a multi-level dict */ +- multi_level = qdict_new(); +- for (ent = qdict_first(two_level); ent != NULL; +- ent = qdict_next(two_level, ent)) { +- QDict *dict = qobject_to(QDict, ent->value); +- if (dict) { +- child = qdict_crumple(dict, errp); +- if (!child) { +- goto error; +- } +- +- qdict_put_obj(multi_level, ent->key, child); +- } else { +- qdict_put_obj(multi_level, ent->key, qobject_ref(ent->value)); +- } +- } +- qobject_unref(two_level); +- two_level = NULL; +- +- /* Step 3: detect if we need to turn our dict into list */ +- is_list = qdict_is_list(multi_level, errp); +- if (is_list < 0) { +- goto error; +- } +- +- if (is_list) { +- dst = QOBJECT(qlist_new()); +- +- for (i = 0; i < qdict_size(multi_level); i++) { +- char *key = g_strdup_printf("%zu", i); +- +- child = qdict_get(multi_level, key); +- g_free(key); +- +- if (!child) { +- error_setg(errp, "Missing list index %zu", i); +- goto error; +- } +- +- qlist_append_obj(qobject_to(QList, dst), qobject_ref(child)); +- } +- qobject_unref(multi_level); +- multi_level = NULL; +- } else { +- dst = QOBJECT(multi_level); +- } +- +- return dst; +- +- error: +- g_free(prefix); +- qobject_unref(multi_level); +- qobject_unref(two_level); +- qobject_unref(dst); +- return NULL; +-} +- +-/** +- * qdict_array_entries(): Returns the number of direct array entries if the +- * sub-QDict of src specified by the prefix in subqdict (or src itself for +- * prefix == "") is valid as an array, i.e. the length of the created list if +- * the sub-QDict would become empty after calling qdict_array_split() on it. If +- * the array is not valid, -EINVAL is returned. +- */ +-int qdict_array_entries(QDict *src, const char *subqdict) +-{ +- const QDictEntry *entry; +- unsigned i; +- unsigned entries = 0; +- size_t subqdict_len = strlen(subqdict); +- +- assert(!subqdict_len || subqdict[subqdict_len - 1] == '.'); +- +- /* qdict_array_split() loops until UINT_MAX, but as we want to return +- * negative errors, we only have a signed return value here. Any additional +- * entries will lead to -EINVAL. */ +- for (i = 0; i < INT_MAX; i++) { +- QObject *subqobj; +- int subqdict_entries; +- char *prefix = g_strdup_printf("%s%u.", subqdict, i); +- +- subqdict_entries = qdict_count_prefixed_entries(src, prefix); +- +- /* Remove ending "." */ +- prefix[strlen(prefix) - 1] = 0; +- subqobj = qdict_get(src, prefix); +- +- g_free(prefix); +- +- if (subqdict_entries < 0) { +- return subqdict_entries; +- } +- +- /* There may be either a single subordinate object (named "%u") or +- * multiple objects (each with a key prefixed "%u."), but not both. */ +- if (subqobj && subqdict_entries) { +- return -EINVAL; +- } else if (!subqobj && !subqdict_entries) { +- break; +- } +- +- entries += subqdict_entries ? subqdict_entries : 1; +- } +- +- /* Consider everything handled that isn't part of the given sub-QDict */ +- for (entry = qdict_first(src); entry; entry = qdict_next(src, entry)) { +- if (!strstart(qdict_entry_key(entry), subqdict, NULL)) { +- entries++; +- } +- } +- +- /* Anything left in the sub-QDict that wasn't handled? */ +- if (qdict_size(src) != entries) { +- return -EINVAL; +- } +- +- return i; +-} +- +-/** +- * qdict_join(): Absorb the src QDict into the dest QDict, that is, move all +- * elements from src to dest. +- * +- * If an element from src has a key already present in dest, it will not be +- * moved unless overwrite is true. +- * +- * If overwrite is true, the conflicting values in dest will be discarded and +- * replaced by the corresponding values from src. +- * +- * Therefore, with overwrite being true, the src QDict will always be empty when +- * this function returns. If overwrite is false, the src QDict will be empty +- * iff there were no conflicts. +- */ +-void qdict_join(QDict *dest, QDict *src, bool overwrite) +-{ +- const QDictEntry *entry, *next; +- +- entry = qdict_first(src); +- while (entry) { +- next = qdict_next(src, entry); +- +- if (overwrite || !qdict_haskey(dest, entry->key)) { +- qdict_put_obj(dest, entry->key, qobject_ref(entry->value)); +- qdict_del(src, entry->key); +- } +- +- entry = next; +- } +-} +- +-/** +- * qdict_rename_keys(): Rename keys in qdict according to the replacements +- * specified in the array renames. The array must be terminated by an entry +- * with from = NULL. +- * +- * The renames are performed individually in the order of the array, so entries +- * may be renamed multiple times and may or may not conflict depending on the +- * order of the renames array. +- * +- * Returns true for success, false in error cases. +- */ +-bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) +-{ +- QObject *qobj; +- +- while (renames->from) { +- if (qdict_haskey(qdict, renames->from)) { +- if (qdict_haskey(qdict, renames->to)) { +- error_setg(errp, "'%s' and its alias '%s' can't be used at the " +- "same time", renames->to, renames->from); +- return false; +- } +- +- qobj = qdict_get(qdict, renames->from); +- qdict_put_obj(qdict, renames->to, qobject_ref(qobj)); +- qdict_del(qdict, renames->from); +- } +- +- renames++; +- } +- return true; +-} +diff --git a/tests/Makefile.include b/tests/Makefile.include +index 0464e52..cb12ee6 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -40,6 +40,8 @@ SYSEMU_TARGET_LIST := $(subst -softmmu.mak,,$(notdir \ + + check-unit-y = tests/check-qdict$(EXESUF) + gcov-files-check-qdict-y = qobject/qdict.c ++check-unit-y = tests/check-block-qdict$(EXESUF) ++gcov-files-check-block-qdict-y = qobject/block-qdict.c + check-unit-y += tests/test-char$(EXESUF) + gcov-files-check-qdict-y = chardev/char.c + check-unit-y += tests/check-qnum$(EXESUF) +@@ -580,6 +582,7 @@ GENERATED_FILES += tests/test-qapi-types.h tests/test-qapi-visit.h \ + test-obj-y = tests/check-qnum.o tests/check-qstring.o tests/check-qdict.o \ + tests/check-qlist.o tests/check-qnull.o tests/check-qobject.o \ + tests/check-qjson.o tests/check-qlit.o \ ++ tests/check-block-qtest.o \ + tests/test-coroutine.o tests/test-string-output-visitor.o \ + tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \ + tests/test-clone-visitor.o \ +@@ -610,6 +613,7 @@ test-block-obj-y = $(block-obj-y) $(test-io-obj-y) tests/iothread.o + tests/check-qnum$(EXESUF): tests/check-qnum.o $(test-util-obj-y) + tests/check-qstring$(EXESUF): tests/check-qstring.o $(test-util-obj-y) + tests/check-qdict$(EXESUF): tests/check-qdict.o $(test-util-obj-y) ++tests/check-block-qdict$(EXESUF): tests/check-block-qdict.o $(test-util-obj-y) + tests/check-qlist$(EXESUF): tests/check-qlist.o $(test-util-obj-y) + tests/check-qnull$(EXESUF): tests/check-qnull.o $(test-util-obj-y) + tests/check-qobject$(EXESUF): tests/check-qobject.o $(test-util-obj-y) +diff --git a/tests/check-block-qdict.c b/tests/check-block-qdict.c +new file mode 100644 +index 0000000..5b9f4d5 +--- /dev/null ++++ b/tests/check-block-qdict.c +@@ -0,0 +1,655 @@ ++/* ++ * Unit-tests for Block layer QDict extras ++ * ++ * Copyright (c) 2013-2018 Red Hat, Inc. ++ * ++ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later. ++ * See the COPYING.LIB file in the top-level directory. ++ */ ++ ++#include "qemu/osdep.h" ++#include "block/qdict.h" ++#include "qapi/qmp/qlist.h" ++#include "qapi/qmp/qnum.h" ++#include "qapi/error.h" ++ ++static void qdict_defaults_test(void) ++{ ++ QDict *dict, *copy; ++ ++ dict = qdict_new(); ++ copy = qdict_new(); ++ ++ qdict_set_default_str(dict, "foo", "abc"); ++ qdict_set_default_str(dict, "foo", "def"); ++ g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc"); ++ qdict_set_default_str(dict, "bar", "ghi"); ++ ++ qdict_copy_default(copy, dict, "foo"); ++ g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc"); ++ qdict_set_default_str(copy, "bar", "xyz"); ++ qdict_copy_default(copy, dict, "bar"); ++ g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); ++ ++ qobject_unref(copy); ++ qobject_unref(dict); ++} ++ ++static void qdict_flatten_test(void) ++{ ++ QList *list1 = qlist_new(); ++ QList *list2 = qlist_new(); ++ QDict *dict1 = qdict_new(); ++ QDict *dict2 = qdict_new(); ++ QDict *dict3 = qdict_new(); ++ ++ /* ++ * Test the flattening of ++ * ++ * { ++ * "e": [ ++ * 42, ++ * [ ++ * 23, ++ * 66, ++ * { ++ * "a": 0, ++ * "b": 1 ++ * } ++ * ] ++ * ], ++ * "f": { ++ * "c": 2, ++ * "d": 3, ++ * }, ++ * "g": 4 ++ * } ++ * ++ * to ++ * ++ * { ++ * "e.0": 42, ++ * "e.1.0": 23, ++ * "e.1.1": 66, ++ * "e.1.2.a": 0, ++ * "e.1.2.b": 1, ++ * "f.c": 2, ++ * "f.d": 3, ++ * "g": 4 ++ * } ++ */ ++ ++ qdict_put_int(dict1, "a", 0); ++ qdict_put_int(dict1, "b", 1); ++ ++ qlist_append_int(list1, 23); ++ qlist_append_int(list1, 66); ++ qlist_append(list1, dict1); ++ qlist_append_int(list2, 42); ++ qlist_append(list2, list1); ++ ++ qdict_put_int(dict2, "c", 2); ++ qdict_put_int(dict2, "d", 3); ++ qdict_put(dict3, "e", list2); ++ qdict_put(dict3, "f", dict2); ++ qdict_put_int(dict3, "g", 4); ++ ++ qdict_flatten(dict3); ++ ++ g_assert(qdict_get_int(dict3, "e.0") == 42); ++ g_assert(qdict_get_int(dict3, "e.1.0") == 23); ++ g_assert(qdict_get_int(dict3, "e.1.1") == 66); ++ g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); ++ g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); ++ g_assert(qdict_get_int(dict3, "f.c") == 2); ++ g_assert(qdict_get_int(dict3, "f.d") == 3); ++ g_assert(qdict_get_int(dict3, "g") == 4); ++ ++ g_assert(qdict_size(dict3) == 8); ++ ++ qobject_unref(dict3); ++} ++ ++static void qdict_array_split_test(void) ++{ ++ QDict *test_dict = qdict_new(); ++ QDict *dict1, *dict2; ++ QNum *int1; ++ QList *test_list; ++ ++ /* ++ * Test the split of ++ * ++ * { ++ * "1.x": 0, ++ * "4.y": 1, ++ * "0.a": 42, ++ * "o.o": 7, ++ * "0.b": 23, ++ * "2": 66 ++ * } ++ * ++ * to ++ * ++ * [ ++ * { ++ * "a": 42, ++ * "b": 23 ++ * }, ++ * { ++ * "x": 0 ++ * }, ++ * 66 ++ * ] ++ * ++ * and ++ * ++ * { ++ * "4.y": 1, ++ * "o.o": 7 ++ * } ++ * ++ * (remaining in the old QDict) ++ * ++ * This example is given in the comment of qdict_array_split(). ++ */ ++ ++ qdict_put_int(test_dict, "1.x", 0); ++ qdict_put_int(test_dict, "4.y", 1); ++ qdict_put_int(test_dict, "0.a", 42); ++ qdict_put_int(test_dict, "o.o", 7); ++ qdict_put_int(test_dict, "0.b", 23); ++ qdict_put_int(test_dict, "2", 66); ++ ++ qdict_array_split(test_dict, &test_list); ++ ++ dict1 = qobject_to(QDict, qlist_pop(test_list)); ++ dict2 = qobject_to(QDict, qlist_pop(test_list)); ++ int1 = qobject_to(QNum, qlist_pop(test_list)); ++ ++ g_assert(dict1); ++ g_assert(dict2); ++ g_assert(int1); ++ g_assert(qlist_empty(test_list)); ++ ++ qobject_unref(test_list); ++ ++ g_assert(qdict_get_int(dict1, "a") == 42); ++ g_assert(qdict_get_int(dict1, "b") == 23); ++ ++ g_assert(qdict_size(dict1) == 2); ++ ++ qobject_unref(dict1); ++ ++ g_assert(qdict_get_int(dict2, "x") == 0); ++ ++ g_assert(qdict_size(dict2) == 1); ++ ++ qobject_unref(dict2); ++ ++ g_assert_cmpint(qnum_get_int(int1), ==, 66); ++ ++ qobject_unref(int1); ++ ++ g_assert(qdict_get_int(test_dict, "4.y") == 1); ++ g_assert(qdict_get_int(test_dict, "o.o") == 7); ++ ++ g_assert(qdict_size(test_dict) == 2); ++ ++ qobject_unref(test_dict); ++ ++ /* ++ * Test the split of ++ * ++ * { ++ * "0": 42, ++ * "1": 23, ++ * "1.x": 84 ++ * } ++ * ++ * to ++ * ++ * [ ++ * 42 ++ * ] ++ * ++ * and ++ * ++ * { ++ * "1": 23, ++ * "1.x": 84 ++ * } ++ * ++ * That is, test whether splitting stops if there is both an entry with key ++ * of "%u" and other entries with keys prefixed "%u." for the same index. ++ */ ++ ++ test_dict = qdict_new(); ++ ++ qdict_put_int(test_dict, "0", 42); ++ qdict_put_int(test_dict, "1", 23); ++ qdict_put_int(test_dict, "1.x", 84); ++ ++ qdict_array_split(test_dict, &test_list); ++ ++ int1 = qobject_to(QNum, qlist_pop(test_list)); ++ ++ g_assert(int1); ++ g_assert(qlist_empty(test_list)); ++ ++ qobject_unref(test_list); ++ ++ g_assert_cmpint(qnum_get_int(int1), ==, 42); ++ ++ qobject_unref(int1); ++ ++ g_assert(qdict_get_int(test_dict, "1") == 23); ++ g_assert(qdict_get_int(test_dict, "1.x") == 84); ++ ++ g_assert(qdict_size(test_dict) == 2); ++ ++ qobject_unref(test_dict); ++} ++ ++static void qdict_array_entries_test(void) ++{ ++ QDict *dict = qdict_new(); ++ ++ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); ++ ++ qdict_put_int(dict, "bar", 0); ++ qdict_put_int(dict, "baz.0", 0); ++ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); ++ ++ qdict_put_int(dict, "foo.1", 0); ++ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); ++ qdict_put_int(dict, "foo.0", 0); ++ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2); ++ qdict_put_int(dict, "foo.bar", 0); ++ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); ++ qdict_del(dict, "foo.bar"); ++ ++ qdict_put_int(dict, "foo.2.a", 0); ++ qdict_put_int(dict, "foo.2.b", 0); ++ qdict_put_int(dict, "foo.2.c", 0); ++ g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); ++ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); ++ ++ qobject_unref(dict); ++ ++ dict = qdict_new(); ++ qdict_put_int(dict, "1", 0); ++ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); ++ qdict_put_int(dict, "0", 0); ++ g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2); ++ qdict_put_int(dict, "bar", 0); ++ g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); ++ qdict_del(dict, "bar"); ++ ++ qdict_put_int(dict, "2.a", 0); ++ qdict_put_int(dict, "2.b", 0); ++ qdict_put_int(dict, "2.c", 0); ++ g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); ++ ++ qobject_unref(dict); ++} ++ ++static void qdict_join_test(void) ++{ ++ QDict *dict1, *dict2; ++ bool overwrite = false; ++ int i; ++ ++ dict1 = qdict_new(); ++ dict2 = qdict_new(); ++ ++ /* Test everything once without overwrite and once with */ ++ do { ++ /* Test empty dicts */ ++ qdict_join(dict1, dict2, overwrite); ++ ++ g_assert(qdict_size(dict1) == 0); ++ g_assert(qdict_size(dict2) == 0); ++ ++ /* First iteration: Test movement */ ++ /* Second iteration: Test empty source and non-empty destination */ ++ qdict_put_int(dict2, "foo", 42); ++ ++ for (i = 0; i < 2; i++) { ++ qdict_join(dict1, dict2, overwrite); ++ ++ g_assert(qdict_size(dict1) == 1); ++ g_assert(qdict_size(dict2) == 0); ++ ++ g_assert(qdict_get_int(dict1, "foo") == 42); ++ } ++ ++ /* Test non-empty source and destination without conflict */ ++ qdict_put_int(dict2, "bar", 23); ++ ++ qdict_join(dict1, dict2, overwrite); ++ ++ g_assert(qdict_size(dict1) == 2); ++ g_assert(qdict_size(dict2) == 0); ++ ++ g_assert(qdict_get_int(dict1, "foo") == 42); ++ g_assert(qdict_get_int(dict1, "bar") == 23); ++ ++ /* Test conflict */ ++ qdict_put_int(dict2, "foo", 84); ++ ++ qdict_join(dict1, dict2, overwrite); ++ ++ g_assert(qdict_size(dict1) == 2); ++ g_assert(qdict_size(dict2) == !overwrite); ++ ++ g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42)); ++ g_assert(qdict_get_int(dict1, "bar") == 23); ++ ++ if (!overwrite) { ++ g_assert(qdict_get_int(dict2, "foo") == 84); ++ } ++ ++ /* Check the references */ ++ g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); ++ g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); ++ ++ if (!overwrite) { ++ g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); ++ } ++ ++ /* Clean up */ ++ qdict_del(dict1, "foo"); ++ qdict_del(dict1, "bar"); ++ ++ if (!overwrite) { ++ qdict_del(dict2, "foo"); ++ } ++ } while (overwrite ^= true); ++ ++ qobject_unref(dict1); ++ qobject_unref(dict2); ++} ++ ++static void qdict_crumple_test_recursive(void) ++{ ++ QDict *src, *dst, *rule, *vnc, *acl, *listen; ++ QList *rules; ++ ++ src = qdict_new(); ++ qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); ++ qdict_put_str(src, "vnc.listen.port", "5901"); ++ qdict_put_str(src, "vnc.acl.rules.0.match", "fred"); ++ qdict_put_str(src, "vnc.acl.rules.0.policy", "allow"); ++ qdict_put_str(src, "vnc.acl.rules.1.match", "bob"); ++ qdict_put_str(src, "vnc.acl.rules.1.policy", "deny"); ++ qdict_put_str(src, "vnc.acl.default", "deny"); ++ qdict_put_str(src, "vnc.acl..name", "acl0"); ++ qdict_put_str(src, "vnc.acl.rule..name", "acl0"); ++ ++ dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); ++ g_assert(dst); ++ g_assert_cmpint(qdict_size(dst), ==, 1); ++ ++ vnc = qdict_get_qdict(dst, "vnc"); ++ g_assert(vnc); ++ g_assert_cmpint(qdict_size(vnc), ==, 3); ++ ++ listen = qdict_get_qdict(vnc, "listen"); ++ g_assert(listen); ++ g_assert_cmpint(qdict_size(listen), ==, 2); ++ g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr")); ++ g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port")); ++ ++ acl = qdict_get_qdict(vnc, "acl"); ++ g_assert(acl); ++ g_assert_cmpint(qdict_size(acl), ==, 3); ++ ++ rules = qdict_get_qlist(acl, "rules"); ++ g_assert(rules); ++ g_assert_cmpint(qlist_size(rules), ==, 2); ++ ++ rule = qobject_to(QDict, qlist_pop(rules)); ++ g_assert(rule); ++ g_assert_cmpint(qdict_size(rule), ==, 2); ++ g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); ++ g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); ++ qobject_unref(rule); ++ ++ rule = qobject_to(QDict, qlist_pop(rules)); ++ g_assert(rule); ++ g_assert_cmpint(qdict_size(rule), ==, 2); ++ g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); ++ g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); ++ qobject_unref(rule); ++ ++ /* With recursive crumpling, we should see all names unescaped */ ++ g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); ++ g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); ++ ++ qobject_unref(src); ++ qobject_unref(dst); ++} ++ ++static void qdict_crumple_test_empty(void) ++{ ++ QDict *src, *dst; ++ ++ src = qdict_new(); ++ ++ dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); ++ ++ g_assert_cmpint(qdict_size(dst), ==, 0); ++ ++ qobject_unref(src); ++ qobject_unref(dst); ++} ++ ++static int qdict_count_entries(QDict *dict) ++{ ++ const QDictEntry *e; ++ int count = 0; ++ ++ for (e = qdict_first(dict); e; e = qdict_next(dict, e)) { ++ count++; ++ } ++ ++ return count; ++} ++ ++static void qdict_rename_keys_test(void) ++{ ++ QDict *dict = qdict_new(); ++ QDict *copy; ++ QDictRenames *renames; ++ Error *local_err = NULL; ++ ++ qdict_put_str(dict, "abc", "foo"); ++ qdict_put_str(dict, "abcdef", "bar"); ++ qdict_put_int(dict, "number", 42); ++ qdict_put_bool(dict, "flag", true); ++ qdict_put_null(dict, "nothing"); ++ ++ /* Empty rename list */ ++ renames = (QDictRenames[]) { ++ { NULL, "this can be anything" } ++ }; ++ copy = qdict_clone_shallow(dict); ++ qdict_rename_keys(copy, renames, &error_abort); ++ ++ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); ++ g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); ++ g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); ++ g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); ++ g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); ++ g_assert_cmpint(qdict_count_entries(copy), ==, 5); ++ ++ qobject_unref(copy); ++ ++ /* Simple rename of all entries */ ++ renames = (QDictRenames[]) { ++ { "abc", "str1" }, ++ { "abcdef", "str2" }, ++ { "number", "int" }, ++ { "flag", "bool" }, ++ { "nothing", "null" }, ++ { NULL , NULL } ++ }; ++ copy = qdict_clone_shallow(dict); ++ qdict_rename_keys(copy, renames, &error_abort); ++ ++ g_assert(!qdict_haskey(copy, "abc")); ++ g_assert(!qdict_haskey(copy, "abcdef")); ++ g_assert(!qdict_haskey(copy, "number")); ++ g_assert(!qdict_haskey(copy, "flag")); ++ g_assert(!qdict_haskey(copy, "nothing")); ++ ++ g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo"); ++ g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar"); ++ g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42); ++ g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true); ++ g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); ++ g_assert_cmpint(qdict_count_entries(copy), ==, 5); ++ ++ qobject_unref(copy); ++ ++ /* Renames are processed top to bottom */ ++ renames = (QDictRenames[]) { ++ { "abc", "tmp" }, ++ { "abcdef", "abc" }, ++ { "number", "abcdef" }, ++ { "flag", "number" }, ++ { "nothing", "flag" }, ++ { "tmp", "nothing" }, ++ { NULL , NULL } ++ }; ++ copy = qdict_clone_shallow(dict); ++ qdict_rename_keys(copy, renames, &error_abort); ++ ++ g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo"); ++ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar"); ++ g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42); ++ g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true); ++ g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL); ++ g_assert(!qdict_haskey(copy, "tmp")); ++ g_assert_cmpint(qdict_count_entries(copy), ==, 5); ++ ++ qobject_unref(copy); ++ ++ /* Conflicting rename */ ++ renames = (QDictRenames[]) { ++ { "abcdef", "abc" }, ++ { NULL , NULL } ++ }; ++ copy = qdict_clone_shallow(dict); ++ qdict_rename_keys(copy, renames, &local_err); ++ ++ g_assert(local_err != NULL); ++ error_free(local_err); ++ local_err = NULL; ++ ++ g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); ++ g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); ++ g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); ++ g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); ++ g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); ++ g_assert_cmpint(qdict_count_entries(copy), ==, 5); ++ ++ qobject_unref(copy); ++ ++ /* Renames in an empty dict */ ++ renames = (QDictRenames[]) { ++ { "abcdef", "abc" }, ++ { NULL , NULL } ++ }; ++ ++ qobject_unref(dict); ++ dict = qdict_new(); ++ ++ qdict_rename_keys(dict, renames, &error_abort); ++ g_assert(qdict_first(dict) == NULL); ++ ++ qobject_unref(dict); ++} ++ ++static void qdict_crumple_test_bad_inputs(void) ++{ ++ QDict *src; ++ Error *error = NULL; ++ ++ src = qdict_new(); ++ /* rule.0 can't be both a string and a dict */ ++ qdict_put_str(src, "rule.0", "fred"); ++ qdict_put_str(src, "rule.0.policy", "allow"); ++ ++ g_assert(qdict_crumple(src, &error) == NULL); ++ g_assert(error != NULL); ++ error_free(error); ++ error = NULL; ++ qobject_unref(src); ++ ++ src = qdict_new(); ++ /* rule can't be both a list and a dict */ ++ qdict_put_str(src, "rule.0", "fred"); ++ qdict_put_str(src, "rule.a", "allow"); ++ ++ g_assert(qdict_crumple(src, &error) == NULL); ++ g_assert(error != NULL); ++ error_free(error); ++ error = NULL; ++ qobject_unref(src); ++ ++ src = qdict_new(); ++ /* The input should be flat, ie no dicts or lists */ ++ qdict_put(src, "rule.a", qdict_new()); ++ qdict_put_str(src, "rule.b", "allow"); ++ ++ g_assert(qdict_crumple(src, &error) == NULL); ++ g_assert(error != NULL); ++ error_free(error); ++ error = NULL; ++ qobject_unref(src); ++ ++ src = qdict_new(); ++ /* List indexes must not have gaps */ ++ qdict_put_str(src, "rule.0", "deny"); ++ qdict_put_str(src, "rule.3", "allow"); ++ ++ g_assert(qdict_crumple(src, &error) == NULL); ++ g_assert(error != NULL); ++ error_free(error); ++ error = NULL; ++ qobject_unref(src); ++ ++ src = qdict_new(); ++ /* List indexes must be in %zu format */ ++ qdict_put_str(src, "rule.0", "deny"); ++ qdict_put_str(src, "rule.+1", "allow"); ++ ++ g_assert(qdict_crumple(src, &error) == NULL); ++ g_assert(error != NULL); ++ error_free(error); ++ error = NULL; ++ qobject_unref(src); ++} ++ ++int main(int argc, char **argv) ++{ ++ g_test_init(&argc, &argv, NULL); ++ ++ g_test_add_func("/public/defaults", qdict_defaults_test); ++ g_test_add_func("/public/flatten", qdict_flatten_test); ++ g_test_add_func("/public/array_split", qdict_array_split_test); ++ g_test_add_func("/public/array_entries", qdict_array_entries_test); ++ g_test_add_func("/public/join", qdict_join_test); ++ g_test_add_func("/public/crumple/recursive", ++ qdict_crumple_test_recursive); ++ g_test_add_func("/public/crumple/empty", ++ qdict_crumple_test_empty); ++ g_test_add_func("/public/crumple/bad_inputs", ++ qdict_crumple_test_bad_inputs); ++ ++ g_test_add_func("/public/rename_keys", qdict_rename_keys_test); ++ ++ return g_test_run(); ++} +diff --git a/tests/check-qdict.c b/tests/check-qdict.c +index 93e2112..86e9fe7 100644 +--- a/tests/check-qdict.c ++++ b/tests/check-qdict.c +@@ -11,13 +11,7 @@ + */ + + #include "qemu/osdep.h" +-#include "block/qdict.h" + #include "qapi/qmp/qdict.h" +-#include "qapi/qmp/qlist.h" +-#include "qapi/qmp/qnum.h" +-#include "qapi/qmp/qstring.h" +-#include "qapi/error.h" +-#include "qemu-common.h" + + /* + * Public Interface test-cases +@@ -157,28 +151,6 @@ static void qdict_get_try_str_test(void) + qobject_unref(tests_dict); + } + +-static void qdict_defaults_test(void) +-{ +- QDict *dict, *copy; +- +- dict = qdict_new(); +- copy = qdict_new(); +- +- qdict_set_default_str(dict, "foo", "abc"); +- qdict_set_default_str(dict, "foo", "def"); +- g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "abc"); +- qdict_set_default_str(dict, "bar", "ghi"); +- +- qdict_copy_default(copy, dict, "foo"); +- g_assert_cmpstr(qdict_get_str(copy, "foo"), ==, "abc"); +- qdict_set_default_str(copy, "bar", "xyz"); +- qdict_copy_default(copy, dict, "bar"); +- g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); +- +- qobject_unref(copy); +- qobject_unref(dict); +-} +- + static void qdict_haskey_not_test(void) + { + QDict *tests_dict = qdict_new(); +@@ -254,606 +226,6 @@ static void qdict_iterapi_test(void) + qobject_unref(tests_dict); + } + +-static void qdict_flatten_test(void) +-{ +- QList *list1 = qlist_new(); +- QList *list2 = qlist_new(); +- QDict *dict1 = qdict_new(); +- QDict *dict2 = qdict_new(); +- QDict *dict3 = qdict_new(); +- +- /* +- * Test the flattening of +- * +- * { +- * "e": [ +- * 42, +- * [ +- * 23, +- * 66, +- * { +- * "a": 0, +- * "b": 1 +- * } +- * ] +- * ], +- * "f": { +- * "c": 2, +- * "d": 3, +- * }, +- * "g": 4 +- * } +- * +- * to +- * +- * { +- * "e.0": 42, +- * "e.1.0": 23, +- * "e.1.1": 66, +- * "e.1.2.a": 0, +- * "e.1.2.b": 1, +- * "f.c": 2, +- * "f.d": 3, +- * "g": 4 +- * } +- */ +- +- qdict_put_int(dict1, "a", 0); +- qdict_put_int(dict1, "b", 1); +- +- qlist_append_int(list1, 23); +- qlist_append_int(list1, 66); +- qlist_append(list1, dict1); +- qlist_append_int(list2, 42); +- qlist_append(list2, list1); +- +- qdict_put_int(dict2, "c", 2); +- qdict_put_int(dict2, "d", 3); +- qdict_put(dict3, "e", list2); +- qdict_put(dict3, "f", dict2); +- qdict_put_int(dict3, "g", 4); +- +- qdict_flatten(dict3); +- +- g_assert(qdict_get_int(dict3, "e.0") == 42); +- g_assert(qdict_get_int(dict3, "e.1.0") == 23); +- g_assert(qdict_get_int(dict3, "e.1.1") == 66); +- g_assert(qdict_get_int(dict3, "e.1.2.a") == 0); +- g_assert(qdict_get_int(dict3, "e.1.2.b") == 1); +- g_assert(qdict_get_int(dict3, "f.c") == 2); +- g_assert(qdict_get_int(dict3, "f.d") == 3); +- g_assert(qdict_get_int(dict3, "g") == 4); +- +- g_assert(qdict_size(dict3) == 8); +- +- qobject_unref(dict3); +-} +- +-static void qdict_array_split_test(void) +-{ +- QDict *test_dict = qdict_new(); +- QDict *dict1, *dict2; +- QNum *int1; +- QList *test_list; +- +- /* +- * Test the split of +- * +- * { +- * "1.x": 0, +- * "4.y": 1, +- * "0.a": 42, +- * "o.o": 7, +- * "0.b": 23, +- * "2": 66 +- * } +- * +- * to +- * +- * [ +- * { +- * "a": 42, +- * "b": 23 +- * }, +- * { +- * "x": 0 +- * }, +- * 66 +- * ] +- * +- * and +- * +- * { +- * "4.y": 1, +- * "o.o": 7 +- * } +- * +- * (remaining in the old QDict) +- * +- * This example is given in the comment of qdict_array_split(). +- */ +- +- qdict_put_int(test_dict, "1.x", 0); +- qdict_put_int(test_dict, "4.y", 1); +- qdict_put_int(test_dict, "0.a", 42); +- qdict_put_int(test_dict, "o.o", 7); +- qdict_put_int(test_dict, "0.b", 23); +- qdict_put_int(test_dict, "2", 66); +- +- qdict_array_split(test_dict, &test_list); +- +- dict1 = qobject_to(QDict, qlist_pop(test_list)); +- dict2 = qobject_to(QDict, qlist_pop(test_list)); +- int1 = qobject_to(QNum, qlist_pop(test_list)); +- +- g_assert(dict1); +- g_assert(dict2); +- g_assert(int1); +- g_assert(qlist_empty(test_list)); +- +- qobject_unref(test_list); +- +- g_assert(qdict_get_int(dict1, "a") == 42); +- g_assert(qdict_get_int(dict1, "b") == 23); +- +- g_assert(qdict_size(dict1) == 2); +- +- qobject_unref(dict1); +- +- g_assert(qdict_get_int(dict2, "x") == 0); +- +- g_assert(qdict_size(dict2) == 1); +- +- qobject_unref(dict2); +- +- g_assert_cmpint(qnum_get_int(int1), ==, 66); +- +- qobject_unref(int1); +- +- g_assert(qdict_get_int(test_dict, "4.y") == 1); +- g_assert(qdict_get_int(test_dict, "o.o") == 7); +- +- g_assert(qdict_size(test_dict) == 2); +- +- qobject_unref(test_dict); +- +- /* +- * Test the split of +- * +- * { +- * "0": 42, +- * "1": 23, +- * "1.x": 84 +- * } +- * +- * to +- * +- * [ +- * 42 +- * ] +- * +- * and +- * +- * { +- * "1": 23, +- * "1.x": 84 +- * } +- * +- * That is, test whether splitting stops if there is both an entry with key +- * of "%u" and other entries with keys prefixed "%u." for the same index. +- */ +- +- test_dict = qdict_new(); +- +- qdict_put_int(test_dict, "0", 42); +- qdict_put_int(test_dict, "1", 23); +- qdict_put_int(test_dict, "1.x", 84); +- +- qdict_array_split(test_dict, &test_list); +- +- int1 = qobject_to(QNum, qlist_pop(test_list)); +- +- g_assert(int1); +- g_assert(qlist_empty(test_list)); +- +- qobject_unref(test_list); +- +- g_assert_cmpint(qnum_get_int(int1), ==, 42); +- +- qobject_unref(int1); +- +- g_assert(qdict_get_int(test_dict, "1") == 23); +- g_assert(qdict_get_int(test_dict, "1.x") == 84); +- +- g_assert(qdict_size(test_dict) == 2); +- +- qobject_unref(test_dict); +-} +- +-static void qdict_array_entries_test(void) +-{ +- QDict *dict = qdict_new(); +- +- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); +- +- qdict_put_int(dict, "bar", 0); +- qdict_put_int(dict, "baz.0", 0); +- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 0); +- +- qdict_put_int(dict, "foo.1", 0); +- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); +- qdict_put_int(dict, "foo.0", 0); +- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 2); +- qdict_put_int(dict, "foo.bar", 0); +- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, -EINVAL); +- qdict_del(dict, "foo.bar"); +- +- qdict_put_int(dict, "foo.2.a", 0); +- qdict_put_int(dict, "foo.2.b", 0); +- qdict_put_int(dict, "foo.2.c", 0); +- g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); +- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); +- +- qobject_unref(dict); +- +- dict = qdict_new(); +- qdict_put_int(dict, "1", 0); +- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); +- qdict_put_int(dict, "0", 0); +- g_assert_cmpint(qdict_array_entries(dict, ""), ==, 2); +- qdict_put_int(dict, "bar", 0); +- g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); +- qdict_del(dict, "bar"); +- +- qdict_put_int(dict, "2.a", 0); +- qdict_put_int(dict, "2.b", 0); +- qdict_put_int(dict, "2.c", 0); +- g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); +- +- qobject_unref(dict); +-} +- +-static void qdict_join_test(void) +-{ +- QDict *dict1, *dict2; +- bool overwrite = false; +- int i; +- +- dict1 = qdict_new(); +- dict2 = qdict_new(); +- +- /* Test everything once without overwrite and once with */ +- do +- { +- /* Test empty dicts */ +- qdict_join(dict1, dict2, overwrite); +- +- g_assert(qdict_size(dict1) == 0); +- g_assert(qdict_size(dict2) == 0); +- +- /* First iteration: Test movement */ +- /* Second iteration: Test empty source and non-empty destination */ +- qdict_put_int(dict2, "foo", 42); +- +- for (i = 0; i < 2; i++) { +- qdict_join(dict1, dict2, overwrite); +- +- g_assert(qdict_size(dict1) == 1); +- g_assert(qdict_size(dict2) == 0); +- +- g_assert(qdict_get_int(dict1, "foo") == 42); +- } +- +- /* Test non-empty source and destination without conflict */ +- qdict_put_int(dict2, "bar", 23); +- +- qdict_join(dict1, dict2, overwrite); +- +- g_assert(qdict_size(dict1) == 2); +- g_assert(qdict_size(dict2) == 0); +- +- g_assert(qdict_get_int(dict1, "foo") == 42); +- g_assert(qdict_get_int(dict1, "bar") == 23); +- +- /* Test conflict */ +- qdict_put_int(dict2, "foo", 84); +- +- qdict_join(dict1, dict2, overwrite); +- +- g_assert(qdict_size(dict1) == 2); +- g_assert(qdict_size(dict2) == !overwrite); +- +- g_assert(qdict_get_int(dict1, "foo") == (overwrite ? 84 : 42)); +- g_assert(qdict_get_int(dict1, "bar") == 23); +- +- if (!overwrite) { +- g_assert(qdict_get_int(dict2, "foo") == 84); +- } +- +- /* Check the references */ +- g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); +- g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); +- +- if (!overwrite) { +- g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); +- } +- +- /* Clean up */ +- qdict_del(dict1, "foo"); +- qdict_del(dict1, "bar"); +- +- if (!overwrite) { +- qdict_del(dict2, "foo"); +- } +- } +- while (overwrite ^= true); +- +- qobject_unref(dict1); +- qobject_unref(dict2); +-} +- +-static void qdict_crumple_test_recursive(void) +-{ +- QDict *src, *dst, *rule, *vnc, *acl, *listen; +- QList *rules; +- +- src = qdict_new(); +- qdict_put_str(src, "vnc.listen.addr", "127.0.0.1"); +- qdict_put_str(src, "vnc.listen.port", "5901"); +- qdict_put_str(src, "vnc.acl.rules.0.match", "fred"); +- qdict_put_str(src, "vnc.acl.rules.0.policy", "allow"); +- qdict_put_str(src, "vnc.acl.rules.1.match", "bob"); +- qdict_put_str(src, "vnc.acl.rules.1.policy", "deny"); +- qdict_put_str(src, "vnc.acl.default", "deny"); +- qdict_put_str(src, "vnc.acl..name", "acl0"); +- qdict_put_str(src, "vnc.acl.rule..name", "acl0"); +- +- dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); +- g_assert(dst); +- g_assert_cmpint(qdict_size(dst), ==, 1); +- +- vnc = qdict_get_qdict(dst, "vnc"); +- g_assert(vnc); +- g_assert_cmpint(qdict_size(vnc), ==, 3); +- +- listen = qdict_get_qdict(vnc, "listen"); +- g_assert(listen); +- g_assert_cmpint(qdict_size(listen), ==, 2); +- g_assert_cmpstr("127.0.0.1", ==, qdict_get_str(listen, "addr")); +- g_assert_cmpstr("5901", ==, qdict_get_str(listen, "port")); +- +- acl = qdict_get_qdict(vnc, "acl"); +- g_assert(acl); +- g_assert_cmpint(qdict_size(acl), ==, 3); +- +- rules = qdict_get_qlist(acl, "rules"); +- g_assert(rules); +- g_assert_cmpint(qlist_size(rules), ==, 2); +- +- rule = qobject_to(QDict, qlist_pop(rules)); +- g_assert(rule); +- g_assert_cmpint(qdict_size(rule), ==, 2); +- g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); +- g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); +- qobject_unref(rule); +- +- rule = qobject_to(QDict, qlist_pop(rules)); +- g_assert(rule); +- g_assert_cmpint(qdict_size(rule), ==, 2); +- g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); +- g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); +- qobject_unref(rule); +- +- /* With recursive crumpling, we should see all names unescaped */ +- g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); +- g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); +- +- qobject_unref(src); +- qobject_unref(dst); +-} +- +-static void qdict_crumple_test_empty(void) +-{ +- QDict *src, *dst; +- +- src = qdict_new(); +- +- dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); +- +- g_assert_cmpint(qdict_size(dst), ==, 0); +- +- qobject_unref(src); +- qobject_unref(dst); +-} +- +-static int qdict_count_entries(QDict *dict) +-{ +- const QDictEntry *e; +- int count = 0; +- +- for (e = qdict_first(dict); e; e = qdict_next(dict, e)) { +- count++; +- } +- +- return count; +-} +- +-static void qdict_rename_keys_test(void) +-{ +- QDict *dict = qdict_new(); +- QDict *copy; +- QDictRenames *renames; +- Error *local_err = NULL; +- +- qdict_put_str(dict, "abc", "foo"); +- qdict_put_str(dict, "abcdef", "bar"); +- qdict_put_int(dict, "number", 42); +- qdict_put_bool(dict, "flag", true); +- qdict_put_null(dict, "nothing"); +- +- /* Empty rename list */ +- renames = (QDictRenames[]) { +- { NULL, "this can be anything" } +- }; +- copy = qdict_clone_shallow(dict); +- qdict_rename_keys(copy, renames, &error_abort); +- +- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); +- g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); +- g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); +- g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); +- g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); +- g_assert_cmpint(qdict_count_entries(copy), ==, 5); +- +- qobject_unref(copy); +- +- /* Simple rename of all entries */ +- renames = (QDictRenames[]) { +- { "abc", "str1" }, +- { "abcdef", "str2" }, +- { "number", "int" }, +- { "flag", "bool" }, +- { "nothing", "null" }, +- { NULL , NULL } +- }; +- copy = qdict_clone_shallow(dict); +- qdict_rename_keys(copy, renames, &error_abort); +- +- g_assert(!qdict_haskey(copy, "abc")); +- g_assert(!qdict_haskey(copy, "abcdef")); +- g_assert(!qdict_haskey(copy, "number")); +- g_assert(!qdict_haskey(copy, "flag")); +- g_assert(!qdict_haskey(copy, "nothing")); +- +- g_assert_cmpstr(qdict_get_str(copy, "str1"), ==, "foo"); +- g_assert_cmpstr(qdict_get_str(copy, "str2"), ==, "bar"); +- g_assert_cmpint(qdict_get_int(copy, "int"), ==, 42); +- g_assert_cmpint(qdict_get_bool(copy, "bool"), ==, true); +- g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); +- g_assert_cmpint(qdict_count_entries(copy), ==, 5); +- +- qobject_unref(copy); +- +- /* Renames are processed top to bottom */ +- renames = (QDictRenames[]) { +- { "abc", "tmp" }, +- { "abcdef", "abc" }, +- { "number", "abcdef" }, +- { "flag", "number" }, +- { "nothing", "flag" }, +- { "tmp", "nothing" }, +- { NULL , NULL } +- }; +- copy = qdict_clone_shallow(dict); +- qdict_rename_keys(copy, renames, &error_abort); +- +- g_assert_cmpstr(qdict_get_str(copy, "nothing"), ==, "foo"); +- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "bar"); +- g_assert_cmpint(qdict_get_int(copy, "abcdef"), ==, 42); +- g_assert_cmpint(qdict_get_bool(copy, "number"), ==, true); +- g_assert(qobject_type(qdict_get(copy, "flag")) == QTYPE_QNULL); +- g_assert(!qdict_haskey(copy, "tmp")); +- g_assert_cmpint(qdict_count_entries(copy), ==, 5); +- +- qobject_unref(copy); +- +- /* Conflicting rename */ +- renames = (QDictRenames[]) { +- { "abcdef", "abc" }, +- { NULL , NULL } +- }; +- copy = qdict_clone_shallow(dict); +- qdict_rename_keys(copy, renames, &local_err); +- +- g_assert(local_err != NULL); +- error_free(local_err); +- local_err = NULL; +- +- g_assert_cmpstr(qdict_get_str(copy, "abc"), ==, "foo"); +- g_assert_cmpstr(qdict_get_str(copy, "abcdef"), ==, "bar"); +- g_assert_cmpint(qdict_get_int(copy, "number"), ==, 42); +- g_assert_cmpint(qdict_get_bool(copy, "flag"), ==, true); +- g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); +- g_assert_cmpint(qdict_count_entries(copy), ==, 5); +- +- qobject_unref(copy); +- +- /* Renames in an empty dict */ +- renames = (QDictRenames[]) { +- { "abcdef", "abc" }, +- { NULL , NULL } +- }; +- +- qobject_unref(dict); +- dict = qdict_new(); +- +- qdict_rename_keys(dict, renames, &error_abort); +- g_assert(qdict_first(dict) == NULL); +- +- qobject_unref(dict); +-} +- +-static void qdict_crumple_test_bad_inputs(void) +-{ +- QDict *src; +- Error *error = NULL; +- +- src = qdict_new(); +- /* rule.0 can't be both a string and a dict */ +- qdict_put_str(src, "rule.0", "fred"); +- qdict_put_str(src, "rule.0.policy", "allow"); +- +- g_assert(qdict_crumple(src, &error) == NULL); +- g_assert(error != NULL); +- error_free(error); +- error = NULL; +- qobject_unref(src); +- +- src = qdict_new(); +- /* rule can't be both a list and a dict */ +- qdict_put_str(src, "rule.0", "fred"); +- qdict_put_str(src, "rule.a", "allow"); +- +- g_assert(qdict_crumple(src, &error) == NULL); +- g_assert(error != NULL); +- error_free(error); +- error = NULL; +- qobject_unref(src); +- +- src = qdict_new(); +- /* The input should be flat, ie no dicts or lists */ +- qdict_put(src, "rule.a", qdict_new()); +- qdict_put_str(src, "rule.b", "allow"); +- +- g_assert(qdict_crumple(src, &error) == NULL); +- g_assert(error != NULL); +- error_free(error); +- error = NULL; +- qobject_unref(src); +- +- src = qdict_new(); +- /* List indexes must not have gaps */ +- qdict_put_str(src, "rule.0", "deny"); +- qdict_put_str(src, "rule.3", "allow"); +- +- g_assert(qdict_crumple(src, &error) == NULL); +- g_assert(error != NULL); +- error_free(error); +- error = NULL; +- qobject_unref(src); +- +- src = qdict_new(); +- /* List indexes must be in %zu format */ +- qdict_put_str(src, "rule.0", "deny"); +- qdict_put_str(src, "rule.+1", "allow"); +- +- g_assert(qdict_crumple(src, &error) == NULL); +- g_assert(error != NULL); +- error_free(error); +- error = NULL; +- qobject_unref(src); +-} +- + /* + * Errors test-cases + */ +@@ -987,29 +359,15 @@ int main(int argc, char **argv) + g_test_add_func("/public/get_try_int", qdict_get_try_int_test); + g_test_add_func("/public/get_str", qdict_get_str_test); + g_test_add_func("/public/get_try_str", qdict_get_try_str_test); +- g_test_add_func("/public/defaults", qdict_defaults_test); + g_test_add_func("/public/haskey_not", qdict_haskey_not_test); + g_test_add_func("/public/haskey", qdict_haskey_test); + g_test_add_func("/public/del", qdict_del_test); + g_test_add_func("/public/to_qdict", qobject_to_qdict_test); + g_test_add_func("/public/iterapi", qdict_iterapi_test); +- g_test_add_func("/public/flatten", qdict_flatten_test); +- g_test_add_func("/public/array_split", qdict_array_split_test); +- g_test_add_func("/public/array_entries", qdict_array_entries_test); +- g_test_add_func("/public/join", qdict_join_test); + + g_test_add_func("/errors/put_exists", qdict_put_exists_test); + g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test); + +- g_test_add_func("/public/crumple/recursive", +- qdict_crumple_test_recursive); +- g_test_add_func("/public/crumple/empty", +- qdict_crumple_test_empty); +- g_test_add_func("/public/crumple/bad_inputs", +- qdict_crumple_test_bad_inputs); +- +- g_test_add_func("/public/rename_keys", qdict_rename_keys_test); +- + /* The Big one */ + if (g_test_slow()) { + g_test_add_func("/stress/test", qdict_stress_test); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qobject-Replace-qobject_incref-QINCREF-qobject_decre.patch b/SOURCES/kvm-qobject-Replace-qobject_incref-QINCREF-qobject_decre.patch new file mode 100644 index 0000000..feab285 --- /dev/null +++ b/SOURCES/kvm-qobject-Replace-qobject_incref-QINCREF-qobject_decre.patch @@ -0,0 +1,5252 @@ +From c01b425215024a64ae633ecfbc6c765bf87ad83e Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:11 +0200 +Subject: [PATCH 013/268] qobject: Replace qobject_incref/QINCREF + qobject_decref/QDECREF +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-5-armbru@redhat.com> +Patchwork-id: 80739 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 04/23] qobject: Replace qobject_incref/QINCREF qobject_decref/QDECREF +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +From: Marc-André Lureau + +Now that we can safely call QOBJECT() on QObject * as well as its +subtypes, we can have macros qobject_ref() / qobject_unref() that work +everywhere instead of having to use QINCREF() / QDECREF() for QObject +and qobject_incref() / qobject_decref() for its subtypes. + +The replacement is mechanical, except I broke a long line, and added a +cast in monitor_qmp_cleanup_req_queue_locked(). Unlike +qobject_decref(), qobject_unref() doesn't accept void *. + +Note that the new macros evaluate their argument exactly once, thus no +need to shout them. + +Signed-off-by: Marc-André Lureau +Reviewed-by: Eric Blake +Message-Id: <20180419150145.24795-4-marcandre.lureau@redhat.com> +Reviewed-by: Markus Armbruster +[Rebased, semantic conflict resolved, commit message improved] +Signed-off-by: Markus Armbruster +(cherry picked from commit cb3e7f08aeaab0ab13e629ce8496dca150a449ba) +[Trivial conflict in tests/migration-test.c resolved] + +Signed-off-by: Miroslav Rezanina +--- + block.c | 78 ++++++++++++++-------------- + block/blkdebug.c | 4 +- + block/blkverify.c | 4 +- + block/crypto.c | 4 +- + block/gluster.c | 4 +- + block/iscsi.c | 2 +- + block/nbd.c | 4 +- + block/nfs.c | 4 +- + block/null.c | 2 +- + block/nvme.c | 2 +- + block/parallels.c | 4 +- + block/qapi.c | 2 +- + block/qcow.c | 8 +-- + block/qcow2.c | 8 +-- + block/qed.c | 4 +- + block/quorum.c | 2 +- + block/rbd.c | 14 ++--- + block/sheepdog.c | 12 ++--- + block/snapshot.c | 4 +- + block/ssh.c | 4 +- + block/vdi.c | 2 +- + block/vhdx.c | 4 +- + block/vpc.c | 4 +- + block/vvfat.c | 2 +- + block/vxhs.c | 2 +- + blockdev.c | 16 +++--- + docs/devel/qapi-code-gen.txt | 2 +- + hw/i386/acpi-build.c | 12 ++--- + hw/ppc/spapr_drc.c | 2 +- + hw/usb/xen-usb.c | 4 +- + include/qapi/qmp/qnull.h | 2 +- + include/qapi/qmp/qobject.h | 40 +++++++-------- + migration/migration.c | 4 +- + migration/qjson.c | 2 +- + monitor.c | 50 +++++++++--------- + qapi/qapi-dealloc-visitor.c | 4 +- + qapi/qmp-dispatch.c | 6 +-- + qapi/qobject-input-visitor.c | 8 +-- + qapi/qobject-output-visitor.c | 8 +-- + qemu-img.c | 18 +++---- + qemu-io.c | 6 +-- + qga/main.c | 12 ++--- + qmp.c | 4 +- + qobject/json-parser.c | 10 ++-- + qobject/qdict.c | 38 +++++++------- + qobject/qjson.c | 2 +- + qobject/qlist.c | 4 +- + qom/object.c | 16 +++--- + qom/object_interfaces.c | 2 +- + scripts/coccinelle/qobject.cocci | 8 +-- + scripts/qapi/events.py | 2 +- + target/ppc/translate_init.c | 2 +- + target/s390x/cpu_models.c | 2 +- + tests/ahci-test.c | 6 +-- + tests/check-qdict.c | 100 ++++++++++++++++++------------------ + tests/check-qjson.c | 84 +++++++++++++++--------------- + tests/check-qlist.c | 8 +-- + tests/check-qlit.c | 10 ++-- + tests/check-qnull.c | 10 ++-- + tests/check-qnum.c | 28 +++++----- + tests/check-qobject.c | 2 +- + tests/check-qstring.c | 10 ++-- + tests/cpu-plug-test.c | 4 +- + tests/device-introspect-test.c | 24 ++++----- + tests/drive_del-test.c | 4 +- + tests/libqos/libqos.c | 8 +-- + tests/libqos/pci-pc.c | 2 +- + tests/libqtest.c | 24 ++++----- + tests/machine-none-test.c | 2 +- + tests/migration-test.c | 24 ++++----- + tests/numa-test.c | 16 +++--- + tests/pvpanic-test.c | 2 +- + tests/q35-test.c | 2 +- + tests/qmp-test.c | 38 +++++++------- + tests/qom-test.c | 8 +-- + tests/tco-test.c | 12 ++--- + tests/test-char.c | 2 +- + tests/test-keyval.c | 82 ++++++++++++++--------------- + tests/test-netfilter.c | 26 +++++----- + tests/test-qemu-opts.c | 14 ++--- + tests/test-qga.c | 76 +++++++++++++-------------- + tests/test-qmp-cmds.c | 24 ++++----- + tests/test-qmp-event.c | 2 +- + tests/test-qobject-input-visitor.c | 10 ++-- + tests/test-qobject-output-visitor.c | 18 +++---- + tests/test-visitor-serialization.c | 6 +-- + tests/test-x86-cpuid-compat.c | 14 ++--- + tests/tmp105-test.c | 4 +- + tests/vhost-user-test.c | 6 +-- + tests/virtio-net-test.c | 6 +-- + tests/vmgenid-test.c | 2 +- + tests/wdt_ib700-test.c | 14 ++--- + util/keyval.c | 12 ++--- + util/qemu-config.c | 4 +- + 94 files changed, 608 insertions(+), 612 deletions(-) + +diff --git a/block.c b/block.c +index a2caadf..55a7984 100644 +--- a/block.c ++++ b/block.c +@@ -1227,9 +1227,9 @@ BlockDriverState *bdrv_new_open_driver(BlockDriver *drv, const char *node_name, + + ret = bdrv_open_driver(bs, drv, node_name, bs->options, flags, errp); + if (ret < 0) { +- QDECREF(bs->explicit_options); ++ qobject_unref(bs->explicit_options); + bs->explicit_options = NULL; +- QDECREF(bs->options); ++ qobject_unref(bs->options); + bs->options = NULL; + bdrv_unref(bs); + return NULL; +@@ -1460,7 +1460,7 @@ static QDict *parse_json_filename(const char *filename, Error **errp) + + options = qobject_to(QDict, options_obj); + if (!options) { +- qobject_decref(options_obj); ++ qobject_unref(options_obj); + error_setg(errp, "Invalid JSON object given"); + return NULL; + } +@@ -1490,7 +1490,7 @@ static void parse_json_protocol(QDict *options, const char **pfilename, + /* Options given in the filename have lower priority than options + * specified directly */ + qdict_join(options, json_options, false); +- QDECREF(json_options); ++ qobject_unref(json_options); + *pfilename = NULL; + } + +@@ -2273,7 +2273,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options, + if (reference || qdict_haskey(options, "file.filename")) { + backing_filename[0] = '\0'; + } else if (bs->backing_file[0] == '\0' && qdict_size(options) == 0) { +- QDECREF(options); ++ qobject_unref(options); + goto free_exit; + } else { + bdrv_get_full_backing_filename(bs, backing_filename, PATH_MAX, +@@ -2281,7 +2281,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options, + if (local_err) { + ret = -EINVAL; + error_propagate(errp, local_err); +- QDECREF(options); ++ qobject_unref(options); + goto free_exit; + } + } +@@ -2289,7 +2289,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options, + if (!bs->drv || !bs->drv->supports_backing) { + ret = -EINVAL; + error_setg(errp, "Driver doesn't support backing files"); +- QDECREF(options); ++ qobject_unref(options); + goto free_exit; + } + +@@ -2323,7 +2323,7 @@ int bdrv_open_backing_file(BlockDriverState *bs, QDict *parent_options, + + free_exit: + g_free(backing_filename); +- QDECREF(tmp_parent_options); ++ qobject_unref(tmp_parent_options); + return ret; + } + +@@ -2356,7 +2356,7 @@ bdrv_open_child_bs(const char *filename, QDict *options, const char *bdref_key, + error_setg(errp, "A block device must be specified for \"%s\"", + bdref_key); + } +- QDECREF(image_options); ++ qobject_unref(image_options); + goto done; + } + +@@ -2449,7 +2449,7 @@ BlockDriverState *bdrv_open_blockdev_ref(BlockdevRef *ref, Error **errp) + obj = NULL; + + fail: +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); + return bs; + } +@@ -2519,7 +2519,7 @@ static BlockDriverState *bdrv_append_temp_snapshot(BlockDriverState *bs, + } + + out: +- QDECREF(snapshot_options); ++ qobject_unref(snapshot_options); + g_free(tmp_filename); + return bs_snapshot; + } +@@ -2530,7 +2530,7 @@ out: + * options is a QDict of options to pass to the block drivers, or NULL for an + * empty set of options. The reference to the QDict belongs to the block layer + * after the call (even on failure), so if the caller intends to reuse the +- * dictionary, it needs to use QINCREF() before calling bdrv_open. ++ * dictionary, it needs to use qobject_ref() before calling bdrv_open. + * + * If *pbs is NULL, a new BDS will be created with a pointer to it stored there. + * If it is not NULL, the referenced BDS will be reused. +@@ -2561,7 +2561,7 @@ static BlockDriverState *bdrv_open_inherit(const char *filename, + + if (reference) { + bool options_non_empty = options ? qdict_size(options) : false; +- QDECREF(options); ++ qobject_unref(options); + + if (filename || options_non_empty) { + error_setg(errp, "Cannot reference an existing block device with " +@@ -2752,7 +2752,7 @@ static BlockDriverState *bdrv_open_inherit(const char *filename, + + bdrv_parent_cb_change_media(bs, true); + +- QDECREF(options); ++ qobject_unref(options); + + /* For snapshot=on, create a temporary qcow2 overlay. bs points to the + * temporary snapshot afterwards. */ +@@ -2776,10 +2776,10 @@ static BlockDriverState *bdrv_open_inherit(const char *filename, + + fail: + blk_unref(file); +- QDECREF(snapshot_options); +- QDECREF(bs->explicit_options); +- QDECREF(bs->options); +- QDECREF(options); ++ qobject_unref(snapshot_options); ++ qobject_unref(bs->explicit_options); ++ qobject_unref(bs->options); ++ qobject_unref(options); + bs->options = NULL; + bs->explicit_options = NULL; + bdrv_unref(bs); +@@ -2788,8 +2788,8 @@ fail: + + close_and_fail: + bdrv_unref(bs); +- QDECREF(snapshot_options); +- QDECREF(options); ++ qobject_unref(snapshot_options); ++ qobject_unref(options); + error_propagate(errp, local_err); + return NULL; + } +@@ -2884,7 +2884,7 @@ static BlockReopenQueue *bdrv_reopen_queue_child(BlockReopenQueue *bs_queue, + old_options = qdict_clone_shallow(bs->explicit_options); + } + bdrv_join_options(bs, options, old_options); +- QDECREF(old_options); ++ qobject_unref(old_options); + + explicit_options = qdict_clone_shallow(options); + +@@ -2899,13 +2899,13 @@ static BlockReopenQueue *bdrv_reopen_queue_child(BlockReopenQueue *bs_queue, + qemu_opts_absorb_qdict(opts, options_copy, NULL); + update_flags_from_options(&flags, opts); + qemu_opts_del(opts); +- QDECREF(options_copy); ++ qobject_unref(options_copy); + } + + /* Old values are used for options that aren't set yet */ + old_options = qdict_clone_shallow(bs->options); + bdrv_join_options(bs, options, old_options); +- QDECREF(old_options); ++ qobject_unref(old_options); + + /* bdrv_open_inherit() sets and clears some additional flags internally */ + flags &= ~BDRV_O_PROTOCOL; +@@ -2917,8 +2917,8 @@ static BlockReopenQueue *bdrv_reopen_queue_child(BlockReopenQueue *bs_queue, + bs_entry = g_new0(BlockReopenQueueEntry, 1); + QSIMPLEQ_INSERT_TAIL(bs_queue, bs_entry, entry); + } else { +- QDECREF(bs_entry->state.options); +- QDECREF(bs_entry->state.explicit_options); ++ qobject_unref(bs_entry->state.options); ++ qobject_unref(bs_entry->state.explicit_options); + } + + bs_entry->state.bs = bs; +@@ -3008,9 +3008,9 @@ cleanup: + if (ret && bs_entry->prepared) { + bdrv_reopen_abort(&bs_entry->state); + } else if (ret) { +- QDECREF(bs_entry->state.explicit_options); ++ qobject_unref(bs_entry->state.explicit_options); + } +- QDECREF(bs_entry->state.options); ++ qobject_unref(bs_entry->state.options); + g_free(bs_entry); + } + g_free(bs_queue); +@@ -3253,7 +3253,7 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state) + } + + /* set BDS specific flags now */ +- QDECREF(bs->explicit_options); ++ qobject_unref(bs->explicit_options); + + bs->explicit_options = reopen_state->explicit_options; + bs->open_flags = reopen_state->flags; +@@ -3296,7 +3296,7 @@ void bdrv_reopen_abort(BDRVReopenState *reopen_state) + drv->bdrv_reopen_abort(reopen_state); + } + +- QDECREF(reopen_state->explicit_options); ++ qobject_unref(reopen_state->explicit_options); + + bdrv_abort_perm_update(reopen_state->bs); + } +@@ -3343,11 +3343,11 @@ static void bdrv_close(BlockDriverState *bs) + bs->total_sectors = 0; + bs->encrypted = false; + bs->sg = false; +- QDECREF(bs->options); +- QDECREF(bs->explicit_options); ++ qobject_unref(bs->options); ++ qobject_unref(bs->explicit_options); + bs->options = NULL; + bs->explicit_options = NULL; +- QDECREF(bs->full_open_options); ++ qobject_unref(bs->full_open_options); + bs->full_open_options = NULL; + + bdrv_release_named_dirty_bitmaps(bs); +@@ -5134,7 +5134,7 @@ static bool append_open_options(QDict *d, BlockDriverState *bs) + continue; + } + +- qobject_incref(qdict_entry_value(entry)); ++ qobject_ref(qdict_entry_value(entry)); + qdict_put_obj(d, qdict_entry_key(entry), qdict_entry_value(entry)); + found_any = true; + } +@@ -5174,21 +5174,21 @@ void bdrv_refresh_filename(BlockDriverState *bs) + * information before refreshing it */ + bs->exact_filename[0] = '\0'; + if (bs->full_open_options) { +- QDECREF(bs->full_open_options); ++ qobject_unref(bs->full_open_options); + bs->full_open_options = NULL; + } + + opts = qdict_new(); + append_open_options(opts, bs); + drv->bdrv_refresh_filename(bs, opts); +- QDECREF(opts); ++ qobject_unref(opts); + } else if (bs->file) { + /* Try to reconstruct valid information from the underlying file */ + bool has_open_options; + + bs->exact_filename[0] = '\0'; + if (bs->full_open_options) { +- QDECREF(bs->full_open_options); ++ qobject_unref(bs->full_open_options); + bs->full_open_options = NULL; + } + +@@ -5207,12 +5207,12 @@ void bdrv_refresh_filename(BlockDriverState *bs) + * suffices without querying the (exact_)filename of this BDS. */ + if (bs->file->bs->full_open_options) { + qdict_put_str(opts, "driver", drv->format_name); +- QINCREF(bs->file->bs->full_open_options); ++ qobject_ref(bs->file->bs->full_open_options); + qdict_put(opts, "file", bs->file->bs->full_open_options); + + bs->full_open_options = opts; + } else { +- QDECREF(opts); ++ qobject_unref(opts); + } + } else if (!bs->full_open_options && qdict_size(bs->options)) { + /* There is no underlying file BDS (at least referenced by BDS.file), +@@ -5246,7 +5246,7 @@ void bdrv_refresh_filename(BlockDriverState *bs) + QString *json = qobject_to_json(QOBJECT(bs->full_open_options)); + snprintf(bs->filename, sizeof(bs->filename), "json:%s", + qstring_get_str(json)); +- QDECREF(json); ++ qobject_unref(json); + } + } + +diff --git a/block/blkdebug.c b/block/blkdebug.c +index 5897124..689703d 100644 +--- a/block/blkdebug.c ++++ b/block/blkdebug.c +@@ -845,12 +845,12 @@ static void blkdebug_refresh_filename(BlockDriverState *bs, QDict *options) + opts = qdict_new(); + qdict_put_str(opts, "driver", "blkdebug"); + +- QINCREF(bs->file->bs->full_open_options); ++ qobject_ref(bs->file->bs->full_open_options); + qdict_put(opts, "image", bs->file->bs->full_open_options); + + for (e = qdict_first(options); e; e = qdict_next(options, e)) { + if (strcmp(qdict_entry_key(e), "x-image")) { +- qobject_incref(qdict_entry_value(e)); ++ qobject_ref(qdict_entry_value(e)); + qdict_put_obj(opts, qdict_entry_key(e), qdict_entry_value(e)); + } + } +diff --git a/block/blkverify.c b/block/blkverify.c +index 331365b..3cffcb1 100644 +--- a/block/blkverify.c ++++ b/block/blkverify.c +@@ -291,9 +291,9 @@ static void blkverify_refresh_filename(BlockDriverState *bs, QDict *options) + QDict *opts = qdict_new(); + qdict_put_str(opts, "driver", "blkverify"); + +- QINCREF(bs->file->bs->full_open_options); ++ qobject_ref(bs->file->bs->full_open_options); + qdict_put(opts, "raw", bs->file->bs->full_open_options); +- QINCREF(s->test_file->bs->full_open_options); ++ qobject_ref(s->test_file->bs->full_open_options); + qdict_put(opts, "test", s->test_file->bs->full_open_options); + + bs->full_open_options = opts; +diff --git a/block/crypto.c b/block/crypto.c +index bc6c7e3..7e7ad2d 100644 +--- a/block/crypto.c ++++ b/block/crypto.c +@@ -305,7 +305,7 @@ static int block_crypto_open_generic(QCryptoBlockFormat format, + + ret = 0; + cleanup: +- QDECREF(cryptoopts); ++ qobject_unref(cryptoopts); + qapi_free_QCryptoBlockOpenOptions(open_opts); + return ret; + } +@@ -635,7 +635,7 @@ static int coroutine_fn block_crypto_co_create_opts_luks(const char *filename, + fail: + bdrv_unref(bs); + qapi_free_QCryptoBlockCreateOptions(create_opts); +- QDECREF(cryptoopts); ++ qobject_unref(cryptoopts); + return ret; + } + +diff --git a/block/gluster.c b/block/gluster.c +index 4adc1a8..55be566 100644 +--- a/block/gluster.c ++++ b/block/gluster.c +@@ -650,7 +650,7 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster *gconf, + } + gsconf = NULL; + +- QDECREF(backing_options); ++ qobject_unref(backing_options); + backing_options = NULL; + g_free(str); + str = NULL; +@@ -663,7 +663,7 @@ out: + qapi_free_SocketAddress(gsconf); + qemu_opts_del(opts); + g_free(str); +- QDECREF(backing_options); ++ qobject_unref(backing_options); + errno = EINVAL; + return -errno; + } +diff --git a/block/iscsi.c b/block/iscsi.c +index f5aecfc..d19ae0e 100644 +--- a/block/iscsi.c ++++ b/block/iscsi.c +@@ -2143,7 +2143,7 @@ static int coroutine_fn iscsi_co_create_opts(const char *filename, QemuOpts *opt + } else { + ret = iscsi_open(bs, bs_options, 0, NULL); + } +- QDECREF(bs_options); ++ qobject_unref(bs_options); + + if (ret != 0) { + goto out; +diff --git a/block/nbd.c b/block/nbd.c +index 1e2b3ba..3e1693c 100644 +--- a/block/nbd.c ++++ b/block/nbd.c +@@ -293,8 +293,8 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options, + } + + done: +- QDECREF(addr); +- qobject_decref(crumpled_addr); ++ qobject_unref(addr); ++ qobject_unref(crumpled_addr); + visit_free(iv); + return saddr; + } +diff --git a/block/nfs.c b/block/nfs.c +index 2577df4..66fddf1 100644 +--- a/block/nfs.c ++++ b/block/nfs.c +@@ -567,7 +567,7 @@ static BlockdevOptionsNfs *nfs_options_qdict_to_qapi(QDict *options, + v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevOptionsNfs(v, NULL, &opts, &local_err); + visit_free(v); +- qobject_decref(crumpled); ++ qobject_unref(crumpled); + + if (local_err) { + return NULL; +@@ -683,7 +683,7 @@ static int coroutine_fn nfs_file_co_create_opts(const char *url, QemuOpts *opts, + + ret = 0; + out: +- QDECREF(options); ++ qobject_unref(options); + qapi_free_BlockdevCreateOptions(create_options); + return ret; + } +diff --git a/block/null.c b/block/null.c +index 806a863..700a2d0 100644 +--- a/block/null.c ++++ b/block/null.c +@@ -244,7 +244,7 @@ static int coroutine_fn null_co_block_status(BlockDriverState *bs, + + static void null_refresh_filename(BlockDriverState *bs, QDict *opts) + { +- QINCREF(opts); ++ qobject_ref(opts); + qdict_del(opts, "filename"); + + if (!qdict_size(opts)) { +diff --git a/block/nvme.c b/block/nvme.c +index c4f3a7b..e192da9 100644 +--- a/block/nvme.c ++++ b/block/nvme.c +@@ -1073,7 +1073,7 @@ static int nvme_reopen_prepare(BDRVReopenState *reopen_state, + + static void nvme_refresh_filename(BlockDriverState *bs, QDict *opts) + { +- QINCREF(opts); ++ qobject_ref(opts); + qdict_del(opts, "filename"); + + if (!qdict_size(opts)) { +diff --git a/block/parallels.c b/block/parallels.c +index 799215e..045810d 100644 +--- a/block/parallels.c ++++ b/block/parallels.c +@@ -651,7 +651,7 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple(qdict, errp); +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; +@@ -682,7 +682,7 @@ static int coroutine_fn parallels_co_create_opts(const char *filename, + ret = 0; + + done: +- QDECREF(qdict); ++ qobject_unref(qdict); + bdrv_unref(bs); + qapi_free_BlockdevCreateOptions(create_options); + return ret; +diff --git a/block/qapi.c b/block/qapi.c +index 04c6fc6..e12968f 100644 +--- a/block/qapi.c ++++ b/block/qapi.c +@@ -773,7 +773,7 @@ void bdrv_image_info_specific_dump(fprintf_function func_fprintf, void *f, + visit_complete(v, &obj); + data = qdict_get(qobject_to(QDict, obj), "data"); + dump_qobject(func_fprintf, f, 1, data); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); + } + +diff --git a/block/qcow.c b/block/qcow.c +index f928916..4b2f7db 100644 +--- a/block/qcow.c ++++ b/block/qcow.c +@@ -315,7 +315,7 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, + goto fail; + } + +- QDECREF(encryptopts); ++ qobject_unref(encryptopts); + qapi_free_QCryptoBlockOpenOptions(crypto_opts); + qemu_co_mutex_init(&s->lock); + return 0; +@@ -326,7 +326,7 @@ static int qcow_open(BlockDriverState *bs, QDict *options, int flags, + g_free(s->cluster_cache); + g_free(s->cluster_data); + qcrypto_block_free(s->crypto); +- QDECREF(encryptopts); ++ qobject_unref(encryptopts); + qapi_free_QCryptoBlockOpenOptions(crypto_opts); + return ret; + } +@@ -995,7 +995,7 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple(qdict, errp); +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; +@@ -1025,7 +1025,7 @@ static int coroutine_fn qcow_co_create_opts(const char *filename, + + ret = 0; + fail: +- QDECREF(qdict); ++ qobject_unref(qdict); + bdrv_unref(bs); + qapi_free_BlockdevCreateOptions(create_options); + return ret; +diff --git a/block/qcow2.c b/block/qcow2.c +index ef68772..2f36e63 100644 +--- a/block/qcow2.c ++++ b/block/qcow2.c +@@ -1063,7 +1063,7 @@ static int qcow2_update_options_prepare(BlockDriverState *bs, + + ret = 0; + fail: +- QDECREF(encryptopts); ++ qobject_unref(encryptopts); + qemu_opts_del(opts); + opts = NULL; + return ret; +@@ -2183,7 +2183,7 @@ static void coroutine_fn qcow2_co_invalidate_cache(BlockDriverState *bs, + qemu_co_mutex_lock(&s->lock); + ret = qcow2_do_open(bs, options, flags, &local_err); + qemu_co_mutex_unlock(&s->lock); +- QDECREF(options); ++ qobject_unref(options); + if (local_err) { + error_propagate(errp, local_err); + error_prepend(errp, "Could not reopen qcow2 layer: "); +@@ -3139,7 +3139,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + + /* Now get the QAPI type BlockdevCreateOptions */ + qobj = qdict_crumple(qdict, errp); +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; +@@ -3168,7 +3168,7 @@ static int coroutine_fn qcow2_co_create_opts(const char *filename, QemuOpts *opt + + ret = 0; + finish: +- QDECREF(qdict); ++ qobject_unref(qdict); + bdrv_unref(bs); + qapi_free_BlockdevCreateOptions(create_options); + return ret; +diff --git a/block/qed.c b/block/qed.c +index 35ff505..1db8eaf 100644 +--- a/block/qed.c ++++ b/block/qed.c +@@ -763,7 +763,7 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple(qdict, errp); +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; +@@ -789,7 +789,7 @@ static int coroutine_fn bdrv_qed_co_create_opts(const char *filename, + ret = bdrv_qed_co_create(create_options, errp); + + fail: +- QDECREF(qdict); ++ qobject_unref(qdict); + bdrv_unref(bs); + qapi_free_BlockdevCreateOptions(create_options); + return ret; +diff --git a/block/quorum.c b/block/quorum.c +index cfe484a..862cea3 100644 +--- a/block/quorum.c ++++ b/block/quorum.c +@@ -1082,7 +1082,7 @@ static void quorum_refresh_filename(BlockDriverState *bs, QDict *options) + + children = qlist_new(); + for (i = 0; i < s->num_children; i++) { +- QINCREF(s->children[i]->bs->full_open_options); ++ qobject_ref(s->children[i]->bs->full_open_options); + qlist_append(children, s->children[i]->bs->full_open_options); + } + +diff --git a/block/rbd.c b/block/rbd.c +index c9359d0..a14b42f 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -226,7 +226,7 @@ static void qemu_rbd_parse_filename(const char *filename, QDict *options, + + done: + g_free(buf); +- QDECREF(keypairs); ++ qobject_unref(keypairs); + return; + } + +@@ -275,17 +275,17 @@ static int qemu_rbd_set_keypairs(rados_t cluster, const char *keypairs_json, + key = qstring_get_str(name); + + ret = rados_conf_set(cluster, key, qstring_get_str(value)); +- QDECREF(value); ++ qobject_unref(value); + if (ret < 0) { + error_setg_errno(errp, -ret, "invalid conf option %s", key); +- QDECREF(name); ++ qobject_unref(name); + ret = -EINVAL; + break; + } +- QDECREF(name); ++ qobject_unref(name); + } + +- QDECREF(keypairs); ++ qobject_unref(keypairs); + return ret; + } + +@@ -449,7 +449,7 @@ static int coroutine_fn qemu_rbd_co_create_opts(const char *filename, + } + + exit: +- QDECREF(options); ++ qobject_unref(options); + qapi_free_BlockdevCreateOptions(create_options); + return ret; + } +@@ -664,7 +664,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevOptionsRbd(v, NULL, &opts, &local_err); + visit_free(v); +- qobject_decref(crumpled); ++ qobject_unref(crumpled); + + if (local_err) { + error_propagate(errp, local_err); +diff --git a/block/sheepdog.c b/block/sheepdog.c +index 387f59c..07529f4 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -567,8 +567,8 @@ static SocketAddress *sd_server_config(QDict *options, Error **errp) + + done: + visit_free(iv); +- qobject_decref(crumpled_server); +- QDECREF(server); ++ qobject_unref(crumpled_server); ++ qobject_unref(server); + return saddr; + } + +@@ -1883,7 +1883,7 @@ static int sd_create_prealloc(BlockdevOptionsSheepdog *location, int64_t size, + + if (local_err) { + error_propagate(errp, local_err); +- qobject_decref(obj); ++ qobject_unref(obj); + return -EINVAL; + } + +@@ -1901,7 +1901,7 @@ static int sd_create_prealloc(BlockdevOptionsSheepdog *location, int64_t size, + ret = sd_prealloc(bs, 0, size, errp); + fail: + bdrv_unref(bs); +- QDECREF(qdict); ++ qobject_unref(qdict); + return ret; + } + +@@ -2226,7 +2226,7 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + v = qobject_input_visitor_new_keyval(crumpled); + visit_type_BlockdevCreateOptions(v, NULL, &create_options, &local_err); + visit_free(v); +- qobject_decref(crumpled); ++ qobject_unref(crumpled); + + if (local_err) { + error_propagate(errp, local_err); +@@ -2252,7 +2252,7 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + ret = sd_co_create(create_options, errp); + fail: + qapi_free_BlockdevCreateOptions(create_options); +- QDECREF(qdict); ++ qobject_unref(qdict); + return ret; + } + +diff --git a/block/snapshot.c b/block/snapshot.c +index eacc1f1..2953d96 100644 +--- a/block/snapshot.c ++++ b/block/snapshot.c +@@ -214,7 +214,7 @@ int bdrv_snapshot_goto(BlockDriverState *bs, + bdrv_ref(file); + + qdict_extract_subqdict(options, &file_options, "file."); +- QDECREF(file_options); ++ qobject_unref(file_options); + qdict_put_str(options, "file", bdrv_get_node_name(file)); + + drv->bdrv_close(bs); +@@ -223,7 +223,7 @@ int bdrv_snapshot_goto(BlockDriverState *bs, + + ret = bdrv_snapshot_goto(file, snapshot_id, errp); + open_ret = drv->bdrv_open(bs, options, bs->open_flags, &local_err); +- QDECREF(options); ++ qobject_unref(options); + if (open_ret < 0) { + bdrv_unref(file); + bs->drv = NULL; +diff --git a/block/ssh.c b/block/ssh.c +index ab3acf0..412a1bf 100644 +--- a/block/ssh.c ++++ b/block/ssh.c +@@ -638,7 +638,7 @@ static BlockdevOptionsSsh *ssh_parse_options(QDict *options, Error **errp) + v = qobject_input_visitor_new(crumpled); + visit_type_BlockdevOptionsSsh(v, NULL, &result, &local_err); + visit_free(v); +- qobject_decref(crumpled); ++ qobject_unref(crumpled); + + if (local_err) { + error_propagate(errp, local_err); +@@ -917,7 +917,7 @@ static int coroutine_fn ssh_co_create_opts(const char *filename, QemuOpts *opts, + ret = ssh_co_create(create_options, errp); + + out: +- QDECREF(uri_options); ++ qobject_unref(uri_options); + qapi_free_BlockdevCreateOptions(create_options); + return ret; + } +diff --git a/block/vdi.c b/block/vdi.c +index 4a2d1ff..96a22b8 100644 +--- a/block/vdi.c ++++ b/block/vdi.c +@@ -951,7 +951,7 @@ static int coroutine_fn vdi_co_create_opts(const char *filename, QemuOpts *opts, + /* Create the vdi image (format layer) */ + ret = vdi_co_do_create(create_options, block_size, errp); + done: +- QDECREF(qdict); ++ qobject_unref(qdict); + qapi_free_BlockdevCreateOptions(create_options); + bdrv_unref(bs_file); + return ret; +diff --git a/block/vhdx.c b/block/vhdx.c +index 6ac0424..c3a4220 100644 +--- a/block/vhdx.c ++++ b/block/vhdx.c +@@ -2003,7 +2003,7 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple(qdict, errp); +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; +@@ -2049,7 +2049,7 @@ static int coroutine_fn vhdx_co_create_opts(const char *filename, + ret = vhdx_co_create(create_options, errp); + + fail: +- QDECREF(qdict); ++ qobject_unref(qdict); + bdrv_unref(bs); + qapi_free_BlockdevCreateOptions(create_options); + return ret; +diff --git a/block/vpc.c b/block/vpc.c +index 44f99a4..0ebfcd3 100644 +--- a/block/vpc.c ++++ b/block/vpc.c +@@ -1119,7 +1119,7 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + qdict_put_str(qdict, "file", bs->node_name); + + qobj = qdict_crumple(qdict, errp); +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qobject_to(QDict, qobj); + if (qdict == NULL) { + ret = -EINVAL; +@@ -1157,7 +1157,7 @@ static int coroutine_fn vpc_co_create_opts(const char *filename, + ret = vpc_co_create(create_options, errp); + + fail: +- QDECREF(qdict); ++ qobject_unref(qdict); + bdrv_unref(bs); + qapi_free_BlockdevCreateOptions(create_options); + return ret; +diff --git a/block/vvfat.c b/block/vvfat.c +index 1569783..662dca0 100644 +--- a/block/vvfat.c ++++ b/block/vvfat.c +@@ -3179,7 +3179,7 @@ static int enable_write_target(BlockDriverState *bs, Error **errp) + qdict_put_str(options, "write-target.driver", "qcow"); + s->qcow = bdrv_open_child(s->qcow_filename, options, "write-target", bs, + &child_vvfat_qcow, false, errp); +- QDECREF(options); ++ qobject_unref(options); + if (!s->qcow) { + ret = -EINVAL; + goto err; +diff --git a/block/vxhs.c b/block/vxhs.c +index 68edb51..96e83d9 100644 +--- a/block/vxhs.c ++++ b/block/vxhs.c +@@ -497,7 +497,7 @@ static int vxhs_open(BlockDriverState *bs, QDict *options, + + out: + g_free(of_vsa_addr); +- QDECREF(backing_options); ++ qobject_unref(backing_options); + qemu_opts_del(tcp_opts); + qemu_opts_del(opts); + g_free(cacert); +diff --git a/blockdev.c b/blockdev.c +index c31bf3d..3808b1f 100644 +--- a/blockdev.c ++++ b/blockdev.c +@@ -576,7 +576,7 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts, + blk_rs->read_only = read_only; + blk_rs->detect_zeroes = detect_zeroes; + +- QDECREF(bs_opts); ++ qobject_unref(bs_opts); + } else { + if (file && !*file) { + file = NULL; +@@ -632,16 +632,16 @@ static BlockBackend *blockdev_init(const char *file, QDict *bs_opts, + + err_no_bs_opts: + qemu_opts_del(opts); +- QDECREF(interval_dict); +- QDECREF(interval_list); ++ qobject_unref(interval_dict); ++ qobject_unref(interval_list); + return blk; + + early_err: + qemu_opts_del(opts); +- QDECREF(interval_dict); +- QDECREF(interval_list); ++ qobject_unref(interval_dict); ++ qobject_unref(interval_list); + err_no_opts: +- QDECREF(bs_opts); ++ qobject_unref(bs_opts); + return NULL; + } + +@@ -1130,7 +1130,7 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) + + fail: + qemu_opts_del(legacy_opts); +- QDECREF(bs_opts); ++ qobject_unref(bs_opts); + return dinfo; + } + +@@ -4022,7 +4022,7 @@ void hmp_drive_add_node(Monitor *mon, const char *optstr) + qdict = qemu_opts_to_qdict(opts, NULL); + + if (!qdict_get_try_str(qdict, "node-name")) { +- QDECREF(qdict); ++ qobject_unref(qdict); + error_report("'node-name' needs to be specified"); + goto out; + } +diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt +index a569d24..b9b6eab 100644 +--- a/docs/devel/qapi-code-gen.txt ++++ b/docs/devel/qapi-code-gen.txt +@@ -1340,7 +1340,7 @@ Example: + emit(EXAMPLE_QAPI_EVENT_MY_EVENT, qmp, &err); + + error_propagate(errp, err); +- QDECREF(qmp); ++ qobject_unref(qmp); + } + + const QEnumLookup example_QAPIEvent_lookup = { +diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c +index 976d151..b309a97 100644 +--- a/hw/i386/acpi-build.c ++++ b/hw/i386/acpi-build.c +@@ -201,21 +201,21 @@ static void acpi_get_pm_info(AcpiPmInfo *pm) + } else { + pm->s3_disabled = false; + } +- qobject_decref(o); ++ qobject_unref(o); + o = object_property_get_qobject(obj, ACPI_PM_PROP_S4_DISABLED, NULL); + if (o) { + pm->s4_disabled = qnum_get_uint(qobject_to(QNum, o)); + } else { + pm->s4_disabled = false; + } +- qobject_decref(o); ++ qobject_unref(o); + o = object_property_get_qobject(obj, ACPI_PM_PROP_S4_VAL, NULL); + if (o) { + pm->s4_val = qnum_get_uint(qobject_to(QNum, o)); + } else { + pm->s4_val = false; + } +- qobject_decref(o); ++ qobject_unref(o); + + pm->pcihp_bridge_en = + object_property_get_bool(obj, "acpi-pci-hotplug-with-bridge-support", +@@ -573,7 +573,7 @@ static void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus, + } + } + aml_append(parent_scope, method); +- qobject_decref(bsel); ++ qobject_unref(bsel); + } + + /** +@@ -2617,12 +2617,12 @@ static bool acpi_get_mcfg(AcpiMcfgInfo *mcfg) + return false; + } + mcfg->mcfg_base = qnum_get_uint(qobject_to(QNum, o)); +- qobject_decref(o); ++ qobject_unref(o); + + o = object_property_get_qobject(pci_host, PCIE_HOST_MCFG_SIZE, NULL); + assert(o); + mcfg->mcfg_size = qnum_get_uint(qobject_to(QNum, o)); +- qobject_decref(o); ++ qobject_unref(o); + return true; + } + +diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c +index aa25113..8a045d6 100644 +--- a/hw/ppc/spapr_drc.c ++++ b/hw/ppc/spapr_drc.c +@@ -305,7 +305,7 @@ static void prop_get_fdt(Object *obj, Visitor *v, const char *name, + + if (!drc->fdt) { + visit_type_null(v, NULL, &null, errp); +- QDECREF(null); ++ qobject_unref(null); + return; + } + +diff --git a/hw/usb/xen-usb.c b/hw/usb/xen-usb.c +index 3beeb0d..b3a90c0 100644 +--- a/hw/usb/xen-usb.c ++++ b/hw/usb/xen-usb.c +@@ -763,7 +763,7 @@ static void usbback_portid_add(struct usbback_info *usbif, unsigned port, + if (!usbif->ports[port - 1].dev) { + goto err; + } +- QDECREF(qdict); ++ qobject_unref(qdict); + speed = usbif->ports[port - 1].dev->speed; + switch (speed) { + case USB_SPEED_LOW: +@@ -796,7 +796,7 @@ static void usbback_portid_add(struct usbback_info *usbif, unsigned port, + return; + + err: +- QDECREF(qdict); ++ qobject_unref(qdict); + xen_pv_printf(&usbif->xendev, 0, "device %s could not be opened\n", busid); + } + +diff --git a/include/qapi/qmp/qnull.h b/include/qapi/qmp/qnull.h +index e8ea2c3..75b29c6 100644 +--- a/include/qapi/qmp/qnull.h ++++ b/include/qapi/qmp/qnull.h +@@ -23,7 +23,7 @@ extern QNull qnull_; + + static inline QNull *qnull(void) + { +- QINCREF(&qnull_); ++ qobject_ref(&qnull_); + return &qnull_; + } + +diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h +index a713c01..e20006f 100644 +--- a/include/qapi/qmp/qobject.h ++++ b/include/qapi/qmp/qobject.h +@@ -15,17 +15,17 @@ + * ------------------------------------ + * + * - Returning references: A function that returns an object may +- * return it as either a weak or a strong reference. If the reference +- * is strong, you are responsible for calling QDECREF() on the reference +- * when you are done. ++ * return it as either a weak or a strong reference. If the ++ * reference is strong, you are responsible for calling ++ * qobject_unref() on the reference when you are done. + * + * If the reference is weak, the owner of the reference may free it at + * any time in the future. Before storing the reference anywhere, you +- * should call QINCREF() to make the reference strong. ++ * should call qobject_ref() to make the reference strong. + * + * - Transferring ownership: when you transfer ownership of a reference + * by calling a function, you are no longer responsible for calling +- * QDECREF() when the reference is no longer needed. In other words, ++ * qobject_unref() when the reference is no longer needed. In other words, + * when the function returns you must behave as if the reference to the + * passed object was weak. + */ +@@ -50,14 +50,6 @@ struct QObject { + _obj ? container_of(&(_obj)->base, QObject, base) : NULL; \ + }) + +-/* High-level interface for qobject_incref() */ +-#define QINCREF(obj) \ +- qobject_incref(QOBJECT(obj)) +- +-/* High-level interface for qobject_decref() */ +-#define QDECREF(obj) \ +- qobject_decref(obj ? QOBJECT(obj) : NULL) +- + /* Required for qobject_to() */ + #define QTYPE_CAST_TO_QNull QTYPE_QNULL + #define QTYPE_CAST_TO_QNum QTYPE_QNUM +@@ -80,10 +72,7 @@ static inline void qobject_init(QObject *obj, QType type) + obj->base.type = type; + } + +-/** +- * qobject_incref(): Increment QObject's reference count +- */ +-static inline void qobject_incref(QObject *obj) ++static inline void qobject_ref_impl(QObject *obj) + { + if (obj) { + obj->base.refcnt++; +@@ -104,11 +93,7 @@ bool qobject_is_equal(const QObject *x, const QObject *y); + */ + void qobject_destroy(QObject *obj); + +-/** +- * qobject_decref(): Decrement QObject's reference count, deallocate +- * when it reaches zero +- */ +-static inline void qobject_decref(QObject *obj) ++static inline void qobject_unref_impl(QObject *obj) + { + assert(!obj || obj->base.refcnt); + if (obj && --obj->base.refcnt == 0) { +@@ -117,6 +102,17 @@ static inline void qobject_decref(QObject *obj) + } + + /** ++ * qobject_ref(): Increment QObject's reference count ++ */ ++#define qobject_ref(obj) qobject_ref_impl(QOBJECT(obj)) ++ ++/** ++ * qobject_unref(): Decrement QObject's reference count, deallocate ++ * when it reaches zero ++ */ ++#define qobject_unref(obj) qobject_unref_impl(QOBJECT(obj)) ++ ++/** + * qobject_type(): Return the QObject's type + */ + static inline QType qobject_type(const QObject *obj) +diff --git a/migration/migration.c b/migration/migration.c +index ceb1697..ef4bb42 100644 +--- a/migration/migration.c ++++ b/migration/migration.c +@@ -1008,14 +1008,14 @@ void qmp_migrate_set_parameters(MigrateSetParameters *params, Error **errp) + /* TODO Rewrite "" to null instead */ + if (params->has_tls_creds + && params->tls_creds->type == QTYPE_QNULL) { +- QDECREF(params->tls_creds->u.n); ++ qobject_unref(params->tls_creds->u.n); + params->tls_creds->type = QTYPE_QSTRING; + params->tls_creds->u.s = strdup(""); + } + /* TODO Rewrite "" to null instead */ + if (params->has_tls_hostname + && params->tls_hostname->type == QTYPE_QNULL) { +- QDECREF(params->tls_hostname->u.n); ++ qobject_unref(params->tls_hostname->u.n); + params->tls_hostname->type = QTYPE_QSTRING; + params->tls_hostname->u.s = strdup(""); + } +diff --git a/migration/qjson.c b/migration/qjson.c +index 9d7f6eb..e9889bd 100644 +--- a/migration/qjson.c ++++ b/migration/qjson.c +@@ -109,6 +109,6 @@ void qjson_finish(QJSON *json) + + void qjson_destroy(QJSON *json) + { +- QDECREF(json->str); ++ qobject_unref(json->str); + g_free(json); + } +diff --git a/monitor.c b/monitor.c +index 39f8ee1..4f43eee 100644 +--- a/monitor.c ++++ b/monitor.c +@@ -329,8 +329,8 @@ int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func, + + static void qmp_request_free(QMPRequest *req) + { +- qobject_decref(req->id); +- qobject_decref(req->req); ++ qobject_unref(req->id); ++ qobject_unref(req->req); + g_free(req); + } + +@@ -346,7 +346,7 @@ static void monitor_qmp_cleanup_req_queue_locked(Monitor *mon) + static void monitor_qmp_cleanup_resp_queue_locked(Monitor *mon) + { + while (!g_queue_is_empty(mon->qmp.qmp_responses)) { +- qobject_decref(g_queue_pop_head(mon->qmp.qmp_responses)); ++ qobject_unref((QObject *)g_queue_pop_head(mon->qmp.qmp_responses)); + } + } + +@@ -391,14 +391,14 @@ static void monitor_flush_locked(Monitor *mon) + rc = qemu_chr_fe_write(&mon->chr, (const uint8_t *) buf, len); + if ((rc < 0 && errno != EAGAIN) || (rc == len)) { + /* all flushed or error */ +- QDECREF(mon->outbuf); ++ qobject_unref(mon->outbuf); + mon->outbuf = qstring_new(); + return; + } + if (rc > 0) { + /* partial write */ + QString *tmp = qstring_from_str(buf + rc); +- QDECREF(mon->outbuf); ++ qobject_unref(mon->outbuf); + mon->outbuf = tmp; + } + if (mon->out_watch == 0) { +@@ -482,7 +482,7 @@ static void monitor_json_emitter_raw(Monitor *mon, + qstring_append_chr(json, '\n'); + monitor_puts(mon, qstring_get_str(json)); + +- QDECREF(json); ++ qobject_unref(json); + } + + static void monitor_json_emitter(Monitor *mon, QObject *data) +@@ -494,9 +494,9 @@ static void monitor_json_emitter(Monitor *mon, QObject *data) + * caller won't free the data (which will be finally freed in + * responder thread). + */ +- qobject_incref(data); ++ qobject_ref(data); + qemu_mutex_lock(&mon->qmp.qmp_queue_lock); +- g_queue_push_tail(mon->qmp.qmp_responses, (void *)data); ++ g_queue_push_tail(mon->qmp.qmp_responses, data); + qemu_mutex_unlock(&mon->qmp.qmp_queue_lock); + qemu_bh_schedule(mon_global.qmp_respond_bh); + } else { +@@ -546,7 +546,7 @@ static void monitor_qmp_bh_responder(void *opaque) + break; + } + monitor_json_emitter_raw(response.mon, response.data); +- qobject_decref(response.data); ++ qobject_unref(response.data); + } + } + +@@ -613,9 +613,9 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, Error **errp) + * last send. Store event for sending when timer fires, + * replacing a prior stored event if any. + */ +- QDECREF(evstate->qdict); ++ qobject_unref(evstate->qdict); + evstate->qdict = qdict; +- QINCREF(evstate->qdict); ++ qobject_ref(evstate->qdict); + } else { + /* + * Last send was (at least) evconf->rate ns ago. +@@ -630,7 +630,7 @@ monitor_qapi_event_queue(QAPIEvent event, QDict *qdict, Error **errp) + evstate = g_new(MonitorQAPIEventState, 1); + evstate->event = event; + evstate->data = data; +- QINCREF(evstate->data); ++ qobject_ref(evstate->data); + evstate->qdict = NULL; + evstate->timer = timer_new_ns(event_clock_type, + monitor_qapi_event_handler, +@@ -660,12 +660,12 @@ static void monitor_qapi_event_handler(void *opaque) + int64_t now = qemu_clock_get_ns(event_clock_type); + + monitor_qapi_event_emit(evstate->event, evstate->qdict); +- QDECREF(evstate->qdict); ++ qobject_unref(evstate->qdict); + evstate->qdict = NULL; + timer_mod_ns(evstate->timer, now + evconf->rate); + } else { + g_hash_table_remove(monitor_qapi_event_state, evstate); +- QDECREF(evstate->data); ++ qobject_unref(evstate->data); + timer_free(evstate->timer); + g_free(evstate); + } +@@ -747,7 +747,7 @@ static void monitor_data_destroy(Monitor *mon) + json_message_parser_destroy(&mon->qmp.parser); + } + readline_free(mon->rs); +- QDECREF(mon->outbuf); ++ qobject_unref(mon->outbuf); + qemu_mutex_destroy(&mon->out_lock); + qemu_mutex_destroy(&mon->qmp.qmp_queue_lock); + monitor_qmp_cleanup_req_queue_locked(mon); +@@ -3362,7 +3362,7 @@ static QDict *monitor_parse_arguments(Monitor *mon, + return qdict; + + fail: +- QDECREF(qdict); ++ qobject_unref(qdict); + g_free(key); + return NULL; + } +@@ -3387,7 +3387,7 @@ static void handle_hmp_command(Monitor *mon, const char *cmdline) + } + + cmd->cmd(mon, qdict); +- QDECREF(qdict); ++ qobject_unref(qdict); + } + + static void cmd_completion(Monitor *mon, const char *name, const char *list) +@@ -4049,15 +4049,15 @@ static void monitor_qmp_respond(Monitor *mon, QObject *rsp, + if (rsp) { + if (id) { + /* This is for the qdict below. */ +- qobject_incref(id); ++ qobject_ref(id); + qdict_put_obj(qobject_to(QDict, rsp), "id", id); + } + + monitor_json_emitter(mon, rsp); + } + +- qobject_decref(id); +- qobject_decref(rsp); ++ qobject_unref(id); ++ qobject_unref(rsp); + } + + /* +@@ -4080,7 +4080,7 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj) + if (trace_event_get_state_backends(TRACE_HANDLE_QMP_COMMAND)) { + QString *req_json = qobject_to_json(req); + trace_handle_qmp_command(mon, qstring_get_str(req_json)); +- QDECREF(req_json); ++ qobject_unref(req_json); + } + + old_mon = cur_mon; +@@ -4098,7 +4098,7 @@ static void monitor_qmp_dispatch_one(QMPRequest *req_obj) + monitor_resume(mon); + } + +- qobject_decref(req); ++ qobject_unref(req); + } + + /* +@@ -4190,7 +4190,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens) + goto err; + } + +- qobject_incref(id); ++ qobject_ref(id); + qdict_del(qdict, "id"); + + req_obj = g_new0(QMPRequest, 1); +@@ -4245,7 +4245,7 @@ static void handle_qmp_command(JSONMessageParser *parser, GQueue *tokens) + + err: + monitor_qmp_respond(mon, NULL, err, NULL); +- qobject_decref(req); ++ qobject_unref(req); + } + + static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size) +@@ -4364,7 +4364,7 @@ static void monitor_qmp_event(void *opaque, int event) + monitor_qmp_caps_reset(mon); + data = get_qmp_greeting(mon); + monitor_json_emitter(mon, data); +- qobject_decref(data); ++ qobject_unref(data); + mon_refcount++; + break; + case CHR_EVENT_CLOSED: +diff --git a/qapi/qapi-dealloc-visitor.c b/qapi/qapi-dealloc-visitor.c +index fd23803..6b24afd 100644 +--- a/qapi/qapi-dealloc-visitor.c ++++ b/qapi/qapi-dealloc-visitor.c +@@ -99,7 +99,7 @@ static void qapi_dealloc_type_anything(Visitor *v, const char *name, + QObject **obj, Error **errp) + { + if (obj) { +- qobject_decref(*obj); ++ qobject_unref(*obj); + } + } + +@@ -107,7 +107,7 @@ static void qapi_dealloc_type_null(Visitor *v, const char *name, + QNull **obj, Error **errp) + { + if (obj) { +- QDECREF(*obj); ++ qobject_unref(*obj); + } + } + +diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c +index dd05907..f9377b2 100644 +--- a/qapi/qmp-dispatch.c ++++ b/qapi/qmp-dispatch.c +@@ -105,7 +105,7 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request, + args = qdict_new(); + } else { + args = qdict_get_qdict(dict, "arguments"); +- QINCREF(args); ++ qobject_ref(args); + } + + cmd->fn(args, &ret, &local_err); +@@ -117,7 +117,7 @@ static QObject *do_qmp_dispatch(QmpCommandList *cmds, QObject *request, + ret = QOBJECT(qdict_new()); + } + +- QDECREF(args); ++ qobject_unref(args); + + return ret; + } +@@ -166,7 +166,7 @@ QObject *qmp_dispatch(QmpCommandList *cmds, QObject *request) + } else if (ret) { + qdict_put_obj(rsp, "return", ret); + } else { +- QDECREF(rsp); ++ qobject_unref(rsp); + return NULL; + } + +diff --git a/qapi/qobject-input-visitor.c b/qapi/qobject-input-visitor.c +index a7569d5..7a290c4 100644 +--- a/qapi/qobject-input-visitor.c ++++ b/qapi/qobject-input-visitor.c +@@ -588,7 +588,7 @@ static void qobject_input_type_any(Visitor *v, const char *name, QObject **obj, + return; + } + +- qobject_incref(qobj); ++ qobject_ref(qobj); + *obj = qobj; + } + +@@ -652,7 +652,7 @@ static void qobject_input_free(Visitor *v) + qobject_input_stack_object_free(tos); + } + +- qobject_decref(qiv->root); ++ qobject_unref(qiv->root); + if (qiv->errname) { + g_string_free(qiv->errname, TRUE); + } +@@ -678,7 +678,7 @@ static QObjectInputVisitor *qobject_input_visitor_base_new(QObject *obj) + v->visitor.free = qobject_input_free; + + v->root = obj; +- qobject_incref(obj); ++ qobject_ref(obj); + + return v; + } +@@ -744,7 +744,7 @@ Visitor *qobject_input_visitor_new_str(const char *str, + } + v = qobject_input_visitor_new_keyval(QOBJECT(args)); + } +- QDECREF(args); ++ qobject_unref(args); + + return v; + } +diff --git a/qapi/qobject-output-visitor.c b/qapi/qobject-output-visitor.c +index 877e37e..3a933b4 100644 +--- a/qapi/qobject-output-visitor.c ++++ b/qapi/qobject-output-visitor.c +@@ -188,7 +188,7 @@ static void qobject_output_type_any(Visitor *v, const char *name, + QObject **obj, Error **errp) + { + QObjectOutputVisitor *qov = to_qov(v); +- qobject_incref(*obj); ++ qobject_ref(*obj); + qobject_output_add_obj(qov, name, *obj); + } + +@@ -201,7 +201,7 @@ static void qobject_output_type_null(Visitor *v, const char *name, + + /* Finish building, and return the root object. + * The root object is never null. The caller becomes the object's +- * owner, and should use qobject_decref() when done with it. */ ++ * owner, and should use qobject_unref() when done with it. */ + static void qobject_output_complete(Visitor *v, void *opaque) + { + QObjectOutputVisitor *qov = to_qov(v); +@@ -210,7 +210,7 @@ static void qobject_output_complete(Visitor *v, void *opaque) + assert(qov->root && QSLIST_EMPTY(&qov->stack)); + assert(opaque == qov->result); + +- qobject_incref(qov->root); ++ qobject_ref(qov->root); + *qov->result = qov->root; + qov->result = NULL; + } +@@ -226,7 +226,7 @@ static void qobject_output_free(Visitor *v) + g_free(e); + } + +- qobject_decref(qov->root); ++ qobject_unref(qov->root); + g_free(qov); + } + +diff --git a/qemu-img.c b/qemu-img.c +index 8320887..62b29e7 100644 +--- a/qemu-img.c ++++ b/qemu-img.c +@@ -279,7 +279,7 @@ static BlockBackend *img_open_opts(const char *optstr, + if (qdict_haskey(options, BDRV_OPT_FORCE_SHARE) + && !qdict_get_bool(options, BDRV_OPT_FORCE_SHARE)) { + error_report("--force-share/-U conflicts with image options"); +- QDECREF(options); ++ qobject_unref(options); + return NULL; + } + qdict_put_bool(options, BDRV_OPT_FORCE_SHARE, true); +@@ -561,9 +561,9 @@ static void dump_json_image_check(ImageCheck *check, bool quiet) + str = qobject_to_json_pretty(obj); + assert(str != NULL); + qprintf(quiet, "%s\n", qstring_get_str(str)); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); +- QDECREF(str); ++ qobject_unref(str); + } + + static void dump_human_image_check(ImageCheck *check, bool quiet) +@@ -2384,9 +2384,9 @@ static void dump_json_image_info_list(ImageInfoList *list) + str = qobject_to_json_pretty(obj); + assert(str != NULL); + printf("%s\n", qstring_get_str(str)); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); +- QDECREF(str); ++ qobject_unref(str); + } + + static void dump_json_image_info(ImageInfo *info) +@@ -2400,9 +2400,9 @@ static void dump_json_image_info(ImageInfo *info) + str = qobject_to_json_pretty(obj); + assert(str != NULL); + printf("%s\n", qstring_get_str(str)); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); +- QDECREF(str); ++ qobject_unref(str); + } + + static void dump_human_image_info_list(ImageInfoList *list) +@@ -4488,9 +4488,9 @@ static void dump_json_block_measure_info(BlockMeasureInfo *info) + str = qobject_to_json_pretty(obj); + assert(str != NULL); + printf("%s\n", qstring_get_str(str)); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); +- QDECREF(str); ++ qobject_unref(str); + } + + static int img_measure(int argc, char **argv) +diff --git a/qemu-io.c b/qemu-io.c +index e692c55..72fee0d 100644 +--- a/qemu-io.c ++++ b/qemu-io.c +@@ -86,7 +86,7 @@ static int openfile(char *name, int flags, bool writethrough, bool force_share, + + if (qemuio_blk) { + error_report("file open already, try 'help close'"); +- QDECREF(opts); ++ qobject_unref(opts); + return 1; + } + +@@ -97,7 +97,7 @@ static int openfile(char *name, int flags, bool writethrough, bool force_share, + if (qdict_haskey(opts, BDRV_OPT_FORCE_SHARE) + && !qdict_get_bool(opts, BDRV_OPT_FORCE_SHARE)) { + error_report("-U conflicts with image options"); +- QDECREF(opts); ++ qobject_unref(opts); + return 1; + } + qdict_put_bool(opts, BDRV_OPT_FORCE_SHARE, true); +@@ -243,7 +243,7 @@ static int open_f(BlockBackend *blk, int argc, char **argv) + } else if (optind == argc) { + openfile(NULL, flags, writethrough, force_share, opts); + } else { +- QDECREF(opts); ++ qobject_unref(opts); + qemuio_command_usage(&open_cmd); + } + return 0; +diff --git a/qga/main.c b/qga/main.c +index df1888e..1e1cec7 100644 +--- a/qga/main.c ++++ b/qga/main.c +@@ -563,7 +563,7 @@ static int send_response(GAState *s, QObject *payload) + response_qstr = qstring_new(); + qstring_append_chr(response_qstr, QGA_SENTINEL_BYTE); + qstring_append(response_qstr, qstring_get_str(payload_qstr)); +- QDECREF(payload_qstr); ++ qobject_unref(payload_qstr); + } else { + response_qstr = payload_qstr; + } +@@ -571,7 +571,7 @@ static int send_response(GAState *s, QObject *payload) + qstring_append_chr(response_qstr, '\n'); + buf = qstring_get_str(response_qstr); + status = ga_channel_write_all(s->channel, buf, strlen(buf)); +- QDECREF(response_qstr); ++ qobject_unref(response_qstr); + if (status != G_IO_STATUS_NORMAL) { + return -EIO; + } +@@ -592,7 +592,7 @@ static void process_command(GAState *s, QDict *req) + if (ret < 0) { + g_warning("error sending response: %s", strerror(-ret)); + } +- qobject_decref(rsp); ++ qobject_unref(rsp); + } + } + +@@ -609,7 +609,7 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens) + g_debug("process_event: called"); + qdict = qobject_to(QDict, json_parser_parse_err(tokens, NULL, &err)); + if (err || !qdict) { +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qdict_new(); + if (!err) { + g_warning("failed to parse event: unknown error"); +@@ -626,7 +626,7 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens) + process_command(s, qdict); + } else { + if (!qdict_haskey(qdict, "error")) { +- QDECREF(qdict); ++ qobject_unref(qdict); + qdict = qdict_new(); + g_warning("unrecognized payload format"); + error_setg(&err, QERR_UNSUPPORTED); +@@ -639,7 +639,7 @@ static void process_event(JSONMessageParser *parser, GQueue *tokens) + } + } + +- QDECREF(qdict); ++ qobject_unref(qdict); + } + + /* false return signals GAChannel to close the current client connection */ +diff --git a/qmp.c b/qmp.c +index f722616..9e95b88 100644 +--- a/qmp.c ++++ b/qmp.c +@@ -710,7 +710,7 @@ void qmp_object_add(const char *type, const char *id, + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, "props", "dict"); + return; + } +- QINCREF(pdict); ++ qobject_ref(pdict); + } else { + pdict = qdict_new(); + } +@@ -721,7 +721,7 @@ void qmp_object_add(const char *type, const char *id, + if (obj) { + object_unref(obj); + } +- QDECREF(pdict); ++ qobject_unref(pdict); + } + + void qmp_object_del(const char *id, Error **errp) +diff --git a/qobject/json-parser.c b/qobject/json-parser.c +index 769b960..a5aa790 100644 +--- a/qobject/json-parser.c ++++ b/qobject/json-parser.c +@@ -222,7 +222,7 @@ static QString *qstring_from_escaped_str(JSONParserContext *ctxt, + return str; + + out: +- QDECREF(str); ++ qobject_unref(str); + return NULL; + } + +@@ -311,12 +311,12 @@ static int parse_pair(JSONParserContext *ctxt, QDict *dict, va_list *ap) + + qdict_put_obj(dict, qstring_get_str(key), value); + +- QDECREF(key); ++ qobject_unref(key); + + return 0; + + out: +- QDECREF(key); ++ qobject_unref(key); + + return -1; + } +@@ -371,7 +371,7 @@ static QObject *parse_object(JSONParserContext *ctxt, va_list *ap) + return QOBJECT(dict); + + out: +- QDECREF(dict); ++ qobject_unref(dict); + return NULL; + } + +@@ -435,7 +435,7 @@ static QObject *parse_array(JSONParserContext *ctxt, va_list *ap) + return QOBJECT(list); + + out: +- QDECREF(list); ++ qobject_unref(list); + return NULL; + } + +diff --git a/qobject/qdict.c b/qobject/qdict.c +index d1997a0..2e9bd53 100644 +--- a/qobject/qdict.c ++++ b/qobject/qdict.c +@@ -123,7 +123,7 @@ void qdict_put_obj(QDict *qdict, const char *key, QObject *value) + entry = qdict_find(qdict, key, bucket); + if (entry) { + /* replace key's value */ +- qobject_decref(entry->value); ++ qobject_unref(entry->value); + entry->value = value; + } else { + /* allocate a new entry */ +@@ -373,7 +373,7 @@ QDict *qdict_clone_shallow(const QDict *src) + + for (i = 0; i < QDICT_BUCKET_MAX; i++) { + QLIST_FOREACH(entry, &src->table[i], next) { +- qobject_incref(entry->value); ++ qobject_ref(entry->value); + qdict_put_obj(dest, entry->key, entry->value); + } + } +@@ -390,7 +390,7 @@ static void qentry_destroy(QDictEntry *e) + assert(e->key != NULL); + assert(e->value != NULL); + +- qobject_decref(e->value); ++ qobject_unref(e->value); + g_free(e->key); + g_free(e); + } +@@ -480,7 +480,7 @@ void qdict_copy_default(QDict *dst, QDict *src, const char *key) + + val = qdict_get(src, key); + if (val) { +- qobject_incref(val); ++ qobject_ref(val); + qdict_put_obj(dst, key, val); + } + } +@@ -526,7 +526,7 @@ static void qdict_flatten_qlist(QList *qlist, QDict *target, const char *prefix) + qdict_flatten_qlist(qobject_to(QList, value), target, new_key); + } else { + /* All other types are moved to the target unchanged. */ +- qobject_incref(value); ++ qobject_ref(value); + qdict_put_obj(target, new_key, value); + } + +@@ -566,7 +566,7 @@ static void qdict_flatten_qdict(QDict *qdict, QDict *target, const char *prefix) + delete = true; + } else if (prefix) { + /* All other objects are moved to the target unchanged. */ +- qobject_incref(value); ++ qobject_ref(value); + qdict_put_obj(target, new_key, value); + delete = true; + } +@@ -610,7 +610,7 @@ void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start) + while (entry != NULL) { + next = qdict_next(src, entry); + if (strstart(entry->key, start, &p)) { +- qobject_incref(entry->value); ++ qobject_ref(entry->value); + qdict_put_obj(*dst, p, entry->value); + qdict_del(src, entry->key); + } +@@ -684,7 +684,7 @@ void qdict_array_split(QDict *src, QList **dst) + qdict_extract_subqdict(src, &subqdict, prefix); + assert(qdict_size(subqdict) > 0); + } else { +- qobject_incref(subqobj); ++ qobject_ref(subqobj); + qdict_del(src, indexstr); + } + +@@ -894,7 +894,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + qdict_put_obj(two_level, prefix, QOBJECT(child_dict)); + } + +- qobject_incref(ent->value); ++ qobject_ref(ent->value); + qdict_put_obj(child_dict, suffix, ent->value); + } else { + if (child) { +@@ -902,7 +902,7 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + prefix); + goto error; + } +- qobject_incref(ent->value); ++ qobject_ref(ent->value); + qdict_put_obj(two_level, prefix, ent->value); + } + +@@ -924,11 +924,11 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + + qdict_put_obj(multi_level, ent->key, child); + } else { +- qobject_incref(ent->value); ++ qobject_ref(ent->value); + qdict_put_obj(multi_level, ent->key, ent->value); + } + } +- QDECREF(two_level); ++ qobject_unref(two_level); + two_level = NULL; + + /* Step 3: detect if we need to turn our dict into list */ +@@ -951,10 +951,10 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + goto error; + } + +- qobject_incref(child); ++ qobject_ref(child); + qlist_append_obj(qobject_to(QList, dst), child); + } +- QDECREF(multi_level); ++ qobject_unref(multi_level); + multi_level = NULL; + } else { + dst = QOBJECT(multi_level); +@@ -964,9 +964,9 @@ QObject *qdict_crumple(const QDict *src, Error **errp) + + error: + g_free(prefix); +- QDECREF(multi_level); +- QDECREF(two_level); +- qobject_decref(dst); ++ qobject_unref(multi_level); ++ qobject_unref(two_level); ++ qobject_unref(dst); + return NULL; + } + +@@ -1055,7 +1055,7 @@ void qdict_join(QDict *dest, QDict *src, bool overwrite) + next = qdict_next(src, entry); + + if (overwrite || !qdict_haskey(dest, entry->key)) { +- qobject_incref(entry->value); ++ qobject_ref(entry->value); + qdict_put_obj(dest, entry->key, entry->value); + qdict_del(src, entry->key); + } +@@ -1088,7 +1088,7 @@ bool qdict_rename_keys(QDict *qdict, const QDictRenames *renames, Error **errp) + } + + qobj = qdict_get(qdict, renames->from); +- qobject_incref(qobj); ++ qobject_ref(qobj); + qdict_put_obj(qdict, renames->to, qobj); + qdict_del(qdict, renames->from); + } +diff --git a/qobject/qjson.c b/qobject/qjson.c +index 655d38a..9816a65 100644 +--- a/qobject/qjson.c ++++ b/qobject/qjson.c +@@ -104,7 +104,7 @@ static void to_json_dict_iter(const char *key, QObject *obj, void *opaque) + + qkey = qstring_from_str(key); + to_json(QOBJECT(qkey), s->str, s->pretty, s->indent); +- QDECREF(qkey); ++ qobject_unref(qkey); + + qstring_append(s->str, ": "); + to_json(obj, s->str, s->pretty, s->indent); +diff --git a/qobject/qlist.c b/qobject/qlist.c +index 954fe98..37c1c16 100644 +--- a/qobject/qlist.c ++++ b/qobject/qlist.c +@@ -39,7 +39,7 @@ static void qlist_copy_elem(QObject *obj, void *opaque) + { + QList *dst = opaque; + +- qobject_incref(obj); ++ qobject_ref(obj); + qlist_append_obj(dst, obj); + } + +@@ -196,7 +196,7 @@ void qlist_destroy_obj(QObject *obj) + + QTAILQ_FOREACH_SAFE(entry, &qlist->head, next, next_entry) { + QTAILQ_REMOVE(&qlist->head, entry, next); +- qobject_decref(entry->value); ++ qobject_unref(entry->value); + g_free(entry); + } + +diff --git a/qom/object.c b/qom/object.c +index 4677951..76a89af 100644 +--- a/qom/object.c ++++ b/qom/object.c +@@ -1129,7 +1129,7 @@ void object_property_set_str(Object *obj, const char *value, + QString *qstr = qstring_from_str(value); + object_property_set_qobject(obj, QOBJECT(qstr), name, errp); + +- QDECREF(qstr); ++ qobject_unref(qstr); + } + + char *object_property_get_str(Object *obj, const char *name, +@@ -1147,7 +1147,7 @@ char *object_property_get_str(Object *obj, const char *name, + error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name, "string"); + } + +- qobject_decref(ret); ++ qobject_unref(ret); + return retval; + } + +@@ -1187,7 +1187,7 @@ void object_property_set_bool(Object *obj, bool value, + QBool *qbool = qbool_from_bool(value); + object_property_set_qobject(obj, QOBJECT(qbool), name, errp); + +- QDECREF(qbool); ++ qobject_unref(qbool); + } + + bool object_property_get_bool(Object *obj, const char *name, +@@ -1208,7 +1208,7 @@ bool object_property_get_bool(Object *obj, const char *name, + retval = qbool_get_bool(qbool); + } + +- qobject_decref(ret); ++ qobject_unref(ret); + return retval; + } + +@@ -1218,7 +1218,7 @@ void object_property_set_int(Object *obj, int64_t value, + QNum *qnum = qnum_from_int(value); + object_property_set_qobject(obj, QOBJECT(qnum), name, errp); + +- QDECREF(qnum); ++ qobject_unref(qnum); + } + + int64_t object_property_get_int(Object *obj, const char *name, +@@ -1238,7 +1238,7 @@ int64_t object_property_get_int(Object *obj, const char *name, + retval = -1; + } + +- qobject_decref(ret); ++ qobject_unref(ret); + return retval; + } + +@@ -1248,7 +1248,7 @@ void object_property_set_uint(Object *obj, uint64_t value, + QNum *qnum = qnum_from_uint(value); + + object_property_set_qobject(obj, QOBJECT(qnum), name, errp); +- QDECREF(qnum); ++ qobject_unref(qnum); + } + + uint64_t object_property_get_uint(Object *obj, const char *name, +@@ -1267,7 +1267,7 @@ uint64_t object_property_get_uint(Object *obj, const char *name, + retval = 0; + } + +- qobject_decref(ret); ++ qobject_unref(ret); + return retval; + } + +diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c +index 2f76e1f..980ffc2 100644 +--- a/qom/object_interfaces.c ++++ b/qom/object_interfaces.c +@@ -140,7 +140,7 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp) + qemu_opts_set_id(opts, (char *) id); + qemu_opt_set(opts, "qom-type", type, &error_abort); + g_free(type); +- QDECREF(pdict); ++ qobject_unref(pdict); + return obj; + } + +diff --git a/scripts/coccinelle/qobject.cocci b/scripts/coccinelle/qobject.cocci +index 47bcafe..9fee9c0 100644 +--- a/scripts/coccinelle/qobject.cocci ++++ b/scripts/coccinelle/qobject.cocci +@@ -3,11 +3,11 @@ + expression Obj, Key, E; + @@ + ( +-- qobject_incref(QOBJECT(E)); +-+ QINCREF(E); ++- qobject_ref(QOBJECT(E)); +++ qobject_ref(E); + | +-- qobject_decref(QOBJECT(E)); +-+ QDECREF(E); ++- qobject_unref(QOBJECT(E)); +++ qobject_unref(E); + | + - qdict_put_obj(Obj, Key, QOBJECT(E)); + + qdict_put(Obj, Key, E); +diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py +index 3dc523c..4426861 100644 +--- a/scripts/qapi/events.py ++++ b/scripts/qapi/events.py +@@ -142,7 +142,7 @@ out: + ''') + ret += mcgen(''' + error_propagate(errp, err); +- QDECREF(qmp); ++ qobject_unref(qmp); + } + ''') + return ret +diff --git a/target/ppc/translate_init.c b/target/ppc/translate_init.c +index 391b94b..17b06c7 100644 +--- a/target/ppc/translate_init.c ++++ b/target/ppc/translate_init.c +@@ -8351,7 +8351,7 @@ static void getset_compat_deprecated(Object *obj, Visitor *v, const char *name, + "use max-cpu-compat machine property instead"); + } + visit_type_null(v, name, &null, NULL); +- QDECREF(null); ++ qobject_unref(null); + } + + static const PropertyInfo ppc_compat_deprecated_propinfo = { +diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c +index 24e689c..0b5d271 100644 +--- a/target/s390x/cpu_models.c ++++ b/target/s390x/cpu_models.c +@@ -555,7 +555,7 @@ static void cpu_info_from_model(CpuModelInfo *info, const S390CPUModel *model, + } + + if (!qdict_size(qdict)) { +- QDECREF(qdict); ++ qobject_unref(qdict); + } else { + info->props = QOBJECT(qdict); + info->has_props = true; +diff --git a/tests/ahci-test.c b/tests/ahci-test.c +index fb3cd84..1a7b761 100644 +--- a/tests/ahci-test.c ++++ b/tests/ahci-test.c +@@ -1566,7 +1566,7 @@ static void atapi_wait_tray(bool open) + } else { + g_assert(!qdict_get_bool(data, "tray-open")); + } +- QDECREF(rsp); ++ qobject_unref(rsp); + } + + static void test_atapi_tray(void) +@@ -1596,7 +1596,7 @@ static void test_atapi_tray(void) + "'arguments': {'id': 'cd0'}}"); + atapi_wait_tray(true); + rsp = qmp_receive(); +- QDECREF(rsp); ++ qobject_unref(rsp); + + qmp_discard_response("{'execute': 'blockdev-remove-medium', " + "'arguments': {'id': 'cd0'}}"); +@@ -1623,7 +1623,7 @@ static void test_atapi_tray(void) + "'arguments': {'id': 'cd0'}}"); + atapi_wait_tray(false); + rsp = qmp_receive(); +- QDECREF(rsp); ++ qobject_unref(rsp); + + /* Now, to convince ATAPI we understand the media has changed... */ + ahci_atapi_test_ready(ahci, port, false, SENSE_NOT_READY); +diff --git a/tests/check-qdict.c b/tests/check-qdict.c +index 07bb8f4..eba5d35 100644 +--- a/tests/check-qdict.c ++++ b/tests/check-qdict.c +@@ -34,7 +34,7 @@ static void qdict_new_test(void) + g_assert(qdict->base.refcnt == 1); + g_assert(qobject_type(QOBJECT(qdict)) == QTYPE_QDICT); + +- QDECREF(qdict); ++ qobject_unref(qdict); + } + + static void qdict_put_obj_test(void) +@@ -54,7 +54,7 @@ static void qdict_put_obj_test(void) + qn = qobject_to(QNum, ent->value); + g_assert_cmpint(qnum_get_int(qn), ==, num); + +- QDECREF(qdict); ++ qobject_unref(qdict); + } + + static void qdict_destroy_simple_test(void) +@@ -65,7 +65,7 @@ static void qdict_destroy_simple_test(void) + qdict_put_int(qdict, "num", 0); + qdict_put_str(qdict, "str", "foo"); + +- QDECREF(qdict); ++ qobject_unref(qdict); + } + + static void qdict_get_test(void) +@@ -84,7 +84,7 @@ static void qdict_get_test(void) + qn = qobject_to(QNum, obj); + g_assert_cmpint(qnum_get_int(qn), ==, value); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_get_int_test(void) +@@ -99,7 +99,7 @@ static void qdict_get_int_test(void) + ret = qdict_get_int(tests_dict, key); + g_assert(ret == value); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_get_try_int_test(void) +@@ -121,7 +121,7 @@ static void qdict_get_try_int_test(void) + ret = qdict_get_try_int(tests_dict, "string", -42); + g_assert_cmpuint(ret, ==, -42); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_get_str_test(void) +@@ -137,7 +137,7 @@ static void qdict_get_str_test(void) + g_assert(p != NULL); + g_assert(strcmp(p, str) == 0); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_get_try_str_test(void) +@@ -153,7 +153,7 @@ static void qdict_get_try_str_test(void) + g_assert(p != NULL); + g_assert(strcmp(p, str) == 0); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_defaults_test(void) +@@ -174,8 +174,8 @@ static void qdict_defaults_test(void) + qdict_copy_default(copy, dict, "bar"); + g_assert_cmpstr(qdict_get_str(copy, "bar"), ==, "xyz"); + +- QDECREF(copy); +- QDECREF(dict); ++ qobject_unref(copy); ++ qobject_unref(dict); + } + + static void qdict_haskey_not_test(void) +@@ -183,7 +183,7 @@ static void qdict_haskey_not_test(void) + QDict *tests_dict = qdict_new(); + g_assert(qdict_haskey(tests_dict, "test") == 0); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_haskey_test(void) +@@ -194,7 +194,7 @@ static void qdict_haskey_test(void) + qdict_put_int(tests_dict, key, 0); + g_assert(qdict_haskey(tests_dict, key) == 1); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_del_test(void) +@@ -210,7 +210,7 @@ static void qdict_del_test(void) + g_assert(qdict_size(tests_dict) == 0); + g_assert(qdict_haskey(tests_dict, key) == 0); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qobject_to_qdict_test(void) +@@ -218,7 +218,7 @@ static void qobject_to_qdict_test(void) + QDict *tests_dict = qdict_new(); + g_assert(qobject_to(QDict, QOBJECT(tests_dict)) == tests_dict); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_iterapi_test(void) +@@ -250,7 +250,7 @@ static void qdict_iterapi_test(void) + + g_assert(count == qdict_size(tests_dict)); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_flatten_test(void) +@@ -325,7 +325,7 @@ static void qdict_flatten_test(void) + + g_assert(qdict_size(dict3) == 8); + +- QDECREF(dict3); ++ qobject_unref(dict3); + } + + static void qdict_array_split_test(void) +@@ -390,31 +390,31 @@ static void qdict_array_split_test(void) + g_assert(int1); + g_assert(qlist_empty(test_list)); + +- QDECREF(test_list); ++ qobject_unref(test_list); + + g_assert(qdict_get_int(dict1, "a") == 42); + g_assert(qdict_get_int(dict1, "b") == 23); + + g_assert(qdict_size(dict1) == 2); + +- QDECREF(dict1); ++ qobject_unref(dict1); + + g_assert(qdict_get_int(dict2, "x") == 0); + + g_assert(qdict_size(dict2) == 1); + +- QDECREF(dict2); ++ qobject_unref(dict2); + + g_assert_cmpint(qnum_get_int(int1), ==, 66); + +- QDECREF(int1); ++ qobject_unref(int1); + + g_assert(qdict_get_int(test_dict, "4.y") == 1); + g_assert(qdict_get_int(test_dict, "o.o") == 7); + + g_assert(qdict_size(test_dict) == 2); + +- QDECREF(test_dict); ++ qobject_unref(test_dict); + + /* + * Test the split of +@@ -455,18 +455,18 @@ static void qdict_array_split_test(void) + g_assert(int1); + g_assert(qlist_empty(test_list)); + +- QDECREF(test_list); ++ qobject_unref(test_list); + + g_assert_cmpint(qnum_get_int(int1), ==, 42); + +- QDECREF(int1); ++ qobject_unref(int1); + + g_assert(qdict_get_int(test_dict, "1") == 23); + g_assert(qdict_get_int(test_dict, "1.x") == 84); + + g_assert(qdict_size(test_dict) == 2); + +- QDECREF(test_dict); ++ qobject_unref(test_dict); + } + + static void qdict_array_entries_test(void) +@@ -493,7 +493,7 @@ static void qdict_array_entries_test(void) + g_assert_cmpint(qdict_array_entries(dict, "foo."), ==, 3); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, -EINVAL); + +- QDECREF(dict); ++ qobject_unref(dict); + + dict = qdict_new(); + qdict_put_int(dict, "1", 0); +@@ -509,7 +509,7 @@ static void qdict_array_entries_test(void) + qdict_put_int(dict, "2.c", 0); + g_assert_cmpint(qdict_array_entries(dict, ""), ==, 3); + +- QDECREF(dict); ++ qobject_unref(dict); + } + + static void qdict_join_test(void) +@@ -587,8 +587,8 @@ static void qdict_join_test(void) + } + while (overwrite ^= true); + +- QDECREF(dict1); +- QDECREF(dict2); ++ qobject_unref(dict1); ++ qobject_unref(dict2); + } + + static void qdict_crumple_test_recursive(void) +@@ -634,21 +634,21 @@ static void qdict_crumple_test_recursive(void) + g_assert_cmpint(qdict_size(rule), ==, 2); + g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match")); + g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy")); +- QDECREF(rule); ++ qobject_unref(rule); + + rule = qobject_to(QDict, qlist_pop(rules)); + g_assert(rule); + g_assert_cmpint(qdict_size(rule), ==, 2); + g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match")); + g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy")); +- QDECREF(rule); ++ qobject_unref(rule); + + /* With recursive crumpling, we should see all names unescaped */ + g_assert_cmpstr("acl0", ==, qdict_get_str(vnc, "acl.name")); + g_assert_cmpstr("acl0", ==, qdict_get_str(acl, "rule.name")); + +- QDECREF(src); +- QDECREF(dst); ++ qobject_unref(src); ++ qobject_unref(dst); + } + + static void qdict_crumple_test_empty(void) +@@ -661,8 +661,8 @@ static void qdict_crumple_test_empty(void) + + g_assert_cmpint(qdict_size(dst), ==, 0); + +- QDECREF(src); +- QDECREF(dst); ++ qobject_unref(src); ++ qobject_unref(dst); + } + + static int qdict_count_entries(QDict *dict) +@@ -704,7 +704,7 @@ static void qdict_rename_keys_test(void) + g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + +- QDECREF(copy); ++ qobject_unref(copy); + + /* Simple rename of all entries */ + renames = (QDictRenames[]) { +@@ -731,7 +731,7 @@ static void qdict_rename_keys_test(void) + g_assert(qobject_type(qdict_get(copy, "null")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + +- QDECREF(copy); ++ qobject_unref(copy); + + /* Renames are processed top to bottom */ + renames = (QDictRenames[]) { +@@ -754,7 +754,7 @@ static void qdict_rename_keys_test(void) + g_assert(!qdict_haskey(copy, "tmp")); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + +- QDECREF(copy); ++ qobject_unref(copy); + + /* Conflicting rename */ + renames = (QDictRenames[]) { +@@ -775,7 +775,7 @@ static void qdict_rename_keys_test(void) + g_assert(qobject_type(qdict_get(copy, "nothing")) == QTYPE_QNULL); + g_assert_cmpint(qdict_count_entries(copy), ==, 5); + +- QDECREF(copy); ++ qobject_unref(copy); + + /* Renames in an empty dict */ + renames = (QDictRenames[]) { +@@ -783,13 +783,13 @@ static void qdict_rename_keys_test(void) + { NULL , NULL } + }; + +- QDECREF(dict); ++ qobject_unref(dict); + dict = qdict_new(); + + qdict_rename_keys(dict, renames, &error_abort); + g_assert(qdict_first(dict) == NULL); + +- QDECREF(dict); ++ qobject_unref(dict); + } + + static void qdict_crumple_test_bad_inputs(void) +@@ -806,7 +806,7 @@ static void qdict_crumple_test_bad_inputs(void) + g_assert(error != NULL); + error_free(error); + error = NULL; +- QDECREF(src); ++ qobject_unref(src); + + src = qdict_new(); + /* rule can't be both a list and a dict */ +@@ -817,7 +817,7 @@ static void qdict_crumple_test_bad_inputs(void) + g_assert(error != NULL); + error_free(error); + error = NULL; +- QDECREF(src); ++ qobject_unref(src); + + src = qdict_new(); + /* The input should be flat, ie no dicts or lists */ +@@ -828,7 +828,7 @@ static void qdict_crumple_test_bad_inputs(void) + g_assert(error != NULL); + error_free(error); + error = NULL; +- QDECREF(src); ++ qobject_unref(src); + + src = qdict_new(); + /* List indexes must not have gaps */ +@@ -839,7 +839,7 @@ static void qdict_crumple_test_bad_inputs(void) + g_assert(error != NULL); + error_free(error); + error = NULL; +- QDECREF(src); ++ qobject_unref(src); + + src = qdict_new(); + /* List indexes must be in %zu format */ +@@ -850,7 +850,7 @@ static void qdict_crumple_test_bad_inputs(void) + g_assert(error != NULL); + error_free(error); + error = NULL; +- QDECREF(src); ++ qobject_unref(src); + } + + /* +@@ -871,7 +871,7 @@ static void qdict_put_exists_test(void) + + g_assert(qdict_size(tests_dict) == 1); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + static void qdict_get_not_exists_test(void) +@@ -879,7 +879,7 @@ static void qdict_get_not_exists_test(void) + QDict *tests_dict = qdict_new(); + g_assert(qdict_get(tests_dict, "foo") == NULL); + +- QDECREF(tests_dict); ++ qobject_unref(tests_dict); + } + + /* +@@ -951,7 +951,7 @@ static void qdict_stress_test(void) + + g_assert(strcmp(str1, str2) == 0); + +- QDECREF(value); ++ qobject_unref(value); + } + + // Delete everything +@@ -962,14 +962,14 @@ static void qdict_stress_test(void) + break; + + qdict_del(qdict, key); +- QDECREF(value); ++ qobject_unref(value); + + g_assert(qdict_haskey(qdict, key) == 0); + } + fclose(test_file); + + g_assert(qdict_size(qdict) == 0); +- QDECREF(qdict); ++ qobject_unref(qdict); + } + + int main(int argc, char **argv) +diff --git a/tests/check-qjson.c b/tests/check-qjson.c +index 997f4d3..da582df 100644 +--- a/tests/check-qjson.c ++++ b/tests/check-qjson.c +@@ -67,10 +67,10 @@ static void escaped_string(void) + if (test_cases[i].skip == 0) { + str = qobject_to_json(obj); + g_assert_cmpstr(qstring_get_str(str), ==, test_cases[i].encoded); +- qobject_decref(obj); ++ qobject_unref(obj); + } + +- QDECREF(str); ++ qobject_unref(str); + } + } + +@@ -99,9 +99,9 @@ static void simple_string(void) + str = qobject_to_json(obj); + g_assert(strcmp(qstring_get_str(str), test_cases[i].encoded) == 0); + +- qobject_decref(obj); ++ qobject_unref(obj); + +- QDECREF(str); ++ qobject_unref(str); + } + } + +@@ -127,7 +127,7 @@ static void single_quote_string(void) + g_assert(str); + g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0); + +- QDECREF(str); ++ qobject_unref(str); + } + } + +@@ -823,7 +823,7 @@ static void utf8_string(void) + } else { + g_assert(!obj); + } +- qobject_decref(obj); ++ qobject_unref(obj); + + obj = QOBJECT(qstring_from_str(utf8_in)); + str = qobject_to_json(obj); +@@ -833,8 +833,8 @@ static void utf8_string(void) + } else { + g_assert(!str); + } +- QDECREF(str); +- qobject_decref(obj); ++ qobject_unref(str); ++ qobject_unref(obj); + + /* + * Disabled, because qobject_from_json() is buggy, and I can't +@@ -869,7 +869,7 @@ static void vararg_string(void) + g_assert(str); + g_assert(strcmp(qstring_get_str(str), test_cases[i].decoded) == 0); + +- QDECREF(str); ++ qobject_unref(str); + } + } + +@@ -904,10 +904,10 @@ static void simple_number(void) + + str = qobject_to_json(QOBJECT(qnum)); + g_assert(strcmp(qstring_get_str(str), test_cases[i].encoded) == 0); +- QDECREF(str); ++ qobject_unref(str); + } + +- QDECREF(qnum); ++ qobject_unref(qnum); + } + } + +@@ -928,8 +928,8 @@ static void large_number(void) + + str = qobject_to_json(QOBJECT(qnum)); + g_assert_cmpstr(qstring_get_str(str), ==, maxu64); +- QDECREF(str); +- QDECREF(qnum); ++ qobject_unref(str); ++ qobject_unref(qnum); + + qnum = qobject_to(QNum, qobject_from_json(gtu64, &error_abort)); + g_assert(qnum); +@@ -939,8 +939,8 @@ static void large_number(void) + + str = qobject_to_json(QOBJECT(qnum)); + g_assert_cmpstr(qstring_get_str(str), ==, gtu64); +- QDECREF(str); +- QDECREF(qnum); ++ qobject_unref(str); ++ qobject_unref(qnum); + + qnum = qobject_to(QNum, qobject_from_json(lti64, &error_abort)); + g_assert(qnum); +@@ -950,8 +950,8 @@ static void large_number(void) + + str = qobject_to_json(QOBJECT(qnum)); + g_assert_cmpstr(qstring_get_str(str), ==, "-9223372036854775808"); +- QDECREF(str); +- QDECREF(qnum); ++ qobject_unref(str); ++ qobject_unref(qnum); + } + + static void float_number(void) +@@ -983,10 +983,10 @@ static void float_number(void) + + str = qobject_to_json(obj); + g_assert(strcmp(qstring_get_str(str), test_cases[i].encoded) == 0); +- QDECREF(str); ++ qobject_unref(str); + } + +- QDECREF(qnum); ++ qobject_unref(qnum); + } + } + +@@ -1001,16 +1001,16 @@ static void vararg_number(void) + qnum = qobject_to(QNum, qobject_from_jsonf("%d", value)); + g_assert(qnum_get_try_int(qnum, &val)); + g_assert_cmpint(val, ==, value); +- QDECREF(qnum); ++ qobject_unref(qnum); + + qnum = qobject_to(QNum, qobject_from_jsonf("%lld", value_ll)); + g_assert(qnum_get_try_int(qnum, &val)); + g_assert_cmpint(val, ==, value_ll); +- QDECREF(qnum); ++ qobject_unref(qnum); + + qnum = qobject_to(QNum, qobject_from_jsonf("%f", valuef)); + g_assert(qnum_get_double(qnum) == valuef); +- QDECREF(qnum); ++ qobject_unref(qnum); + } + + static void keyword_literal(void) +@@ -1027,9 +1027,9 @@ static void keyword_literal(void) + + str = qobject_to_json(obj); + g_assert(strcmp(qstring_get_str(str), "true") == 0); +- QDECREF(str); ++ qobject_unref(str); + +- QDECREF(qbool); ++ qobject_unref(qbool); + + obj = qobject_from_json("false", &error_abort); + qbool = qobject_to(QBool, obj); +@@ -1038,20 +1038,20 @@ static void keyword_literal(void) + + str = qobject_to_json(obj); + g_assert(strcmp(qstring_get_str(str), "false") == 0); +- QDECREF(str); ++ qobject_unref(str); + +- QDECREF(qbool); ++ qobject_unref(qbool); + + qbool = qobject_to(QBool, qobject_from_jsonf("%i", false)); + g_assert(qbool); + g_assert(qbool_get_bool(qbool) == false); +- QDECREF(qbool); ++ qobject_unref(qbool); + + /* Test that non-zero values other than 1 get collapsed to true */ + qbool = qobject_to(QBool, qobject_from_jsonf("%i", 2)); + g_assert(qbool); + g_assert(qbool_get_bool(qbool) == true); +- QDECREF(qbool); ++ qobject_unref(qbool); + + obj = qobject_from_json("null", &error_abort); + g_assert(obj != NULL); +@@ -1060,8 +1060,8 @@ static void keyword_literal(void) + null = qnull(); + g_assert(QOBJECT(null) == obj); + +- qobject_decref(obj); +- QDECREF(null); ++ qobject_unref(obj); ++ qobject_unref(null); + } + + static void simple_dict(void) +@@ -1101,12 +1101,12 @@ static void simple_dict(void) + g_assert(qlit_equal_qobject(&test_cases[i].decoded, obj)); + + str = qobject_to_json(obj); +- qobject_decref(obj); ++ qobject_unref(obj); + + obj = qobject_from_json(qstring_get_str(str), &error_abort); + g_assert(qlit_equal_qobject(&test_cases[i].decoded, obj)); +- qobject_decref(obj); +- QDECREF(str); ++ qobject_unref(obj); ++ qobject_unref(str); + } + } + +@@ -1158,7 +1158,7 @@ static void large_dict(void) + obj = qobject_from_json(gstr->str, &error_abort); + g_assert(obj != NULL); + +- qobject_decref(obj); ++ qobject_unref(obj); + g_string_free(gstr, true); + } + +@@ -1210,12 +1210,12 @@ static void simple_list(void) + g_assert(qlit_equal_qobject(&test_cases[i].decoded, obj)); + + str = qobject_to_json(obj); +- qobject_decref(obj); ++ qobject_unref(obj); + + obj = qobject_from_json(qstring_get_str(str), &error_abort); + g_assert(qlit_equal_qobject(&test_cases[i].decoded, obj)); +- qobject_decref(obj); +- QDECREF(str); ++ qobject_unref(obj); ++ qobject_unref(str); + } + } + +@@ -1272,13 +1272,13 @@ static void simple_whitespace(void) + g_assert(qlit_equal_qobject(&test_cases[i].decoded, obj)); + + str = qobject_to_json(obj); +- qobject_decref(obj); ++ qobject_unref(obj); + + obj = qobject_from_json(qstring_get_str(str), &error_abort); + g_assert(qlit_equal_qobject(&test_cases[i].decoded, obj)); + +- qobject_decref(obj); +- QDECREF(str); ++ qobject_unref(obj); ++ qobject_unref(str); + } + } + +@@ -1301,7 +1301,7 @@ static void simple_varargs(void) + obj = qobject_from_jsonf("[%d, 2, %p]", 1, embedded_obj); + g_assert(qlit_equal_qobject(&decoded, obj)); + +- qobject_decref(obj); ++ qobject_unref(obj); + } + + static void empty_input(void) +@@ -1410,7 +1410,7 @@ static void limits_nesting(void) + + obj = qobject_from_json(make_nest(buf, max_nesting), &error_abort); + g_assert(obj != NULL); +- qobject_decref(obj); ++ qobject_unref(obj); + + obj = qobject_from_json(make_nest(buf, max_nesting + 1), &err); + error_free_or_abort(&err); +diff --git a/tests/check-qlist.c b/tests/check-qlist.c +index a1c69ed..ece83e2 100644 +--- a/tests/check-qlist.c ++++ b/tests/check-qlist.c +@@ -29,7 +29,7 @@ static void qlist_new_test(void) + g_assert(qlist->base.refcnt == 1); + g_assert(qobject_type(QOBJECT(qlist)) == QTYPE_QLIST); + +- QDECREF(qlist); ++ qobject_unref(qlist); + } + + static void qlist_append_test(void) +@@ -47,7 +47,7 @@ static void qlist_append_test(void) + g_assert(entry != NULL); + g_assert(entry->value == QOBJECT(qi)); + +- QDECREF(qlist); ++ qobject_unref(qlist); + } + + static void qobject_to_qlist_test(void) +@@ -58,7 +58,7 @@ static void qobject_to_qlist_test(void) + + g_assert(qobject_to(QList, QOBJECT(qlist)) == qlist); + +- QDECREF(qlist); ++ qobject_unref(qlist); + } + + static int iter_called; +@@ -96,7 +96,7 @@ static void qlist_iter_test(void) + + g_assert(iter_called == iter_max); + +- QDECREF(qlist); ++ qobject_unref(qlist); + } + + int main(int argc, char **argv) +diff --git a/tests/check-qlit.c b/tests/check-qlit.c +index 96bbb06..bd6798d 100644 +--- a/tests/check-qlit.c ++++ b/tests/check-qlit.c +@@ -62,7 +62,7 @@ static void qlit_equal_qobject_test(void) + qdict_put(qobject_to(QDict, qobj), "bee", qlist_new()); + g_assert(!qlit_equal_qobject(&qlit, qobj)); + +- qobject_decref(qobj); ++ qobject_unref(qobj); + } + + static void qobject_from_qlit_test(void) +@@ -79,15 +79,15 @@ static void qobject_from_qlit_test(void) + bee = qdict_get_qlist(qdict, "bee"); + obj = qlist_pop(bee); + g_assert_cmpint(qnum_get_int(qobject_to(QNum, obj)), ==, 43); +- qobject_decref(obj); ++ qobject_unref(obj); + obj = qlist_pop(bee); + g_assert_cmpint(qnum_get_int(qobject_to(QNum, obj)), ==, 44); +- qobject_decref(obj); ++ qobject_unref(obj); + obj = qlist_pop(bee); + g_assert(qbool_get_bool(qobject_to(QBool, obj))); +- qobject_decref(obj); ++ qobject_unref(obj); + +- qobject_decref(qobj); ++ qobject_unref(qobj); + } + + int main(int argc, char **argv) +diff --git a/tests/check-qnull.c b/tests/check-qnull.c +index afa4400..ebf21db 100644 +--- a/tests/check-qnull.c ++++ b/tests/check-qnull.c +@@ -30,7 +30,7 @@ static void qnull_ref_test(void) + g_assert(obj == QOBJECT(&qnull_)); + g_assert(qnull_.base.refcnt == 2); + g_assert(qobject_type(obj) == QTYPE_QNULL); +- qobject_decref(obj); ++ qobject_unref(obj); + g_assert(qnull_.base.refcnt == 1); + } + +@@ -49,10 +49,10 @@ static void qnull_visit_test(void) + g_assert(qnull_.base.refcnt == 1); + obj = QOBJECT(qnull()); + v = qobject_input_visitor_new(obj); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_type_null(v, NULL, &null, &error_abort); + g_assert(obj == QOBJECT(&qnull_)); +- QDECREF(null); ++ qobject_unref(null); + visit_free(v); + + null = NULL; +@@ -60,8 +60,8 @@ static void qnull_visit_test(void) + visit_type_null(v, NULL, &null, &error_abort); + visit_complete(v, &obj); + g_assert(obj == QOBJECT(&qnull_)); +- QDECREF(null); +- qobject_decref(obj); ++ qobject_unref(null); ++ qobject_unref(obj); + visit_free(v); + + g_assert(qnull_.base.refcnt == 1); +diff --git a/tests/check-qnum.c b/tests/check-qnum.c +index 9187da7..4105015 100644 +--- a/tests/check-qnum.c ++++ b/tests/check-qnum.c +@@ -35,7 +35,7 @@ static void qnum_from_int_test(void) + g_assert_cmpint(qn->base.refcnt, ==, 1); + g_assert_cmpint(qobject_type(QOBJECT(qn)), ==, QTYPE_QNUM); + +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qnum_from_uint_test(void) +@@ -50,7 +50,7 @@ static void qnum_from_uint_test(void) + g_assert(qn->base.refcnt == 1); + g_assert(qobject_type(QOBJECT(qn)) == QTYPE_QNUM); + +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qnum_from_double_test(void) +@@ -65,7 +65,7 @@ static void qnum_from_double_test(void) + g_assert_cmpint(qn->base.refcnt, ==, 1); + g_assert_cmpint(qobject_type(QOBJECT(qn)), ==, QTYPE_QNUM); + +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qnum_from_int64_test(void) +@@ -76,7 +76,7 @@ static void qnum_from_int64_test(void) + qn = qnum_from_int(value); + g_assert_cmpint((int64_t) qn->u.i64, ==, value); + +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qnum_get_int_test(void) +@@ -87,7 +87,7 @@ static void qnum_get_int_test(void) + qn = qnum_from_int(value); + g_assert_cmpint(qnum_get_int(qn), ==, value); + +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qnum_get_uint_test(void) +@@ -100,25 +100,25 @@ static void qnum_get_uint_test(void) + qn = qnum_from_uint(value); + g_assert(qnum_get_try_uint(qn, &val)); + g_assert_cmpuint(val, ==, value); +- QDECREF(qn); ++ qobject_unref(qn); + + qn = qnum_from_int(value); + g_assert(qnum_get_try_uint(qn, &val)); + g_assert_cmpuint(val, ==, value); +- QDECREF(qn); ++ qobject_unref(qn); + + /* invalid cases */ + qn = qnum_from_int(-1); + g_assert(!qnum_get_try_uint(qn, &val)); +- QDECREF(qn); ++ qobject_unref(qn); + + qn = qnum_from_uint(-1ULL); + g_assert(!qnum_get_try_int(qn, &ival)); +- QDECREF(qn); ++ qobject_unref(qn); + + qn = qnum_from_double(0.42); + g_assert(!qnum_get_try_uint(qn, &val)); +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qobject_to_qnum_test(void) +@@ -127,11 +127,11 @@ static void qobject_to_qnum_test(void) + + qn = qnum_from_int(0); + g_assert(qobject_to(QNum, QOBJECT(qn)) == qn); +- QDECREF(qn); ++ qobject_unref(qn); + + qn = qnum_from_double(0); + g_assert(qobject_to(QNum, QOBJECT(qn)) == qn); +- QDECREF(qn); ++ qobject_unref(qn); + } + + static void qnum_to_string_test(void) +@@ -143,13 +143,13 @@ static void qnum_to_string_test(void) + tmp = qnum_to_string(qn); + g_assert_cmpstr(tmp, ==, "123456"); + g_free(tmp); +- QDECREF(qn); ++ qobject_unref(qn); + + qn = qnum_from_double(0.42); + tmp = qnum_to_string(qn); + g_assert_cmpstr(tmp, ==, "0.42"); + g_free(tmp); +- QDECREF(qn); ++ qobject_unref(qn); + } + + int main(int argc, char **argv) +diff --git a/tests/check-qobject.c b/tests/check-qobject.c +index 7629b80..5cb08fc 100644 +--- a/tests/check-qobject.c ++++ b/tests/check-qobject.c +@@ -80,7 +80,7 @@ static void do_free_all(int _, ...) + + va_start(ap, _); + while ((obj = va_arg(ap, QObject *)) != NULL) { +- qobject_decref(obj); ++ qobject_unref(obj); + } + va_end(ap); + } +diff --git a/tests/check-qstring.c b/tests/check-qstring.c +index 9c4dd3f..f11a7a8 100644 +--- a/tests/check-qstring.c ++++ b/tests/check-qstring.c +@@ -31,7 +31,7 @@ static void qstring_from_str_test(void) + g_assert(strcmp(str, qstring->string) == 0); + g_assert(qobject_type(QOBJECT(qstring)) == QTYPE_QSTRING); + +- QDECREF(qstring); ++ qobject_unref(qstring); + } + + static void qstring_get_str_test(void) +@@ -44,7 +44,7 @@ static void qstring_get_str_test(void) + ret_str = qstring_get_str(qstring); + g_assert(strcmp(ret_str, str) == 0); + +- QDECREF(qstring); ++ qobject_unref(qstring); + } + + static void qstring_append_chr_test(void) +@@ -59,7 +59,7 @@ static void qstring_append_chr_test(void) + qstring_append_chr(qstring, str[i]); + + g_assert(strcmp(str, qstring_get_str(qstring)) == 0); +- QDECREF(qstring); ++ qobject_unref(qstring); + } + + static void qstring_from_substr_test(void) +@@ -70,7 +70,7 @@ static void qstring_from_substr_test(void) + g_assert(qs != NULL); + g_assert(strcmp(qstring_get_str(qs), "tualiza") == 0); + +- QDECREF(qs); ++ qobject_unref(qs); + } + + +@@ -81,7 +81,7 @@ static void qobject_to_qstring_test(void) + qstring = qstring_from_str("foo"); + g_assert(qobject_to(QString, QOBJECT(qstring)) == qstring); + +- QDECREF(qstring); ++ qobject_unref(qstring); + } + + int main(int argc, char **argv) +diff --git a/tests/cpu-plug-test.c b/tests/cpu-plug-test.c +index 112869b..48b8d09 100644 +--- a/tests/cpu-plug-test.c ++++ b/tests/cpu-plug-test.c +@@ -42,7 +42,7 @@ static void test_plug_with_cpu_add(gconstpointer data) + " 'arguments': { 'id': %d } }", i); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + } + + qtest_end(); +@@ -66,7 +66,7 @@ static void test_plug_without_cpu_add(gconstpointer data) + s->sockets * s->cores * s->threads); + g_assert(response); + g_assert(qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + qtest_end(); + g_free(args); +diff --git a/tests/device-introspect-test.c b/tests/device-introspect-test.c +index a01321a..0b4f221 100644 +--- a/tests/device-introspect-test.c ++++ b/tests/device-introspect-test.c +@@ -40,8 +40,8 @@ static QList *qom_list_types(const char *implements, bool abstract) + " 'arguments': %p }", args); + g_assert(qdict_haskey(resp, "return")); + ret = qdict_get_qlist(resp, "return"); +- QINCREF(ret); +- QDECREF(resp); ++ qobject_ref(ret); ++ qobject_unref(resp); + return ret; + } + +@@ -54,7 +54,7 @@ static QDict *qom_type_index(QList *types) + QLIST_FOREACH_ENTRY(types, e) { + QDict *d = qobject_to(QDict, qlist_entry_obj(e)); + const char *name = qdict_get_str(d, "name"); +- QINCREF(d); ++ qobject_ref(d); + qdict_put(index, name, d); + } + return index; +@@ -108,7 +108,7 @@ static void test_one_device(const char *type) + resp = qmp("{'execute': 'device-list-properties'," + " 'arguments': {'typename': %s}}", + type); +- QDECREF(resp); ++ qobject_unref(resp); + + help = hmp("device_add \"%s,help\"", type); + g_free(help); +@@ -129,7 +129,7 @@ static void test_device_intro_list(void) + qtest_start(common_args); + + types = device_type_list(true); +- QDECREF(types); ++ qobject_unref(types); + + help = hmp("device_add help"); + g_free(help); +@@ -157,8 +157,8 @@ static void test_qom_list_parents(const char *parent) + g_assert(qom_has_parent(index, name, parent)); + } + +- QDECREF(types); +- QDECREF(index); ++ qobject_unref(types); ++ qobject_unref(index); + } + + static void test_qom_list_fields(void) +@@ -187,8 +187,8 @@ static void test_qom_list_fields(void) + test_qom_list_parents("device"); + test_qom_list_parents("sys-bus-device"); + +- QDECREF(all_types); +- QDECREF(non_abstract); ++ qobject_unref(all_types); ++ qobject_unref(non_abstract); + qtest_end(); + } + +@@ -222,7 +222,7 @@ static void test_device_intro_concrete(void) + test_one_device(type); + } + +- QDECREF(types); ++ qobject_unref(types); + qtest_end(); + } + +@@ -255,8 +255,8 @@ static void test_abstract_interfaces(void) + g_assert(qdict_haskey(d, "abstract") && qdict_get_bool(d, "abstract")); + } + +- QDECREF(all_types); +- QDECREF(index); ++ qobject_unref(all_types); ++ qobject_unref(index); + qtest_end(); + } + +diff --git a/tests/drive_del-test.c b/tests/drive_del-test.c +index 313030a..852fefc 100644 +--- a/tests/drive_del-test.c ++++ b/tests/drive_del-test.c +@@ -41,7 +41,7 @@ static void device_del(void) + response = qmp_receive(); + g_assert(response); + g_assert(qdict_haskey(response, "return")); +- QDECREF(response); ++ qobject_unref(response); + } + + static void test_drive_without_dev(void) +@@ -78,7 +78,7 @@ static void test_after_failed_device_add(void) + g_assert(response); + error = qdict_get_qdict(response, "error"); + g_assert_cmpstr(qdict_get_try_str(error, "class"), ==, "GenericError"); +- QDECREF(response); ++ qobject_unref(response); + + /* Delete the drive */ + drive_del(); +diff --git a/tests/libqos/libqos.c b/tests/libqos/libqos.c +index 5124e98..013ca68 100644 +--- a/tests/libqos/libqos.c ++++ b/tests/libqos/libqos.c +@@ -100,14 +100,14 @@ void migrate(QOSState *from, QOSState *to, const char *uri) + sub = qdict_get_qdict(rsp, "return"); + g_assert(qdict_haskey(sub, "running")); + running = qdict_get_bool(sub, "running"); +- QDECREF(rsp); ++ qobject_unref(rsp); + + /* Issue the migrate command. */ + rsp = qtest_qmp(from->qts, + "{ 'execute': 'migrate', 'arguments': { 'uri': %s }}", + uri); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + + /* Wait for STOP event, but only if we were running: */ + if (running) { +@@ -132,12 +132,12 @@ void migrate(QOSState *from, QOSState *to, const char *uri) + + /* "setup", "active", "completed", "failed", "cancelled" */ + if (strcmp(st, "completed") == 0) { +- QDECREF(rsp); ++ qobject_unref(rsp); + break; + } + + if ((strcmp(st, "setup") == 0) || (strcmp(st, "active") == 0)) { +- QDECREF(rsp); ++ qobject_unref(rsp); + g_usleep(5000); + continue; + } +diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c +index a2daf61..a780330 100644 +--- a/tests/libqos/pci-pc.c ++++ b/tests/libqos/pci-pc.c +@@ -170,7 +170,7 @@ void qpci_unplug_acpi_device_test(const char *id, uint8_t slot) + g_free(cmd); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + outb(ACPI_PCIHP_ADDR + PCI_EJ_BASE, 1 << slot); + +diff --git a/tests/libqtest.c b/tests/libqtest.c +index 6f33a37..43fb97e 100644 +--- a/tests/libqtest.c ++++ b/tests/libqtest.c +@@ -517,8 +517,8 @@ void qmp_fd_sendv(int fd, const char *fmt, va_list ap) + /* Send QMP request */ + socket_send(fd, str, qstring_get_length(qstr)); + +- QDECREF(qstr); +- qobject_decref(qobj); ++ qobject_unref(qstr); ++ qobject_unref(qobj); + } + } + +@@ -585,7 +585,7 @@ void qtest_async_qmp(QTestState *s, const char *fmt, ...) + void qtest_qmpv_discard_response(QTestState *s, const char *fmt, va_list ap) + { + QDict *response = qtest_qmpv(s, fmt, ap); +- QDECREF(response); ++ qobject_unref(response); + } + + void qtest_qmp_discard_response(QTestState *s, const char *fmt, ...) +@@ -596,7 +596,7 @@ void qtest_qmp_discard_response(QTestState *s, const char *fmt, ...) + va_start(ap, fmt); + response = qtest_qmpv(s, fmt, ap); + va_end(ap); +- QDECREF(response); ++ qobject_unref(response); + } + + QDict *qtest_qmp_eventwait_ref(QTestState *s, const char *event) +@@ -609,7 +609,7 @@ QDict *qtest_qmp_eventwait_ref(QTestState *s, const char *event) + (strcmp(qdict_get_str(response, "event"), event) == 0)) { + return response; + } +- QDECREF(response); ++ qobject_unref(response); + } + } + +@@ -618,7 +618,7 @@ void qtest_qmp_eventwait(QTestState *s, const char *event) + QDict *response; + + response = qtest_qmp_eventwait_ref(s, event); +- QDECREF(response); ++ qobject_unref(response); + } + + char *qtest_hmpv(QTestState *s, const char *fmt, va_list ap) +@@ -634,12 +634,12 @@ char *qtest_hmpv(QTestState *s, const char *fmt, va_list ap) + ret = g_strdup(qdict_get_try_str(resp, "return")); + while (ret == NULL && qdict_get_try_str(resp, "event")) { + /* Ignore asynchronous QMP events */ +- QDECREF(resp); ++ qobject_unref(resp); + resp = qtest_qmp_receive(s); + ret = g_strdup(qdict_get_try_str(resp, "return")); + } + g_assert(ret); +- QDECREF(resp); ++ qobject_unref(resp); + g_free(cmd); + return ret; + } +@@ -1021,7 +1021,7 @@ void qtest_cb_for_every_machine(void (*cb)(const char *machine)) + } + + qtest_end(); +- QDECREF(response); ++ qobject_unref(response); + } + + /* +@@ -1050,7 +1050,7 @@ void qtest_qmp_device_add(const char *driver, const char *id, const char *fmt, + g_assert(response); + g_assert(!qdict_haskey(response, "event")); /* We don't expect any events */ + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + } + + /* +@@ -1095,6 +1095,6 @@ void qtest_qmp_device_del(const char *id) + g_assert(event); + g_assert_cmpstr(qdict_get_str(event, "event"), ==, "DEVICE_DELETED"); + +- QDECREF(response1); +- QDECREF(response2); ++ qobject_unref(response1); ++ qobject_unref(response2); + } +diff --git a/tests/machine-none-test.c b/tests/machine-none-test.c +index efdd4be..f286557 100644 +--- a/tests/machine-none-test.c ++++ b/tests/machine-none-test.c +@@ -88,7 +88,7 @@ static void test_machine_cpu_cli(void) + + response = qmp("{ 'execute': 'quit' }"); + g_assert(qdict_haskey(response, "return")); +- QDECREF(response); ++ qobject_unref(response); + + qtest_quit(global_qtest); + } +diff --git a/tests/migration-test.c b/tests/migration-test.c +index 422bf1a..0acd715 100644 +--- a/tests/migration-test.c ++++ b/tests/migration-test.c +@@ -193,7 +193,7 @@ static QDict *wait_command(QTestState *who, const char *command) + if (!strcmp(event_string, "STOP")) { + got_stop = true; + } +- QDECREF(response); ++ qobject_unref(response); + response = qtest_qmp_receive(who); + } + return response; +@@ -219,7 +219,7 @@ static uint64_t get_migration_pass(QTestState *who) + rsp_ram = qdict_get_qdict(rsp_return, "ram"); + result = qdict_get_try_int(rsp_ram, "dirty-sync-count", 0); + } +- QDECREF(rsp); ++ qobject_unref(rsp); + return result; + } + +@@ -235,7 +235,7 @@ static void wait_for_migration_complete(QTestState *who) + status = qdict_get_str(rsp_return, "status"); + completed = strcmp(status, "completed") == 0; + g_assert_cmpstr(status, !=, "failed"); +- QDECREF(rsp); ++ qobject_unref(rsp); + if (completed) { + return; + } +@@ -322,7 +322,7 @@ static void migrate_check_parameter(QTestState *who, const char *parameter, + qdict_get_try_int(rsp_return, parameter, -1)); + g_assert_cmpstr(result, ==, value); + g_free(result); +- QDECREF(rsp); ++ qobject_unref(rsp); + } + + static void migrate_set_parameter(QTestState *who, const char *parameter, +@@ -337,7 +337,7 @@ static void migrate_set_parameter(QTestState *who, const char *parameter, + rsp = qtest_qmp(who, cmd); + g_free(cmd); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + migrate_check_parameter(who, parameter, value); + } + +@@ -355,7 +355,7 @@ static void migrate_set_capability(QTestState *who, const char *capability, + rsp = qtest_qmp(who, cmd); + g_free(cmd); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + } + + static void migrate(QTestState *who, const char *uri) +@@ -369,7 +369,7 @@ static void migrate(QTestState *who, const char *uri) + rsp = qtest_qmp(who, cmd); + g_free(cmd); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + } + + static void migrate_start_postcopy(QTestState *who) +@@ -378,7 +378,7 @@ static void migrate_start_postcopy(QTestState *who) + + rsp = wait_command(who, "{ 'execute': 'migrate-start-postcopy' }"); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + } + + static void test_migrate_start(QTestState **from, QTestState **to, +@@ -491,7 +491,7 @@ static void deprecated_set_downtime(QTestState *who, const double value) + rsp = qtest_qmp(who, cmd); + g_free(cmd); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + result_int = value * 1000L; + expected = g_strdup_printf("%" PRId64, result_int); + migrate_check_parameter(who, "downtime-limit", expected); +@@ -508,7 +508,7 @@ static void deprecated_set_speed(QTestState *who, const char *value) + rsp = qtest_qmp(who, cmd); + g_free(cmd); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + migrate_check_parameter(who, "max-bandwidth", value); + } + +@@ -581,7 +581,7 @@ static void test_baddest(void) + + g_assert(!strcmp(status, "setup") || !(strcmp(status, "failed"))); + failed = !strcmp(status, "failed"); +- QDECREF(rsp); ++ qobject_unref(rsp); + } while (!failed); + + /* Is the machine currently running? */ +@@ -590,7 +590,7 @@ static void test_baddest(void) + rsp_return = qdict_get_qdict(rsp, "return"); + g_assert(qdict_haskey(rsp_return, "running")); + g_assert(qdict_get_bool(rsp_return, "running")); +- QDECREF(rsp); ++ qobject_unref(rsp); + + test_migrate_end(from, to, false); + } +diff --git a/tests/numa-test.c b/tests/numa-test.c +index 0f861d8..169213f 100644 +--- a/tests/numa-test.c ++++ b/tests/numa-test.c +@@ -111,10 +111,10 @@ static void test_query_cpus(const void *data) + } else { + g_assert_cmpint(node, ==, 1); + } +- qobject_decref(e); ++ qobject_unref(e); + } + +- QDECREF(resp); ++ qobject_unref(resp); + qtest_end(); + g_free(cli); + } +@@ -164,10 +164,10 @@ static void pc_numa_cpu(const void *data) + } else { + g_assert(false); + } +- qobject_decref(e); ++ qobject_unref(e); + } + +- QDECREF(resp); ++ qobject_unref(resp); + qtest_end(); + g_free(cli); + } +@@ -209,10 +209,10 @@ static void spapr_numa_cpu(const void *data) + } else { + g_assert(false); + } +- qobject_decref(e); ++ qobject_unref(e); + } + +- QDECREF(resp); ++ qobject_unref(resp); + qtest_end(); + g_free(cli); + } +@@ -252,10 +252,10 @@ static void aarch64_numa_cpu(const void *data) + } else { + g_assert(false); + } +- qobject_decref(e); ++ qobject_unref(e); + } + +- QDECREF(resp); ++ qobject_unref(resp); + qtest_end(); + g_free(cli); + } +diff --git a/tests/pvpanic-test.c b/tests/pvpanic-test.c +index ebdf32c..7461a72 100644 +--- a/tests/pvpanic-test.c ++++ b/tests/pvpanic-test.c +@@ -28,7 +28,7 @@ static void test_panic(void) + data = qdict_get_qdict(response, "data"); + g_assert(qdict_haskey(data, "action")); + g_assert_cmpstr(qdict_get_str(data, "action"), ==, "pause"); +- QDECREF(response); ++ qobject_unref(response); + } + + int main(int argc, char **argv) +diff --git a/tests/q35-test.c b/tests/q35-test.c +index 3eaedf4..7ea7acc 100644 +--- a/tests/q35-test.c ++++ b/tests/q35-test.c +@@ -109,7 +109,7 @@ static void test_smram_lock(void) + response = qmp("{'execute': 'system_reset', 'arguments': {} }"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + /* check open is settable again */ + smram_set_bit(pcidev, MCH_HOST_BRIDGE_SMRAM_D_OPEN, false); +diff --git a/tests/qmp-test.c b/tests/qmp-test.c +index 772058f..88f867f 100644 +--- a/tests/qmp-test.c ++++ b/tests/qmp-test.c +@@ -52,27 +52,27 @@ static void test_malformed(QTestState *qts) + /* Not even a dictionary */ + resp = qtest_qmp(qts, "null"); + g_assert_cmpstr(get_error_class(resp), ==, "GenericError"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* No "execute" key */ + resp = qtest_qmp(qts, "{}"); + g_assert_cmpstr(get_error_class(resp), ==, "GenericError"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* "execute" isn't a string */ + resp = qtest_qmp(qts, "{ 'execute': true }"); + g_assert_cmpstr(get_error_class(resp), ==, "GenericError"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* "arguments" isn't a dictionary */ + resp = qtest_qmp(qts, "{ 'execute': 'no-such-cmd', 'arguments': [] }"); + g_assert_cmpstr(get_error_class(resp), ==, "GenericError"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* extra key */ + resp = qtest_qmp(qts, "{ 'execute': 'no-such-cmd', 'extra': true }"); + g_assert_cmpstr(get_error_class(resp), ==, "GenericError"); +- QDECREF(resp); ++ qobject_unref(resp); + } + + static void test_qmp_protocol(void) +@@ -90,12 +90,12 @@ static void test_qmp_protocol(void) + test_version(qdict_get(q, "version")); + capabilities = qdict_get_qlist(q, "capabilities"); + g_assert(capabilities && qlist_empty(capabilities)); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Test valid command before handshake */ + resp = qtest_qmp(qts, "{ 'execute': 'query-version' }"); + g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Test malformed commands before handshake */ + test_malformed(qts); +@@ -104,17 +104,17 @@ static void test_qmp_protocol(void) + resp = qtest_qmp(qts, "{ 'execute': 'qmp_capabilities' }"); + ret = qdict_get_qdict(resp, "return"); + g_assert(ret && !qdict_size(ret)); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Test repeated handshake */ + resp = qtest_qmp(qts, "{ 'execute': 'qmp_capabilities' }"); + g_assert_cmpstr(get_error_class(resp), ==, "CommandNotFound"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Test valid command */ + resp = qtest_qmp(qts, "{ 'execute': 'query-version' }"); + test_version(qdict_get(resp, "return")); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Test malformed commands */ + test_malformed(qts); +@@ -124,13 +124,13 @@ static void test_qmp_protocol(void) + ret = qdict_get_qdict(resp, "return"); + g_assert(ret); + g_assert_cmpstr(qdict_get_try_str(resp, "id"), ==, "cookie#1"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Test command failure with 'id' */ + resp = qtest_qmp(qts, "{ 'execute': 'human-monitor-command', 'id': 2 }"); + g_assert_cmpstr(get_error_class(resp), ==, "GenericError"); + g_assert_cmpint(qdict_get_int(resp, "id"), ==, 2); +- QDECREF(resp); ++ qobject_unref(resp); + + qtest_quit(qts); + } +@@ -159,21 +159,21 @@ static void test_qmp_oob(void) + qstr = qobject_to(QString, entry->value); + g_assert(qstr); + g_assert_cmpstr(qstring_get_str(qstr), ==, "oob"); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Try a fake capability, it should fail. */ + resp = qtest_qmp(qts, + "{ 'execute': 'qmp_capabilities', " + " 'arguments': { 'enable': [ 'cap-does-not-exist' ] } }"); + g_assert(qdict_haskey(resp, "error")); +- QDECREF(resp); ++ qobject_unref(resp); + + /* Now, enable OOB in current QMP session, it should succeed. */ + resp = qtest_qmp(qts, + "{ 'execute': 'qmp_capabilities', " + " 'arguments': { 'enable': [ 'oob' ] } }"); + g_assert(qdict_haskey(resp, "return")); +- QDECREF(resp); ++ qobject_unref(resp); + + /* + * Try any command that does not support OOB but with OOB flag. We +@@ -183,7 +183,7 @@ static void test_qmp_oob(void) + "{ 'execute': 'query-cpus'," + " 'control': { 'run-oob': true } }"); + g_assert(qdict_haskey(resp, "error")); +- QDECREF(resp); ++ qobject_unref(resp); + + /* + * First send the "x-oob-test" command with lock=true and +@@ -210,7 +210,7 @@ static void test_qmp_oob(void) + !g_strcmp0(cmd_id, "unlock-cmd")) { + acks++; + } +- QDECREF(resp); ++ qobject_unref(resp); + } + + qtest_quit(qts); +@@ -271,7 +271,7 @@ static void test_query(const void *data) + -1, &error_abort), + ==, expected_error_class); + } +- QDECREF(resp); ++ qobject_unref(resp); + + qtest_end(); + } +@@ -321,7 +321,7 @@ static void qmp_schema_init(QmpSchema *schema) + visit_type_SchemaInfoList(qiv, NULL, &schema->list, &error_abort); + visit_free(qiv); + +- QDECREF(resp); ++ qobject_unref(resp); + qtest_end(); + + schema->hash = g_hash_table_new(g_str_hash, g_str_equal); +diff --git a/tests/qom-test.c b/tests/qom-test.c +index 2fc2670..ebd15fd 100644 +--- a/tests/qom-test.c ++++ b/tests/qom-test.c +@@ -57,7 +57,7 @@ static void test_properties(const char *path, bool recurse) + g_assert(response); + + if (!recurse) { +- QDECREF(response); ++ qobject_unref(response); + return; + } + +@@ -82,10 +82,10 @@ static void test_properties(const char *path, bool recurse) + path, prop); + /* qom-get may fail but should not, e.g., segfault. */ + g_assert(tmp); +- QDECREF(tmp); ++ qobject_unref(tmp); + } + } +- QDECREF(response); ++ qobject_unref(response); + } + + static void test_machine(gconstpointer data) +@@ -101,7 +101,7 @@ static void test_machine(gconstpointer data) + + response = qmp("{ 'execute': 'quit' }"); + g_assert(qdict_haskey(response, "return")); +- QDECREF(response); ++ qobject_unref(response); + + qtest_end(); + g_free(args); +diff --git a/tests/tco-test.c b/tests/tco-test.c +index aee17af..9945fb8 100644 +--- a/tests/tco-test.c ++++ b/tests/tco-test.c +@@ -241,8 +241,8 @@ static QDict *get_watchdog_action(void) + QDict *data; + + data = qdict_get_qdict(ev, "data"); +- QINCREF(data); +- QDECREF(ev); ++ qobject_ref(data); ++ qobject_unref(ev); + return data; + } + +@@ -265,7 +265,7 @@ static void test_tco_second_timeout_pause(void) + clock_step(ticks * TCO_TICK_NSEC * 2); + ad = get_watchdog_action(); + g_assert(!strcmp(qdict_get_str(ad, "action"), "pause")); +- QDECREF(ad); ++ qobject_unref(ad); + + stop_tco(&td); + test_end(&td); +@@ -290,7 +290,7 @@ static void test_tco_second_timeout_reset(void) + clock_step(ticks * TCO_TICK_NSEC * 2); + ad = get_watchdog_action(); + g_assert(!strcmp(qdict_get_str(ad, "action"), "reset")); +- QDECREF(ad); ++ qobject_unref(ad); + + stop_tco(&td); + test_end(&td); +@@ -315,7 +315,7 @@ static void test_tco_second_timeout_shutdown(void) + clock_step(ticks * TCO_TICK_NSEC * 2); + ad = get_watchdog_action(); + g_assert(!strcmp(qdict_get_str(ad, "action"), "shutdown")); +- QDECREF(ad); ++ qobject_unref(ad); + + stop_tco(&td); + test_end(&td); +@@ -340,7 +340,7 @@ static void test_tco_second_timeout_none(void) + clock_step(ticks * TCO_TICK_NSEC * 2); + ad = get_watchdog_action(); + g_assert(!strcmp(qdict_get_str(ad, "action"), "none")); +- QDECREF(ad); ++ qobject_unref(ad); + + stop_tco(&td); + test_end(&td); +diff --git a/tests/test-char.c b/tests/test-char.c +index 306c728..1880d36 100644 +--- a/tests/test-char.c ++++ b/tests/test-char.c +@@ -322,7 +322,7 @@ static void char_socket_test_common(Chardev *chr) + qdict = qobject_to(QDict, addr); + port = qdict_get_str(qdict, "port"); + tmp = g_strdup_printf("tcp:127.0.0.1:%s", port); +- QDECREF(qdict); ++ qobject_unref(qdict); + + qemu_chr_fe_init(&be, chr, &error_abort); + qemu_chr_fe_set_handlers(&be, socket_can_read, socket_read, +diff --git a/tests/test-keyval.c b/tests/test-keyval.c +index 029f052..63cb146 100644 +--- a/tests/test-keyval.c ++++ b/tests/test-keyval.c +@@ -30,7 +30,7 @@ static void test_keyval_parse(void) + /* Nothing */ + qdict = keyval_parse("", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), ==, 0); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Empty key (qemu_opts_parse() accepts this) */ + qdict = keyval_parse("=val", NULL, &err); +@@ -70,7 +70,7 @@ static void test_keyval_parse(void) + qdict = keyval_parse(params + 2, NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, long_key + 1), ==, "v"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Long key fragment */ + qdict = keyval_parse(params, NULL, &error_abort); +@@ -79,7 +79,7 @@ static void test_keyval_parse(void) + g_assert(sub_qdict); + g_assert_cmpuint(qdict_size(sub_qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(sub_qdict, long_key + 1), ==, "v"); +- QDECREF(qdict); ++ qobject_unref(qdict); + g_free(params); + + /* Crap after valid key */ +@@ -92,13 +92,13 @@ static void test_keyval_parse(void) + g_assert_cmpuint(qdict_size(qdict), ==, 2); + g_assert_cmpstr(qdict_get_try_str(qdict, "a"), ==, "3"); + g_assert_cmpstr(qdict_get_try_str(qdict, "b"), ==, "2,x"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Even when it doesn't in qemu_opts_parse() */ + qdict = keyval_parse("id=foo,id=bar", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "id"), ==, "bar"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Dotted keys */ + qdict = keyval_parse("a.b.c=1,a.b.c=2,d=3", NULL, &error_abort); +@@ -111,7 +111,7 @@ static void test_keyval_parse(void) + g_assert_cmpuint(qdict_size(sub_qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(sub_qdict, "c"), ==, "2"); + g_assert_cmpstr(qdict_get_try_str(qdict, "d"), ==, "3"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Inconsistent dotted keys */ + qdict = keyval_parse("a.b=1,a=2", NULL, &err); +@@ -125,7 +125,7 @@ static void test_keyval_parse(void) + qdict = keyval_parse("x=y,", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "x"), ==, "y"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Except when it isn't */ + qdict = keyval_parse(",", NULL, &err); +@@ -136,13 +136,13 @@ static void test_keyval_parse(void) + qdict = keyval_parse("x=,,id=bar", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "x"), ==, ",id=bar"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Anti-social ID is left to caller (qemu_opts_parse() rejects it) */ + qdict = keyval_parse("id=666", NULL, &error_abort); + g_assert_cmpuint(qdict_size(qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(qdict, "id"), ==, "666"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Implied value not supported (unlike qemu_opts_parse()) */ + qdict = keyval_parse("an,noaus,noaus=", NULL, &err); +@@ -160,7 +160,7 @@ static void test_keyval_parse(void) + g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "an"); + g_assert_cmpstr(qdict_get_try_str(qdict, "aus"), ==, "off"); + g_assert_cmpstr(qdict_get_try_str(qdict, "noaus"), ==, ""); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Implied dotted key */ + qdict = keyval_parse("val", "eins.zwei", &error_abort); +@@ -169,7 +169,7 @@ static void test_keyval_parse(void) + g_assert(sub_qdict); + g_assert_cmpuint(qdict_size(sub_qdict), ==, 1); + g_assert_cmpstr(qdict_get_try_str(sub_qdict, "zwei"), ==, "val"); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Implied key with empty value (qemu_opts_parse() accepts this) */ + qdict = keyval_parse(",", "implied", &err); +@@ -198,7 +198,7 @@ static void check_list012(QList *qlist) + qstr = qobject_to(QString, qlist_pop(qlist)); + g_assert(qstr); + g_assert_cmpstr(qstring_get_str(qstr), ==, expected[i]); +- QDECREF(qstr); ++ qobject_unref(qstr); + } + g_assert(qlist_empty(qlist)); + } +@@ -218,14 +218,14 @@ static void test_keyval_parse_list(void) + NULL, &error_abort); + g_assert_cmpint(qdict_size(qdict), ==, 1); + check_list012(qdict_get_qlist(qdict, "list")); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Multiple indexes, last one wins */ + qdict = keyval_parse("list.1=goner,list.0=null,list.01=eins,list.2=zwei", + NULL, &error_abort); + g_assert_cmpint(qdict_size(qdict), ==, 1); + check_list012(qdict_get_qlist(qdict, "list")); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* List at deeper nesting */ + qdict = keyval_parse("a.list.1=eins,a.list.00=null,a.list.2=zwei", +@@ -234,7 +234,7 @@ static void test_keyval_parse_list(void) + sub_qdict = qdict_get_qdict(qdict, "a"); + g_assert_cmpint(qdict_size(sub_qdict), ==, 1); + check_list012(qdict_get_qlist(sub_qdict, "list")); +- QDECREF(qdict); ++ qobject_unref(qdict); + + /* Inconsistent dotted keys: both list and dictionary */ + qdict = keyval_parse("a.b.c=1,a.b.0=2", NULL, &err); +@@ -262,7 +262,7 @@ static void test_keyval_visit_bool(void) + + qdict = keyval_parse("bool1=on,bool2=off", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_bool(v, "bool1", &b, &error_abort); + g_assert(b); +@@ -274,7 +274,7 @@ static void test_keyval_visit_bool(void) + + qdict = keyval_parse("bool1=offer", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_bool(v, "bool1", &b, &err); + error_free_or_abort(&err); +@@ -292,7 +292,7 @@ static void test_keyval_visit_number(void) + /* Lower limit zero */ + qdict = keyval_parse("number1=0", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_uint64(v, "number1", &u, &error_abort); + g_assert_cmpuint(u, ==, 0); +@@ -304,7 +304,7 @@ static void test_keyval_visit_number(void) + qdict = keyval_parse("number1=18446744073709551615,number2=-1", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_uint64(v, "number1", &u, &error_abort); + g_assert_cmphex(u, ==, UINT64_MAX); +@@ -318,7 +318,7 @@ static void test_keyval_visit_number(void) + qdict = keyval_parse("number1=18446744073709551616", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_uint64(v, "number1", &u, &err); + error_free_or_abort(&err); +@@ -329,7 +329,7 @@ static void test_keyval_visit_number(void) + qdict = keyval_parse("number1=-18446744073709551616", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_uint64(v, "number1", &u, &err); + error_free_or_abort(&err); +@@ -340,7 +340,7 @@ static void test_keyval_visit_number(void) + qdict = keyval_parse("number1=0x2a,number2=052", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_uint64(v, "number1", &u, &error_abort); + g_assert_cmpuint(u, ==, 42); +@@ -354,7 +354,7 @@ static void test_keyval_visit_number(void) + qdict = keyval_parse("number1=3.14,number2=08", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_uint64(v, "number1", &u, &err); + error_free_or_abort(&err); +@@ -374,7 +374,7 @@ static void test_keyval_visit_size(void) + /* Lower limit zero */ + qdict = keyval_parse("sz1=0", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &error_abort); + g_assert_cmpuint(sz, ==, 0); +@@ -390,7 +390,7 @@ static void test_keyval_visit_size(void) + "sz3=9007199254740993", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &error_abort); + g_assert_cmphex(sz, ==, 0x1fffffffffffff); +@@ -407,7 +407,7 @@ static void test_keyval_visit_size(void) + "sz2=9223372036854775295", /* 7ffffffffffffdff */ + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &error_abort); + g_assert_cmphex(sz, ==, 0x7ffffffffffffc00); +@@ -422,7 +422,7 @@ static void test_keyval_visit_size(void) + "sz2=18446744073709550591", /* fffffffffffffbff */ + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &error_abort); + g_assert_cmphex(sz, ==, 0xfffffffffffff800); +@@ -437,7 +437,7 @@ static void test_keyval_visit_size(void) + "sz2=18446744073709550592", /* fffffffffffffc00 */ + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &err); + error_free_or_abort(&err); +@@ -450,7 +450,7 @@ static void test_keyval_visit_size(void) + qdict = keyval_parse("sz1=8b,sz2=1.5k,sz3=2M,sz4=0.1G,sz5=16777215T", + NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &error_abort); + g_assert_cmpuint(sz, ==, 8); +@@ -469,7 +469,7 @@ static void test_keyval_visit_size(void) + /* Beyond limit with suffix */ + qdict = keyval_parse("sz1=16777216T", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &err); + error_free_or_abort(&err); +@@ -479,7 +479,7 @@ static void test_keyval_visit_size(void) + /* Trailing crap */ + qdict = keyval_parse("sz1=16E,sz2=16Gi", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_size(v, "sz1", &sz, &err); + error_free_or_abort(&err); +@@ -498,7 +498,7 @@ static void test_keyval_visit_dict(void) + + qdict = keyval_parse("a.b.c=1,a.b.c=2,d=3", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_start_struct(v, "a", NULL, 0, &error_abort); + visit_start_struct(v, "b", NULL, 0, &error_abort); +@@ -516,7 +516,7 @@ static void test_keyval_visit_dict(void) + + qdict = keyval_parse("a.b=", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_start_struct(v, "a", NULL, 0, &error_abort); + visit_type_int(v, "c", &i, &err); /* a.c missing */ +@@ -539,7 +539,7 @@ static void test_keyval_visit_list(void) + qdict = keyval_parse("a.0=,a.1=I,a.2.0=II", NULL, &error_abort); + /* TODO empty list */ + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_start_list(v, "a", NULL, 0, &error_abort); + visit_type_str(v, NULL, &s, &error_abort); +@@ -562,7 +562,7 @@ static void test_keyval_visit_list(void) + + qdict = keyval_parse("a.0=,b.0.0=head", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_start_list(v, "a", NULL, 0, &error_abort); + visit_check_list(v, &err); /* a[0] unexpected */ +@@ -591,7 +591,7 @@ static void test_keyval_visit_optional(void) + + qdict = keyval_parse("a.b=1", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_optional(v, "b", &present); + g_assert(!present); /* b missing */ +@@ -627,7 +627,7 @@ static void test_keyval_visit_alternate(void) + */ + qdict = keyval_parse("a=1,b=2,c=on", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_AltStrObj(v, "a", &aso, &error_abort); + g_assert_cmpint(aso->type, ==, QTYPE_QSTRING); +@@ -651,19 +651,19 @@ static void test_keyval_visit_any(void) + + qdict = keyval_parse("a.0=null,a.1=1", NULL, &error_abort); + v = qobject_input_visitor_new_keyval(QOBJECT(qdict)); +- QDECREF(qdict); ++ qobject_unref(qdict); + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_any(v, "a", &any, &error_abort); + qlist = qobject_to(QList, any); + g_assert(qlist); + qstr = qobject_to(QString, qlist_pop(qlist)); + g_assert_cmpstr(qstring_get_str(qstr), ==, "null"); +- QDECREF(qstr); ++ qobject_unref(qstr); + qstr = qobject_to(QString, qlist_pop(qlist)); + g_assert_cmpstr(qstring_get_str(qstr), ==, "1"); + g_assert(qlist_empty(qlist)); +- QDECREF(qstr); +- qobject_decref(any); ++ qobject_unref(qstr); ++ qobject_unref(any); + visit_check_struct(v, &error_abort); + visit_end_struct(v, NULL); + visit_free(v); +diff --git a/tests/test-netfilter.c b/tests/test-netfilter.c +index 95f7839..e47075d 100644 +--- a/tests/test-netfilter.c ++++ b/tests/test-netfilter.c +@@ -29,7 +29,7 @@ static void add_one_netfilter(void) + + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'object-del'," + " 'arguments': {" +@@ -37,7 +37,7 @@ static void add_one_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + } + + /* add a netfilter to a netdev and then remove the netdev */ +@@ -57,7 +57,7 @@ static void remove_netdev_with_one_netfilter(void) + + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'netdev_del'," + " 'arguments': {" +@@ -65,7 +65,7 @@ static void remove_netdev_with_one_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + /* add back the netdev */ + response = qmp("{'execute': 'netdev_add'," +@@ -75,7 +75,7 @@ static void remove_netdev_with_one_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + } + + /* add multi(2) netfilters to a netdev and then remove them */ +@@ -95,7 +95,7 @@ static void add_multi_netfilter(void) + + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'object-add'," + " 'arguments': {" +@@ -109,7 +109,7 @@ static void add_multi_netfilter(void) + + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'object-del'," + " 'arguments': {" +@@ -117,7 +117,7 @@ static void add_multi_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'object-del'," + " 'arguments': {" +@@ -125,7 +125,7 @@ static void add_multi_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + } + + /* add multi(2) netfilters to a netdev and then remove the netdev */ +@@ -145,7 +145,7 @@ static void remove_netdev_with_multi_netfilter(void) + + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'object-add'," + " 'arguments': {" +@@ -159,7 +159,7 @@ static void remove_netdev_with_multi_netfilter(void) + + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + response = qmp("{'execute': 'netdev_del'," + " 'arguments': {" +@@ -167,7 +167,7 @@ static void remove_netdev_with_multi_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + + /* add back the netdev */ + response = qmp("{'execute': 'netdev_add'," +@@ -177,7 +177,7 @@ static void remove_netdev_with_multi_netfilter(void) + "}}"); + g_assert(response); + g_assert(!qdict_haskey(response, "error")); +- QDECREF(response); ++ qobject_unref(response); + } + + int main(int argc, char **argv) +diff --git a/tests/test-qemu-opts.c b/tests/test-qemu-opts.c +index 2c422ab..77dd72b 100644 +--- a/tests/test-qemu-opts.c ++++ b/tests/test-qemu-opts.c +@@ -887,7 +887,7 @@ static void test_opts_to_qdict_basic(void) + g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42"); + g_assert_false(qdict_haskey(dict, "number2")); + +- QDECREF(dict); ++ qobject_unref(dict); + qemu_opts_del(opts); + } + +@@ -914,7 +914,7 @@ static void test_opts_to_qdict_filtered(void) + g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42"); + g_assert_false(qdict_haskey(dict, "number2")); + g_assert_false(qdict_haskey(dict, "bool1")); +- QDECREF(dict); ++ qobject_unref(dict); + + dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_02, false); + g_assert(dict != NULL); +@@ -924,7 +924,7 @@ static void test_opts_to_qdict_filtered(void) + g_assert_false(qdict_haskey(dict, "str3")); + g_assert_false(qdict_haskey(dict, "number1")); + g_assert_false(qdict_haskey(dict, "number2")); +- QDECREF(dict); ++ qobject_unref(dict); + + /* Now delete converted options from opts */ + dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_01, true); +@@ -935,7 +935,7 @@ static void test_opts_to_qdict_filtered(void) + g_assert_cmpstr(qdict_get_str(dict, "number1"), ==, "42"); + g_assert_false(qdict_haskey(dict, "number2")); + g_assert_false(qdict_haskey(dict, "bool1")); +- QDECREF(dict); ++ qobject_unref(dict); + + dict = qemu_opts_to_qdict_filtered(opts, NULL, &opts_list_02, true); + g_assert(dict != NULL); +@@ -945,7 +945,7 @@ static void test_opts_to_qdict_filtered(void) + g_assert_false(qdict_haskey(dict, "str3")); + g_assert_false(qdict_haskey(dict, "number1")); + g_assert_false(qdict_haskey(dict, "number2")); +- QDECREF(dict); ++ qobject_unref(dict); + + g_assert_true(QTAILQ_EMPTY(&opts->head)); + +@@ -978,13 +978,13 @@ static void test_opts_to_qdict_duplicates(void) + dict = qemu_opts_to_qdict(opts, NULL); + g_assert(dict != NULL); + g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "b"); +- QDECREF(dict); ++ qobject_unref(dict); + + /* The last one still wins if entries are deleted, and both are deleted */ + dict = qemu_opts_to_qdict_filtered(opts, NULL, NULL, true); + g_assert(dict != NULL); + g_assert_cmpstr(qdict_get_str(dict, "foo"), ==, "b"); +- QDECREF(dict); ++ qobject_unref(dict); + + g_assert_true(QTAILQ_EMPTY(&opts->head)); + +diff --git a/tests/test-qga.c b/tests/test-qga.c +index e6ab788..18e63cb 100644 +--- a/tests/test-qga.c ++++ b/tests/test-qga.c +@@ -180,7 +180,7 @@ static void test_qga_sync_delimited(gconstpointer fix) + v = qdict_get_int(ret, "return"); + g_assert_cmpint(r, ==, v); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_sync(gconstpointer fix) +@@ -212,7 +212,7 @@ static void test_qga_sync(gconstpointer fix) + v = qdict_get_int(ret, "return"); + g_assert_cmpint(r, ==, v); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_ping(gconstpointer fix) +@@ -224,7 +224,7 @@ static void test_qga_ping(gconstpointer fix) + g_assert_nonnull(ret); + qmp_assert_no_error(ret); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_invalid_args(gconstpointer fix) +@@ -244,7 +244,7 @@ static void test_qga_invalid_args(gconstpointer fix) + g_assert_cmpstr(class, ==, "GenericError"); + g_assert_cmpstr(desc, ==, "Parameter 'foo' is unexpected"); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_invalid_cmd(gconstpointer fix) +@@ -263,7 +263,7 @@ static void test_qga_invalid_cmd(gconstpointer fix) + g_assert_cmpstr(class, ==, "CommandNotFound"); + g_assert_cmpint(strlen(desc), >, 0); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_info(gconstpointer fix) +@@ -280,7 +280,7 @@ static void test_qga_info(gconstpointer fix) + version = qdict_get_try_str(val, "version"); + g_assert_cmpstr(version, ==, QEMU_VERSION); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_get_vcpus(gconstpointer fix) +@@ -300,7 +300,7 @@ static void test_qga_get_vcpus(gconstpointer fix) + g_assert(qdict_haskey(qobject_to(QDict, entry->value), "online")); + g_assert(qdict_haskey(qobject_to(QDict, entry->value), "logical-id")); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_get_fsinfo(gconstpointer fix) +@@ -324,7 +324,7 @@ static void test_qga_get_fsinfo(gconstpointer fix) + g_assert(qdict_haskey(qobject_to(QDict, entry->value), "disk")); + } + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_get_memory_block_info(gconstpointer fix) +@@ -344,7 +344,7 @@ static void test_qga_get_memory_block_info(gconstpointer fix) + g_assert_cmpint(size, >, 0); + } + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_get_memory_blocks(gconstpointer fix) +@@ -369,7 +369,7 @@ static void test_qga_get_memory_blocks(gconstpointer fix) + } + } + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_network_get_interfaces(gconstpointer fix) +@@ -388,7 +388,7 @@ static void test_qga_network_get_interfaces(gconstpointer fix) + entry = qlist_first(list); + g_assert(qdict_haskey(qobject_to(QDict, entry->value), "name")); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_file_ops(gconstpointer fix) +@@ -410,7 +410,7 @@ static void test_qga_file_ops(gconstpointer fix) + g_assert_nonnull(ret); + qmp_assert_no_error(ret); + id = qdict_get_int(ret, "return"); +- QDECREF(ret); ++ qobject_unref(ret); + + enc = g_base64_encode(helloworld, sizeof(helloworld)); + /* write */ +@@ -426,7 +426,7 @@ static void test_qga_file_ops(gconstpointer fix) + eof = qdict_get_bool(val, "eof"); + g_assert_cmpint(count, ==, sizeof(helloworld)); + g_assert_cmpint(eof, ==, 0); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* flush */ +@@ -434,7 +434,7 @@ static void test_qga_file_ops(gconstpointer fix) + " 'arguments': {'handle': %" PRId64 "} }", + id); + ret = qmp_fd(fixture->fd, cmd); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* close */ +@@ -442,7 +442,7 @@ static void test_qga_file_ops(gconstpointer fix) + " 'arguments': {'handle': %" PRId64 "} }", + id); + ret = qmp_fd(fixture->fd, cmd); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* check content */ +@@ -462,7 +462,7 @@ static void test_qga_file_ops(gconstpointer fix) + g_assert_nonnull(ret); + qmp_assert_no_error(ret); + id = qdict_get_int(ret, "return"); +- QDECREF(ret); ++ qobject_unref(ret); + + /* read */ + cmd = g_strdup_printf("{'execute': 'guest-file-read'," +@@ -477,7 +477,7 @@ static void test_qga_file_ops(gconstpointer fix) + g_assert(eof); + g_assert_cmpstr(b64, ==, enc); + +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + g_free(enc); + +@@ -493,7 +493,7 @@ static void test_qga_file_ops(gconstpointer fix) + g_assert_cmpint(count, ==, 0); + g_assert(eof); + g_assert_cmpstr(b64, ==, ""); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* seek */ +@@ -508,7 +508,7 @@ static void test_qga_file_ops(gconstpointer fix) + eof = qdict_get_bool(val, "eof"); + g_assert_cmpint(count, ==, 6); + g_assert(!eof); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* partial read */ +@@ -527,7 +527,7 @@ static void test_qga_file_ops(gconstpointer fix) + g_assert_cmpmem(dec, count, helloworld + 6, sizeof(helloworld) - 6); + g_free(dec); + +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* close */ +@@ -535,7 +535,7 @@ static void test_qga_file_ops(gconstpointer fix) + " 'arguments': {'handle': %" PRId64 "} }", + id); + ret = qmp_fd(fixture->fd, cmd); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + } + +@@ -555,7 +555,7 @@ static void test_qga_file_write_read(gconstpointer fix) + g_assert_nonnull(ret); + qmp_assert_no_error(ret); + id = qdict_get_int(ret, "return"); +- QDECREF(ret); ++ qobject_unref(ret); + + enc = g_base64_encode(helloworld, sizeof(helloworld)); + /* write */ +@@ -571,7 +571,7 @@ static void test_qga_file_write_read(gconstpointer fix) + eof = qdict_get_bool(val, "eof"); + g_assert_cmpint(count, ==, sizeof(helloworld)); + g_assert_cmpint(eof, ==, 0); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* read (check implicit flush) */ +@@ -586,7 +586,7 @@ static void test_qga_file_write_read(gconstpointer fix) + g_assert_cmpint(count, ==, 0); + g_assert(eof); + g_assert_cmpstr(b64, ==, ""); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* seek to 0 */ +@@ -601,7 +601,7 @@ static void test_qga_file_write_read(gconstpointer fix) + eof = qdict_get_bool(val, "eof"); + g_assert_cmpint(count, ==, 0); + g_assert(!eof); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + + /* read */ +@@ -616,7 +616,7 @@ static void test_qga_file_write_read(gconstpointer fix) + g_assert_cmpint(count, ==, sizeof(helloworld)); + g_assert(eof); + g_assert_cmpstr(b64, ==, enc); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + g_free(enc); + +@@ -625,7 +625,7 @@ static void test_qga_file_write_read(gconstpointer fix) + " 'arguments': {'handle': %" PRId64 "} }", + id); + ret = qmp_fd(fixture->fd, cmd); +- QDECREF(ret); ++ qobject_unref(ret); + g_free(cmd); + } + +@@ -642,7 +642,7 @@ static void test_qga_get_time(gconstpointer fix) + time = qdict_get_int(ret, "return"); + g_assert_cmpint(time, >, 0); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_blacklist(gconstpointer data) +@@ -661,7 +661,7 @@ static void test_qga_blacklist(gconstpointer data) + desc = qdict_get_try_str(error, "desc"); + g_assert_cmpstr(class, ==, "GenericError"); + g_assert_nonnull(g_strstr_len(desc, -1, "has been disabled")); +- QDECREF(ret); ++ qobject_unref(ret); + + ret = qmp_fd(fix.fd, "{'execute': 'guest-get-time'}"); + g_assert_nonnull(ret); +@@ -670,12 +670,12 @@ static void test_qga_blacklist(gconstpointer data) + desc = qdict_get_try_str(error, "desc"); + g_assert_cmpstr(class, ==, "GenericError"); + g_assert_nonnull(g_strstr_len(desc, -1, "has been disabled")); +- QDECREF(ret); ++ qobject_unref(ret); + + /* check something work */ + ret = qmp_fd(fix.fd, "{'execute': 'guest-get-fsinfo'}"); + qmp_assert_no_error(ret); +- QDECREF(ret); ++ qobject_unref(ret); + + fixture_tear_down(&fix, NULL); + } +@@ -772,7 +772,7 @@ static void test_qga_fsfreeze_status(gconstpointer fix) + status = qdict_get_try_str(ret, "return"); + g_assert_cmpstr(status, ==, "thawed"); + +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_guest_exec(gconstpointer fix) +@@ -795,7 +795,7 @@ static void test_qga_guest_exec(gconstpointer fix) + val = qdict_get_qdict(ret, "return"); + pid = qdict_get_int(val, "pid"); + g_assert_cmpint(pid, >, 0); +- QDECREF(ret); ++ qobject_unref(ret); + + /* wait for completion */ + now = g_get_monotonic_time(); +@@ -807,7 +807,7 @@ static void test_qga_guest_exec(gconstpointer fix) + val = qdict_get_qdict(ret, "return"); + exited = qdict_get_bool(val, "exited"); + if (!exited) { +- QDECREF(ret); ++ qobject_unref(ret); + } + } while (!exited && + g_get_monotonic_time() < now + 5 * G_TIME_SPAN_SECOND); +@@ -822,7 +822,7 @@ static void test_qga_guest_exec(gconstpointer fix) + g_assert_cmpint(len, ==, 12); + g_assert_cmpstr((char *)decoded, ==, "\" test_str \""); + g_free(decoded); +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_guest_exec_invalid(gconstpointer fix) +@@ -841,7 +841,7 @@ static void test_qga_guest_exec_invalid(gconstpointer fix) + desc = qdict_get_str(error, "desc"); + g_assert_cmpstr(class, ==, "GenericError"); + g_assert_cmpint(strlen(desc), >, 0); +- QDECREF(ret); ++ qobject_unref(ret); + + /* invalid pid */ + ret = qmp_fd(fixture->fd, "{'execute': 'guest-exec-status'," +@@ -853,7 +853,7 @@ static void test_qga_guest_exec_invalid(gconstpointer fix) + desc = qdict_get_str(error, "desc"); + g_assert_cmpstr(class, ==, "GenericError"); + g_assert_cmpint(strlen(desc), >, 0); +- QDECREF(ret); ++ qobject_unref(ret); + } + + static void test_qga_guest_get_osinfo(gconstpointer data) +@@ -905,7 +905,7 @@ static void test_qga_guest_get_osinfo(gconstpointer data) + g_assert_nonnull(str); + g_assert_cmpstr(str, ==, "unit-test"); + +- QDECREF(ret); ++ qobject_unref(ret); + g_free(env[0]); + fixture_tear_down(&fixture, NULL); + } +diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c +index db690cc..e0ed461 100644 +--- a/tests/test-qmp-cmds.c ++++ b/tests/test-qmp-cmds.c +@@ -106,8 +106,8 @@ static void test_dispatch_cmd(void) + assert(resp != NULL); + assert(!qdict_haskey(qobject_to(QDict, resp), "error")); + +- qobject_decref(resp); +- QDECREF(req); ++ qobject_unref(resp); ++ qobject_unref(req); + } + + /* test commands that return an error due to invalid parameters */ +@@ -123,8 +123,8 @@ static void test_dispatch_cmd_failure(void) + assert(resp != NULL); + assert(qdict_haskey(qobject_to(QDict, resp), "error")); + +- qobject_decref(resp); +- QDECREF(req); ++ qobject_unref(resp); ++ qobject_unref(req); + + /* check that with extra arguments it throws an error */ + req = qdict_new(); +@@ -137,8 +137,8 @@ static void test_dispatch_cmd_failure(void) + assert(resp != NULL); + assert(qdict_haskey(qobject_to(QDict, resp), "error")); + +- qobject_decref(resp); +- QDECREF(req); ++ qobject_unref(resp); ++ qobject_unref(req); + } + + static QObject *test_qmp_dispatch(QDict *req) +@@ -153,8 +153,8 @@ static QObject *test_qmp_dispatch(QDict *req) + assert(resp && !qdict_haskey(resp, "error")); + ret = qdict_get(resp, "return"); + assert(ret); +- qobject_incref(ret); +- qobject_decref(resp_obj); ++ qobject_ref(ret); ++ qobject_unref(resp_obj); + return ret; + } + +@@ -195,7 +195,7 @@ static void test_dispatch_cmd_io(void) + assert(qdict_get_int(ret_dict_dict2_userdef, "integer") == 422); + assert(!strcmp(qdict_get_str(ret_dict_dict2_userdef, "string"), "hello2")); + assert(!strcmp(qdict_get_str(ret_dict_dict2, "string"), "blah4")); +- QDECREF(ret); ++ qobject_unref(ret); + + qdict_put_int(args3, "a", 66); + qdict_put(req, "arguments", args3); +@@ -204,9 +204,9 @@ static void test_dispatch_cmd_io(void) + ret3 = qobject_to(QNum, test_qmp_dispatch(req)); + g_assert(qnum_get_try_int(ret3, &val)); + g_assert_cmpint(val, ==, 66); +- QDECREF(ret3); ++ qobject_unref(ret3); + +- QDECREF(req); ++ qobject_unref(req); + } + + /* test generated dealloc functions for generated types */ +@@ -257,7 +257,7 @@ static void test_dealloc_partial(void) + v = qobject_input_visitor_new(QOBJECT(ud2_dict)); + visit_type_UserDefTwo(v, NULL, &ud2, &err); + visit_free(v); +- QDECREF(ud2_dict); ++ qobject_unref(ud2_dict); + } + + /* verify that visit_type_XXX() cleans up properly on error */ +diff --git a/tests/test-qmp-event.c b/tests/test-qmp-event.c +index bb10366..3a7c227 100644 +--- a/tests/test-qmp-event.c ++++ b/tests/test-qmp-event.c +@@ -133,7 +133,7 @@ static void event_prepare(TestEventData *data, + static void event_teardown(TestEventData *data, + const void *unused) + { +- QDECREF(data->expect); ++ qobject_unref(data->expect); + test_event_data = NULL; + + g_mutex_unlock(&test_event_lock); +diff --git a/tests/test-qobject-input-visitor.c b/tests/test-qobject-input-visitor.c +index 6dc59c6..0f4d234 100644 +--- a/tests/test-qobject-input-visitor.c ++++ b/tests/test-qobject-input-visitor.c +@@ -35,7 +35,7 @@ typedef struct TestInputVisitorData { + static void visitor_input_teardown(TestInputVisitorData *data, + const void *unused) + { +- qobject_decref(data->obj); ++ qobject_unref(data->obj); + data->obj = NULL; + + if (data->qiv) { +@@ -483,7 +483,7 @@ static void test_visitor_in_any(TestInputVisitorData *data, + g_assert(qnum); + g_assert(qnum_get_try_int(qnum, &val)); + g_assert_cmpint(val, ==, -42); +- qobject_decref(res); ++ qobject_unref(res); + + v = visitor_input_test_init(data, "{ 'integer': -42, 'boolean': true, 'string': 'foo' }"); + visit_type_any(v, NULL, &res, &error_abort); +@@ -505,7 +505,7 @@ static void test_visitor_in_any(TestInputVisitorData *data, + qstring = qobject_to(QString, qobj); + g_assert(qstring); + g_assert_cmpstr(qstring_get_str(qstring), ==, "foo"); +- qobject_decref(res); ++ qobject_unref(res); + } + + static void test_visitor_in_null(TestInputVisitorData *data, +@@ -530,7 +530,7 @@ static void test_visitor_in_null(TestInputVisitorData *data, + visit_start_struct(v, NULL, NULL, 0, &error_abort); + visit_type_null(v, "a", &null, &error_abort); + g_assert(qobject_type(QOBJECT(null)) == QTYPE_QNULL); +- QDECREF(null); ++ qobject_unref(null); + visit_type_null(v, "b", &null, &err); + error_free_or_abort(&err); + g_assert(!null); +@@ -1262,7 +1262,7 @@ static void do_test_visitor_in_qmp_introspect(TestInputVisitorData *data, + g_assert(schema); + + qapi_free_SchemaInfoList(schema); +- qobject_decref(obj); ++ qobject_unref(obj); + visit_free(v); + } + +diff --git a/tests/test-qobject-output-visitor.c b/tests/test-qobject-output-visitor.c +index ecf21c0..be63585 100644 +--- a/tests/test-qobject-output-visitor.c ++++ b/tests/test-qobject-output-visitor.c +@@ -40,7 +40,7 @@ static void visitor_output_teardown(TestOutputVisitorData *data, + { + visit_free(data->ov); + data->ov = NULL; +- qobject_decref(data->obj); ++ qobject_unref(data->obj); + data->obj = NULL; + } + +@@ -346,7 +346,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data, + g_assert(qnum); + g_assert(qnum_get_try_int(qnum, &val)); + g_assert_cmpint(val, ==, -42); +- qobject_decref(qobj); ++ qobject_unref(qobj); + + visitor_reset(data); + qdict = qdict_new(); +@@ -355,7 +355,7 @@ static void test_visitor_out_any(TestOutputVisitorData *data, + qdict_put_str(qdict, "string", "foo"); + qobj = QOBJECT(qdict); + visit_type_any(data->ov, NULL, &qobj, &error_abort); +- qobject_decref(qobj); ++ qobject_unref(qobj); + qdict = qobject_to(QDict, visitor_get(data)); + g_assert(qdict); + qnum = qobject_to(QNum, qdict_get(qdict, "integer")); +@@ -630,7 +630,7 @@ static void check_native_list(QObject *qobj, + qvalue = qobject_to(QNum, tmp); + g_assert(qnum_get_try_uint(qvalue, &val)); + g_assert_cmpint(val, ==, i); +- qobject_decref(qlist_pop(qlist)); ++ qobject_unref(qlist_pop(qlist)); + } + break; + +@@ -654,7 +654,7 @@ static void check_native_list(QObject *qobj, + qvalue = qobject_to(QNum, tmp); + g_assert(qnum_get_try_int(qvalue, &val)); + g_assert_cmpint(val, ==, i); +- qobject_decref(qlist_pop(qlist)); ++ qobject_unref(qlist_pop(qlist)); + } + break; + case USER_DEF_NATIVE_LIST_UNION_KIND_BOOLEAN: +@@ -665,7 +665,7 @@ static void check_native_list(QObject *qobj, + g_assert(tmp); + qvalue = qobject_to(QBool, tmp); + g_assert_cmpint(qbool_get_bool(qvalue), ==, i % 3 == 0); +- qobject_decref(qlist_pop(qlist)); ++ qobject_unref(qlist_pop(qlist)); + } + break; + case USER_DEF_NATIVE_LIST_UNION_KIND_STRING: +@@ -678,7 +678,7 @@ static void check_native_list(QObject *qobj, + qvalue = qobject_to(QString, tmp); + sprintf(str, "%d", i); + g_assert_cmpstr(qstring_get_str(qvalue), ==, str); +- qobject_decref(qlist_pop(qlist)); ++ qobject_unref(qlist_pop(qlist)); + } + break; + case USER_DEF_NATIVE_LIST_UNION_KIND_NUMBER: +@@ -695,7 +695,7 @@ static void check_native_list(QObject *qobj, + g_string_printf(double_actual, "%.6f", qnum_get_double(qvalue)); + g_assert_cmpstr(double_actual->str, ==, double_expected->str); + +- qobject_decref(qlist_pop(qlist)); ++ qobject_unref(qlist_pop(qlist)); + g_string_free(double_expected, true); + g_string_free(double_actual, true); + } +@@ -703,7 +703,7 @@ static void check_native_list(QObject *qobj, + default: + g_assert_not_reached(); + } +- QDECREF(qlist); ++ qobject_unref(qlist); + } + + static void test_native_list(TestOutputVisitorData *data, +diff --git a/tests/test-visitor-serialization.c b/tests/test-visitor-serialization.c +index d18d90d..1c5a8b9 100644 +--- a/tests/test-visitor-serialization.c ++++ b/tests/test-visitor-serialization.c +@@ -1036,10 +1036,10 @@ static void qmp_deserialize(void **native_out, void *datap, + output_json = qobject_to_json(obj_orig); + obj = qobject_from_json(qstring_get_str(output_json), &error_abort); + +- QDECREF(output_json); ++ qobject_unref(output_json); + d->qiv = qobject_input_visitor_new(obj); +- qobject_decref(obj_orig); +- qobject_decref(obj); ++ qobject_unref(obj_orig); ++ qobject_unref(obj); + visit(d->qiv, native_out, errp); + } + +diff --git a/tests/test-x86-cpuid-compat.c b/tests/test-x86-cpuid-compat.c +index 9e4a508..c1ee197 100644 +--- a/tests/test-x86-cpuid-compat.c ++++ b/tests/test-x86-cpuid-compat.c +@@ -19,7 +19,7 @@ static char *get_cpu0_qom_path(void) + + cpu0 = qobject_to(QDict, qlist_peek(ret)); + path = g_strdup(qdict_get_str(cpu0, "qom_path")); +- QDECREF(resp); ++ qobject_unref(resp); + return path; + } + +@@ -30,8 +30,8 @@ static QObject *qom_get(const char *path, const char *prop) + " 'property': %s } }", + path, prop); + QObject *ret = qdict_get(resp, "return"); +- qobject_incref(ret); +- QDECREF(resp); ++ qobject_ref(ret); ++ qobject_unref(resp); + return ret; + } + +@@ -41,7 +41,7 @@ static bool qom_get_bool(const char *path, const char *prop) + QBool *value = qobject_to(QBool, qom_get(path, prop)); + bool b = qbool_get_bool(value); + +- QDECREF(value); ++ qobject_unref(value); + return b; + } + #endif +@@ -66,7 +66,7 @@ static void test_cpuid_prop(const void *data) + g_assert_cmpint(val, ==, args->expected_value); + qtest_end(); + +- QDECREF(value); ++ qobject_unref(value); + g_free(path); + } + +@@ -142,8 +142,8 @@ static void test_feature_flag(const void *data) + + g_assert(!!(value & (1U << args->bitnr)) == args->expected_value); + +- QDECREF(present); +- QDECREF(filtered); ++ qobject_unref(present); ++ qobject_unref(filtered); + g_free(path); + } + +diff --git a/tests/tmp105-test.c b/tests/tmp105-test.c +index 66c7a01..d093cff 100644 +--- a/tests/tmp105-test.c ++++ b/tests/tmp105-test.c +@@ -74,7 +74,7 @@ static int qmp_tmp105_get_temperature(const char *id) + "'property': 'temperature' } }", id); + g_assert(qdict_haskey(response, "return")); + ret = qdict_get_int(response, "return"); +- QDECREF(response); ++ qobject_unref(response); + return ret; + } + +@@ -85,7 +85,7 @@ static void qmp_tmp105_set_temperature(const char *id, int value) + response = qmp("{ 'execute': 'qom-set', 'arguments': { 'path': %s, " + "'property': 'temperature', 'value': %d } }", id, value); + g_assert(qdict_haskey(response, "return")); +- QDECREF(response); ++ qobject_unref(response); + } + + #define TMP105_PRECISION (1000/16) +diff --git a/tests/vhost-user-test.c b/tests/vhost-user-test.c +index 61d9972..bbc8091 100644 +--- a/tests/vhost-user-test.c ++++ b/tests/vhost-user-test.c +@@ -727,7 +727,7 @@ static void test_migrate(void) + rsp = qmp("{ 'execute': 'migrate_set_speed'," + "'arguments': { 'value': 10 } }"); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + + cmd = g_strdup_printf("{ 'execute': 'migrate'," + "'arguments': { 'uri': '%s' } }", +@@ -735,7 +735,7 @@ static void test_migrate(void) + rsp = qmp(cmd); + g_free(cmd); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + + wait_for_log_fd(s); + +@@ -751,7 +751,7 @@ static void test_migrate(void) + rsp = qmp("{ 'execute': 'migrate_set_speed'," + "'arguments': { 'value': 0 } }"); + g_assert(qdict_haskey(rsp, "return")); +- QDECREF(rsp); ++ qobject_unref(rsp); + + qmp_eventwait("STOP"); + +diff --git a/tests/virtio-net-test.c b/tests/virtio-net-test.c +index 0a3c5dd..b285a26 100644 +--- a/tests/virtio-net-test.c ++++ b/tests/virtio-net-test.c +@@ -173,7 +173,7 @@ static void rx_stop_cont_test(QVirtioDevice *dev, + qvirtqueue_kick(dev, vq, free_head); + + rsp = qmp("{ 'execute' : 'stop'}"); +- QDECREF(rsp); ++ qobject_unref(rsp); + + ret = iov_send(socket, iov, 2, 0, sizeof(len) + sizeof(test)); + g_assert_cmpint(ret, ==, sizeof(test) + sizeof(len)); +@@ -182,9 +182,9 @@ static void rx_stop_cont_test(QVirtioDevice *dev, + * ensure the packet data gets queued in QEMU, before we do 'cont'. + */ + rsp = qmp("{ 'execute' : 'query-status'}"); +- QDECREF(rsp); ++ qobject_unref(rsp); + rsp = qmp("{ 'execute' : 'cont'}"); +- QDECREF(rsp); ++ qobject_unref(rsp); + + qvirtio_wait_used_elem(dev, vq, free_head, NULL, QVIRTIO_NET_TIMEOUT_US); + memread(req_addr + VNET_HDR_SIZE, buffer, sizeof(test)); +diff --git a/tests/vmgenid-test.c b/tests/vmgenid-test.c +index 2ec274e..8d915c6 100644 +--- a/tests/vmgenid-test.c ++++ b/tests/vmgenid-test.c +@@ -125,7 +125,7 @@ static void read_guid_from_monitor(QemuUUID *guid) + guid_str = qdict_get_str(rsp_ret, "guid"); + g_assert(qemu_uuid_parse(guid_str, guid) == 0); + } +- QDECREF(rsp); ++ qobject_unref(rsp); + } + + static char disk[] = "tests/vmgenid-test-disk-XXXXXX"; +diff --git a/tests/wdt_ib700-test.c b/tests/wdt_ib700-test.c +index 3b5bbcf..797288d 100644 +--- a/tests/wdt_ib700-test.c ++++ b/tests/wdt_ib700-test.c +@@ -16,7 +16,7 @@ static void qmp_check_no_event(QTestState *s) + { + QDict *resp = qtest_qmp(s, "{'execute':'query-status'}"); + g_assert(qdict_haskey(resp, "return")); +- QDECREF(resp); ++ qobject_unref(resp); + } + + static QDict *ib700_program_and_wait(QTestState *s) +@@ -48,8 +48,8 @@ static QDict *ib700_program_and_wait(QTestState *s) + qtest_clock_step(s, 2 * NANOSECONDS_PER_SECOND); + event = qtest_qmp_eventwait_ref(s, "WATCHDOG"); + data = qdict_get_qdict(event, "data"); +- QINCREF(data); +- QDECREF(event); ++ qobject_ref(data); ++ qobject_unref(event); + return data; + } + +@@ -62,7 +62,7 @@ static void ib700_pause(void) + qtest_irq_intercept_in(s, "ioapic"); + d = ib700_program_and_wait(s); + g_assert(!strcmp(qdict_get_str(d, "action"), "pause")); +- QDECREF(d); ++ qobject_unref(d); + qtest_qmp_eventwait(s, "STOP"); + qtest_quit(s); + } +@@ -75,7 +75,7 @@ static void ib700_reset(void) + qtest_irq_intercept_in(s, "ioapic"); + d = ib700_program_and_wait(s); + g_assert(!strcmp(qdict_get_str(d, "action"), "reset")); +- QDECREF(d); ++ qobject_unref(d); + qtest_qmp_eventwait(s, "RESET"); + qtest_quit(s); + } +@@ -89,7 +89,7 @@ static void ib700_shutdown(void) + qtest_irq_intercept_in(s, "ioapic"); + d = ib700_program_and_wait(s); + g_assert(!strcmp(qdict_get_str(d, "action"), "reset")); +- QDECREF(d); ++ qobject_unref(d); + qtest_qmp_eventwait(s, "SHUTDOWN"); + qtest_quit(s); + } +@@ -102,7 +102,7 @@ static void ib700_none(void) + qtest_irq_intercept_in(s, "ioapic"); + d = ib700_program_and_wait(s); + g_assert(!strcmp(qdict_get_str(d, "action"), "none")); +- QDECREF(d); ++ qobject_unref(d); + qtest_quit(s); + } + +diff --git a/util/keyval.c b/util/keyval.c +index 1c7351a..13def4a 100644 +--- a/util/keyval.c ++++ b/util/keyval.c +@@ -126,7 +126,7 @@ static int key_to_index(const char *key, const char **end) + * Else, fail because we have conflicting needs on how to map + * @key_in_cur. + * In any case, take over the reference to @value, i.e. if the caller +- * wants to hold on to a reference, it needs to QINCREF(). ++ * wants to hold on to a reference, it needs to qobject_ref(). + * Use @key up to @key_cursor to identify the key in error messages. + * On success, return the mapped value. + * On failure, store an error through @errp and return NULL. +@@ -143,7 +143,7 @@ static QObject *keyval_parse_put(QDict *cur, + if (qobject_type(old) != (value ? QTYPE_QSTRING : QTYPE_QDICT)) { + error_setg(errp, "Parameters '%.*s.*' used inconsistently", + (int)(key_cursor - key), key); +- QDECREF(value); ++ qobject_unref(value); + return NULL; + } + if (!value) { +@@ -375,10 +375,10 @@ static QObject *keyval_listify(QDict *cur, GSList *key_of_cur, Error **errp) + error_setg(errp, "Parameter '%s%d' missing", key, i); + g_free(key); + g_free(elt); +- QDECREF(list); ++ qobject_unref(list); + return NULL; + } +- qobject_incref(elt[i]); ++ qobject_ref(elt[i]); + qlist_append_obj(list, elt[i]); + } + +@@ -404,7 +404,7 @@ QDict *keyval_parse(const char *params, const char *implied_key, + while (*s) { + s = keyval_parse_one(qdict, s, implied_key, errp); + if (!s) { +- QDECREF(qdict); ++ qobject_unref(qdict); + return NULL; + } + implied_key = NULL; +@@ -412,7 +412,7 @@ QDict *keyval_parse(const char *params, const char *implied_key, + + listified = keyval_listify(qdict, NULL, errp); + if (!listified) { +- QDECREF(qdict); ++ qobject_unref(qdict); + return NULL; + } + assert(listified == QOBJECT(qdict)); +diff --git a/util/qemu-config.c b/util/qemu-config.c +index 20f7d14..14d8402 100644 +--- a/util/qemu-config.c ++++ b/util/qemu-config.c +@@ -562,8 +562,8 @@ static void config_parse_qdict_section(QDict *options, QemuOptsList *opts, + } + + out: +- QDECREF(subqdict); +- QDECREF(list); ++ qobject_unref(subqdict); ++ qobject_unref(list); + } + + void qemu_config_parse_qdict(QDict *options, QemuOptsList **lists, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qobject-Use-qobject_to-instead-of-type-cast.patch b/SOURCES/kvm-qobject-Use-qobject_to-instead-of-type-cast.patch new file mode 100644 index 0000000..5b91f3f --- /dev/null +++ b/SOURCES/kvm-qobject-Use-qobject_to-instead-of-type-cast.patch @@ -0,0 +1,55 @@ +From 901ef4e58539427f6a26e134d915e02db0fd0df3 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:08 +0200 +Subject: [PATCH 010/268] qobject: Use qobject_to() instead of type cast +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-2-armbru@redhat.com> +Patchwork-id: 80732 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 01/23] qobject: Use qobject_to() instead of type cast +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +The proper way to convert from (abstract) QObject to a (concrete) +subtype is qobject_to(). Look for offenders that type cast instead: + + $ git-grep '(Q[A-Z][a-z]* \*)' + hmp.c: qmp_device_add((QDict *)qdict, NULL, &err); + include/qapi/qmp/qobject.h: return (QObject *)obj; + qobject/qobject.c:static void (*qdestroy[QTYPE__MAX])(QObject *) = { + tests/check-qdict.c: dst = (QDict *)qdict_crumple(src, &error_abort); + +The first two cast away const, the third isn't a type cast. Fix the +fourth. + +Signed-off-by: Markus Armbruster +Message-Id: <20180426152805.8469-1-armbru@redhat.com> +Reviewed-by: Eric Blake +Reviewed-by: Marc-André Lureau +(cherry picked from commit 46cfbf13b06d551072ed17fbfca67c103edf814f) +Signed-off-by: Miroslav Rezanina +--- + tests/check-qdict.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/tests/check-qdict.c b/tests/check-qdict.c +index 2e73c2f..08d4303 100644 +--- a/tests/check-qdict.c ++++ b/tests/check-qdict.c +@@ -657,7 +657,7 @@ static void qdict_crumple_test_empty(void) + + src = qdict_new(); + +- dst = (QDict *)qdict_crumple(src, &error_abort); ++ dst = qobject_to(QDict, qdict_crumple(src, &error_abort)); + + g_assert_cmpint(qdict_size(dst), ==, 0); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qobject-use-a-QObjectBase_-struct.patch b/SOURCES/kvm-qobject-use-a-QObjectBase_-struct.patch new file mode 100644 index 0000000..6727663 --- /dev/null +++ b/SOURCES/kvm-qobject-use-a-QObjectBase_-struct.patch @@ -0,0 +1,254 @@ +From 219fdf2ab8905618e25b426e1844a67367f0c073 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:10 +0200 +Subject: [PATCH 012/268] qobject: use a QObjectBase_ struct +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-4-armbru@redhat.com> +Patchwork-id: 80742 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 03/23] qobject: use a QObjectBase_ struct +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +From: Marc-André Lureau + +By moving the base fields to a QObjectBase_, QObject can be a type +which also has a 'base' field. This allows writing a generic QOBJECT() +macro that will work with any QObject type, including QObject +itself. The container_of() macro ensures that the object to cast has a +QObjectBase_ base field, giving some type safety guarantees. QObject +must have no members but QObjectBase_ base, or else QOBJECT() breaks. + +QObjectBase_ is not a typedef and uses a trailing underscore to make +it obvious it is not for normal use and to avoid potential abuse. + +Signed-off-by: Marc-André Lureau +Reviewed-by: Eric Blake +Message-Id: <20180419150145.24795-3-marcandre.lureau@redhat.com> +Reviewed-by: Markus Armbruster +Signed-off-by: Markus Armbruster +(cherry picked from commit 3d3eacaeccaab718ea0e2ddaa578bfae9e311c59) +Signed-off-by: Miroslav Rezanina +--- + include/qapi/qmp/qbool.h | 2 +- + include/qapi/qmp/qdict.h | 2 +- + include/qapi/qmp/qlist.h | 2 +- + include/qapi/qmp/qnull.h | 2 +- + include/qapi/qmp/qnum.h | 2 +- + include/qapi/qmp/qobject.h | 31 ++++++++++++++++++++----------- + include/qapi/qmp/qstring.h | 2 +- + qobject/qobject.c | 12 ++++++------ + tests/check-qdict.c | 6 +++--- + 9 files changed, 35 insertions(+), 26 deletions(-) + +diff --git a/include/qapi/qmp/qbool.h b/include/qapi/qmp/qbool.h +index b9a44a1..5f61e38 100644 +--- a/include/qapi/qmp/qbool.h ++++ b/include/qapi/qmp/qbool.h +@@ -17,7 +17,7 @@ + #include "qapi/qmp/qobject.h" + + struct QBool { +- QObject base; ++ struct QObjectBase_ base; + bool value; + }; + +diff --git a/include/qapi/qmp/qdict.h b/include/qapi/qmp/qdict.h +index 2cc3e90..921a28d 100644 +--- a/include/qapi/qmp/qdict.h ++++ b/include/qapi/qmp/qdict.h +@@ -25,7 +25,7 @@ typedef struct QDictEntry { + } QDictEntry; + + struct QDict { +- QObject base; ++ struct QObjectBase_ base; + size_t size; + QLIST_HEAD(,QDictEntry) table[QDICT_BUCKET_MAX]; + }; +diff --git a/include/qapi/qmp/qlist.h b/include/qapi/qmp/qlist.h +index 5c673ac..8d2c32c 100644 +--- a/include/qapi/qmp/qlist.h ++++ b/include/qapi/qmp/qlist.h +@@ -22,7 +22,7 @@ typedef struct QListEntry { + } QListEntry; + + struct QList { +- QObject base; ++ struct QObjectBase_ base; + QTAILQ_HEAD(,QListEntry) head; + }; + +diff --git a/include/qapi/qmp/qnull.h b/include/qapi/qmp/qnull.h +index c992ee2..e8ea2c3 100644 +--- a/include/qapi/qmp/qnull.h ++++ b/include/qapi/qmp/qnull.h +@@ -16,7 +16,7 @@ + #include "qapi/qmp/qobject.h" + + struct QNull { +- QObject base; ++ struct QObjectBase_ base; + }; + + extern QNull qnull_; +diff --git a/include/qapi/qmp/qnum.h b/include/qapi/qmp/qnum.h +index 3e47475..45bf02a 100644 +--- a/include/qapi/qmp/qnum.h ++++ b/include/qapi/qmp/qnum.h +@@ -45,7 +45,7 @@ typedef enum { + * convert under the hood. + */ + struct QNum { +- QObject base; ++ struct QObjectBase_ base; + QNumKind kind; + union { + int64_t i64; +diff --git a/include/qapi/qmp/qobject.h b/include/qapi/qmp/qobject.h +index 5206ff9..a713c01 100644 +--- a/include/qapi/qmp/qobject.h ++++ b/include/qapi/qmp/qobject.h +@@ -34,13 +34,21 @@ + + #include "qapi/qapi-builtin-types.h" + +-struct QObject { ++/* Not for use outside include/qapi/qmp/ */ ++struct QObjectBase_ { + QType type; + size_t refcnt; + }; + +-/* Get the 'base' part of an object */ +-#define QOBJECT(obj) (&(obj)->base) ++/* this struct must have no other members than base */ ++struct QObject { ++ struct QObjectBase_ base; ++}; ++ ++#define QOBJECT(obj) ({ \ ++ typeof(obj) _obj = (obj); \ ++ _obj ? container_of(&(_obj)->base, QObject, base) : NULL; \ ++}) + + /* High-level interface for qobject_incref() */ + #define QINCREF(obj) \ +@@ -68,8 +76,8 @@ QEMU_BUILD_BUG_MSG(QTYPE__MAX != 7, + static inline void qobject_init(QObject *obj, QType type) + { + assert(QTYPE_NONE < type && type < QTYPE__MAX); +- obj->refcnt = 1; +- obj->type = type; ++ obj->base.refcnt = 1; ++ obj->base.type = type; + } + + /** +@@ -77,8 +85,9 @@ static inline void qobject_init(QObject *obj, QType type) + */ + static inline void qobject_incref(QObject *obj) + { +- if (obj) +- obj->refcnt++; ++ if (obj) { ++ obj->base.refcnt++; ++ } + } + + /** +@@ -101,8 +110,8 @@ void qobject_destroy(QObject *obj); + */ + static inline void qobject_decref(QObject *obj) + { +- assert(!obj || obj->refcnt); +- if (obj && --obj->refcnt == 0) { ++ assert(!obj || obj->base.refcnt); ++ if (obj && --obj->base.refcnt == 0) { + qobject_destroy(obj); + } + } +@@ -112,8 +121,8 @@ static inline void qobject_decref(QObject *obj) + */ + static inline QType qobject_type(const QObject *obj) + { +- assert(QTYPE_NONE < obj->type && obj->type < QTYPE__MAX); +- return obj->type; ++ assert(QTYPE_NONE < obj->base.type && obj->base.type < QTYPE__MAX); ++ return obj->base.type; + } + + /** +diff --git a/include/qapi/qmp/qstring.h b/include/qapi/qmp/qstring.h +index 30ae260..b3b3d44 100644 +--- a/include/qapi/qmp/qstring.h ++++ b/include/qapi/qmp/qstring.h +@@ -16,7 +16,7 @@ + #include "qapi/qmp/qobject.h" + + struct QString { +- QObject base; ++ struct QObjectBase_ base; + char *string; + size_t length; + size_t capacity; +diff --git a/qobject/qobject.c b/qobject/qobject.c +index 87649c5..cf4b7e2 100644 +--- a/qobject/qobject.c ++++ b/qobject/qobject.c +@@ -37,9 +37,9 @@ static void (*qdestroy[QTYPE__MAX])(QObject *) = { + + void qobject_destroy(QObject *obj) + { +- assert(!obj->refcnt); +- assert(QTYPE_QNULL < obj->type && obj->type < QTYPE__MAX); +- qdestroy[obj->type](obj); ++ assert(!obj->base.refcnt); ++ assert(QTYPE_QNULL < obj->base.type && obj->base.type < QTYPE__MAX); ++ qdestroy[obj->base.type](obj); + } + + +@@ -62,11 +62,11 @@ bool qobject_is_equal(const QObject *x, const QObject *y) + return true; + } + +- if (!x || !y || x->type != y->type) { ++ if (!x || !y || x->base.type != y->base.type) { + return false; + } + +- assert(QTYPE_NONE < x->type && x->type < QTYPE__MAX); ++ assert(QTYPE_NONE < x->base.type && x->base.type < QTYPE__MAX); + +- return qis_equal[x->type](x, y); ++ return qis_equal[x->base.type](x, y); + } +diff --git a/tests/check-qdict.c b/tests/check-qdict.c +index 08d4303..07bb8f4 100644 +--- a/tests/check-qdict.c ++++ b/tests/check-qdict.c +@@ -570,11 +570,11 @@ static void qdict_join_test(void) + } + + /* Check the references */ +- g_assert(qdict_get(dict1, "foo")->refcnt == 1); +- g_assert(qdict_get(dict1, "bar")->refcnt == 1); ++ g_assert(qdict_get(dict1, "foo")->base.refcnt == 1); ++ g_assert(qdict_get(dict1, "bar")->base.refcnt == 1); + + if (!overwrite) { +- g_assert(qdict_get(dict2, "foo")->refcnt == 1); ++ g_assert(qdict_get(dict2, "foo")->base.refcnt == 1); + } + + /* Clean up */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qxl-fix-local-renderer-crash.patch b/SOURCES/kvm-qxl-fix-local-renderer-crash.patch new file mode 100644 index 0000000..79e4cd6 --- /dev/null +++ b/SOURCES/kvm-qxl-fix-local-renderer-crash.patch @@ -0,0 +1,52 @@ +From f44bd8a3bc71029a53cc06e848787cd867390b47 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 13 Jun 2018 13:15:57 +0200 +Subject: [PATCH 063/268] qxl: fix local renderer crash +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20180613131557.12255-2-kraxel@redhat.com> +Patchwork-id: 80662 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] qxl: fix local renderer crash +Bugzilla: 1567733 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Laszlo Ersek + +Make sure we only ask the spice local renderer for display updates in +case we have a valid primary surface. Without that spice is confused +and throws errors in case a display update request (triggered by +screendump for example) happens in parallel to a mode switch and hits +the race window where the old primary surface is gone and the new isn't +establisted yet. + +Cc: qemu-stable@nongnu.org +Fixes: https://bugzilla.redhat.com//show_bug.cgi?id=1567733 +Signed-off-by: Gerd Hoffmann +Reviewed-by: Marc-André Lureau +Message-id: 20180427115528.345-1-kraxel@redhat.com +(cherry picked from commit 5bd5c27c7d284d01477c5cc022ce22438c46bf9f) +Signed-off-by: Miroslav Rezanina +--- + hw/display/qxl-render.c | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c +index e7ac4f8..c62b9a5 100644 +--- a/hw/display/qxl-render.c ++++ b/hw/display/qxl-render.c +@@ -169,7 +169,8 @@ void qxl_render_update(PCIQXLDevice *qxl) + + qemu_mutex_lock(&qxl->ssd.lock); + +- if (!runstate_is_running() || !qxl->guest_primary.commands) { ++ if (!runstate_is_running() || !qxl->guest_primary.commands || ++ qxl->mode == QXL_MODE_UNDEFINED) { + qxl_render_update_area_unlocked(qxl); + qemu_mutex_unlock(&qxl->ssd.lock); + return; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-qxl-use-guest_monitor_config-for-local-renderer.patch b/SOURCES/kvm-qxl-use-guest_monitor_config-for-local-renderer.patch new file mode 100644 index 0000000..b747c50 --- /dev/null +++ b/SOURCES/kvm-qxl-use-guest_monitor_config-for-local-renderer.patch @@ -0,0 +1,141 @@ +From 53f7b5bea75509998a5280e714029d639d255e9e Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Thu, 22 Nov 2018 11:33:36 +0000 +Subject: [PATCH 06/16] qxl: use guest_monitor_config for local renderer. +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20181122113336.2925-2-kraxel@redhat.com> +Patchwork-id: 83094 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] qxl: use guest_monitor_config for local renderer. +Bugzilla: 1610163 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: John Snow + +When processing monitor config from guest store head0 width and height +for single-head configurations. Use these when creating the +DisplaySurface in the local renderer. + +This fixes a rendering issue with wayland. Wayland rounds up the +framebuffer width and height to a multiple of 64, so with odd +resolutions (800x600 for example) the framebuffer is larger than the +actual screen. The monitor config has the actual screen size though. + +This fixes guest display for anything using the local renderer +(non-spice UI, screendump monitor command). + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Marc-André Lureau +Message-id: 20180919103057.9666-1-kraxel@redhat.com +(cherry picked from commit 979f7ef8966bc4495a710ed9e4af42098f92ee79) +Signed-off-by: Danilo C. L. de Paula +--- + hw/display/qxl-render.c | 18 ++++++++++-------- + hw/display/qxl.c | 12 ++++++++++++ + hw/display/qxl.h | 2 ++ + 3 files changed, 24 insertions(+), 8 deletions(-) + +diff --git a/hw/display/qxl-render.c b/hw/display/qxl-render.c +index c62b9a5..6debe8f 100644 +--- a/hw/display/qxl-render.c ++++ b/hw/display/qxl-render.c +@@ -98,6 +98,8 @@ static void qxl_render_update_area_unlocked(PCIQXLDevice *qxl) + { + VGACommonState *vga = &qxl->vga; + DisplaySurface *surface; ++ int width = qxl->guest_head0_width ?: qxl->guest_primary.surface.width; ++ int height = qxl->guest_head0_height ?: qxl->guest_primary.surface.height; + int i; + + if (qxl->guest_primary.resized) { +@@ -111,8 +113,8 @@ static void qxl_render_update_area_unlocked(PCIQXLDevice *qxl) + qxl_set_rect_to_surface(qxl, &qxl->dirty[0]); + qxl->num_dirty_rects = 1; + trace_qxl_render_guest_primary_resized( +- qxl->guest_primary.surface.width, +- qxl->guest_primary.surface.height, ++ width, ++ height, + qxl->guest_primary.qxl_stride, + qxl->guest_primary.bytes_pp, + qxl->guest_primary.bits_pp); +@@ -120,15 +122,15 @@ static void qxl_render_update_area_unlocked(PCIQXLDevice *qxl) + pixman_format_code_t format = + qemu_default_pixman_format(qxl->guest_primary.bits_pp, true); + surface = qemu_create_displaysurface_from +- (qxl->guest_primary.surface.width, +- qxl->guest_primary.surface.height, ++ (width, ++ height, + format, + qxl->guest_primary.abs_stride, + qxl->guest_primary.data); + } else { + surface = qemu_create_displaysurface +- (qxl->guest_primary.surface.width, +- qxl->guest_primary.surface.height); ++ (width, ++ height); + } + dpy_gfx_replace_surface(vga->con, surface); + } +@@ -144,8 +146,8 @@ static void qxl_render_update_area_unlocked(PCIQXLDevice *qxl) + qxl->dirty[i].top < 0 || + qxl->dirty[i].left > qxl->dirty[i].right || + qxl->dirty[i].top > qxl->dirty[i].bottom || +- qxl->dirty[i].right > qxl->guest_primary.surface.width || +- qxl->dirty[i].bottom > qxl->guest_primary.surface.height) { ++ qxl->dirty[i].right > width || ++ qxl->dirty[i].bottom > height) { + continue; + } + qxl_blit(qxl, qxl->dirty+i); +diff --git a/hw/display/qxl.c b/hw/display/qxl.c +index a71714c..e36ef32 100644 +--- a/hw/display/qxl.c ++++ b/hw/display/qxl.c +@@ -258,6 +258,8 @@ static void qxl_spice_destroy_surfaces(PCIQXLDevice *qxl, qxl_async_io async) + + static void qxl_spice_monitors_config_async(PCIQXLDevice *qxl, int replay) + { ++ QXLMonitorsConfig *cfg; ++ + trace_qxl_spice_monitors_config(qxl->id); + if (replay) { + /* +@@ -285,6 +287,16 @@ static void qxl_spice_monitors_config_async(PCIQXLDevice *qxl, int replay) + (uintptr_t)qxl_cookie_new(QXL_COOKIE_TYPE_IO, + QXL_IO_MONITORS_CONFIG_ASYNC)); + } ++ ++ cfg = qxl_phys2virt(qxl, qxl->guest_monitors_config, MEMSLOT_GROUP_GUEST); ++ if (cfg->count == 1) { ++ qxl->guest_primary.resized = 1; ++ qxl->guest_head0_width = cfg->heads[0].width; ++ qxl->guest_head0_height = cfg->heads[0].height; ++ } else { ++ qxl->guest_head0_width = 0; ++ qxl->guest_head0_height = 0; ++ } + } + + void qxl_spice_reset_image_cache(PCIQXLDevice *qxl) +diff --git a/hw/display/qxl.h b/hw/display/qxl.h +index 089696e..ffa6260 100644 +--- a/hw/display/qxl.h ++++ b/hw/display/qxl.h +@@ -79,6 +79,8 @@ typedef struct PCIQXLDevice { + QXLPHYSICAL guest_cursor; + + QXLPHYSICAL guest_monitors_config; ++ uint32_t guest_head0_width; ++ uint32_t guest_head0_height; + + QemuMutex track_lock; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-raw-Check-byte-range-uniformly.patch b/SOURCES/kvm-raw-Check-byte-range-uniformly.patch new file mode 100644 index 0000000..b8ab259 --- /dev/null +++ b/SOURCES/kvm-raw-Check-byte-range-uniformly.patch @@ -0,0 +1,159 @@ +From 1a834ba95f7179391ad82b3a22464b43731fbcb9 Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:42 +0200 +Subject: [PATCH 168/268] raw: Check byte range uniformly + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-3-famz@redhat.com> +Patchwork-id: 81152 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 02/13] raw: Check byte range uniformly +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +We don't verify the request range against s->size in the I/O callbacks +except for raw_co_pwritev. This is inconsistent (especially for +raw_co_pwrite_zeroes and raw_co_pdiscard), so fix them, in the meanwhile +make the helper reusable by the coming new callbacks. + +Note that in most cases the block layer already verifies the request +byte range against our reported image length, before invoking the driver +callbacks. The exception is during image creating, after +blk_set_allow_write_beyond_eof(blk, true) is called. But in that case, +the requests are not directly from the user or guest. So there is no +visible behavior change in adding the check code. + +The int64_t -> uint64_t inconsistency, as shown by the type casting, is +pre-existing due to the interface. + +Reviewed-by: Stefan Hajnoczi +Reviewed-by: Eric Blake +Signed-off-by: Fam Zheng +Message-id: 20180601092648.24614-3-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 384455385248762e74a080978f18f0c8f74757fe) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/raw-format.c | 64 +++++++++++++++++++++++++++++++++--------------------- + 1 file changed, 39 insertions(+), 25 deletions(-) + +diff --git a/block/raw-format.c b/block/raw-format.c +index fe33693..b69a067 100644 +--- a/block/raw-format.c ++++ b/block/raw-format.c +@@ -167,16 +167,37 @@ static void raw_reopen_abort(BDRVReopenState *state) + state->opaque = NULL; + } + ++/* Check and adjust the offset, against 'offset' and 'size' options. */ ++static inline int raw_adjust_offset(BlockDriverState *bs, uint64_t *offset, ++ uint64_t bytes, bool is_write) ++{ ++ BDRVRawState *s = bs->opaque; ++ ++ if (s->has_size && (*offset > s->size || bytes > (s->size - *offset))) { ++ /* There's not enough space for the write, or the read request is ++ * out-of-range. Don't read/write anything to prevent leaking out of ++ * the size specified in options. */ ++ return is_write ? -ENOSPC : -EINVAL;; ++ } ++ ++ if (*offset > INT64_MAX - s->offset) { ++ return -EINVAL; ++ } ++ *offset += s->offset; ++ ++ return 0; ++} ++ + static int coroutine_fn raw_co_preadv(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, + int flags) + { +- BDRVRawState *s = bs->opaque; ++ int ret; + +- if (offset > UINT64_MAX - s->offset) { +- return -EINVAL; ++ ret = raw_adjust_offset(bs, &offset, bytes, false); ++ if (ret) { ++ return ret; + } +- offset += s->offset; + + BLKDBG_EVENT(bs->file, BLKDBG_READ_AIO); + return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags); +@@ -186,23 +207,11 @@ static int coroutine_fn raw_co_pwritev(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, QEMUIOVector *qiov, + int flags) + { +- BDRVRawState *s = bs->opaque; + void *buf = NULL; + BlockDriver *drv; + QEMUIOVector local_qiov; + int ret; + +- if (s->has_size && (offset > s->size || bytes > (s->size - offset))) { +- /* There's not enough space for the data. Don't write anything and just +- * fail to prevent leaking out of the size specified in options. */ +- return -ENOSPC; +- } +- +- if (offset > UINT64_MAX - s->offset) { +- ret = -EINVAL; +- goto fail; +- } +- + if (bs->probed && offset < BLOCK_PROBE_BUF_SIZE && bytes) { + /* Handling partial writes would be a pain - so we just + * require that guests have 512-byte request alignment if +@@ -237,7 +246,10 @@ static int coroutine_fn raw_co_pwritev(BlockDriverState *bs, uint64_t offset, + qiov = &local_qiov; + } + +- offset += s->offset; ++ ret = raw_adjust_offset(bs, &offset, bytes, true); ++ if (ret) { ++ goto fail; ++ } + + BLKDBG_EVENT(bs->file, BLKDBG_WRITE_AIO); + ret = bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags); +@@ -267,22 +279,24 @@ static int coroutine_fn raw_co_pwrite_zeroes(BlockDriverState *bs, + int64_t offset, int bytes, + BdrvRequestFlags flags) + { +- BDRVRawState *s = bs->opaque; +- if (offset > UINT64_MAX - s->offset) { +- return -EINVAL; ++ int ret; ++ ++ ret = raw_adjust_offset(bs, (uint64_t *)&offset, bytes, true); ++ if (ret) { ++ return ret; + } +- offset += s->offset; + return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags); + } + + static int coroutine_fn raw_co_pdiscard(BlockDriverState *bs, + int64_t offset, int bytes) + { +- BDRVRawState *s = bs->opaque; +- if (offset > UINT64_MAX - s->offset) { +- return -EINVAL; ++ int ret; ++ ++ ret = raw_adjust_offset(bs, (uint64_t *)&offset, bytes, true); ++ if (ret) { ++ return ret; + } +- offset += s->offset; + return bdrv_co_pdiscard(bs->file->bs, offset, bytes); + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-raw-Implement-copy-offloading.patch b/SOURCES/kvm-raw-Implement-copy-offloading.patch new file mode 100644 index 0000000..0cefead --- /dev/null +++ b/SOURCES/kvm-raw-Implement-copy-offloading.patch @@ -0,0 +1,80 @@ +From 3995a70bd22fb0d3d7fe8edb692d2c03794a779f Mon Sep 17 00:00:00 2001 +From: Fam Zheng +Date: Fri, 29 Jun 2018 06:11:43 +0200 +Subject: [PATCH 169/268] raw: Implement copy offloading + +RH-Author: Fam Zheng +Message-id: <20180629061153.12687-4-famz@redhat.com> +Patchwork-id: 81155 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 03/13] raw: Implement copy offloading +Bugzilla: 1482537 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +Just pass down to ->file. + +Signed-off-by: Fam Zheng +Reviewed-by: Stefan Hajnoczi +Message-id: 20180601092648.24614-4-famz@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 72d219e2f916adeec9845473d239571a267f3314) +Signed-off-by: Fam Zheng +Signed-off-by: Miroslav Rezanina +--- + block/raw-format.c | 32 ++++++++++++++++++++++++++++++++ + 1 file changed, 32 insertions(+) + +diff --git a/block/raw-format.c b/block/raw-format.c +index b69a067..f2e468d 100644 +--- a/block/raw-format.c ++++ b/block/raw-format.c +@@ -497,6 +497,36 @@ static int raw_probe_geometry(BlockDriverState *bs, HDGeometry *geo) + return bdrv_probe_geometry(bs->file->bs, geo); + } + ++static int coroutine_fn raw_co_copy_range_from(BlockDriverState *bs, ++ BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ int ret; ++ ++ ret = raw_adjust_offset(bs, &src_offset, bytes, false); ++ if (ret) { ++ return ret; ++ } ++ return bdrv_co_copy_range_from(bs->file, src_offset, dst, dst_offset, ++ bytes, flags); ++} ++ ++static int coroutine_fn raw_co_copy_range_to(BlockDriverState *bs, ++ BdrvChild *src, uint64_t src_offset, ++ BdrvChild *dst, uint64_t dst_offset, ++ uint64_t bytes, BdrvRequestFlags flags) ++{ ++ int ret; ++ ++ ret = raw_adjust_offset(bs, &dst_offset, bytes, true); ++ if (ret) { ++ return ret; ++ } ++ return bdrv_co_copy_range_to(src, src_offset, bs->file, dst_offset, bytes, ++ flags); ++} ++ + BlockDriver bdrv_raw = { + .format_name = "raw", + .instance_size = sizeof(BDRVRawState), +@@ -513,6 +543,8 @@ BlockDriver bdrv_raw = { + .bdrv_co_pwrite_zeroes = &raw_co_pwrite_zeroes, + .bdrv_co_pdiscard = &raw_co_pdiscard, + .bdrv_co_block_status = &raw_co_block_status, ++ .bdrv_co_copy_range_from = &raw_co_copy_range_from, ++ .bdrv_co_copy_range_to = &raw_co_copy_range_to, + .bdrv_truncate = &raw_truncate, + .bdrv_getlength = &raw_getlength, + .has_variable_length = true, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-rbd-Close-image-in-qemu_rbd_open-error-path.patch b/SOURCES/kvm-rbd-Close-image-in-qemu_rbd_open-error-path.patch new file mode 100644 index 0000000..310ba84 --- /dev/null +++ b/SOURCES/kvm-rbd-Close-image-in-qemu_rbd_open-error-path.patch @@ -0,0 +1,49 @@ +From e39d1867ceea7910937bf8bfbd9491e33dc26111 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Thu, 10 Jan 2019 12:44:33 +0000 +Subject: [PATCH 03/14] rbd: Close image in qemu_rbd_open() error path +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Kevin Wolf +Message-id: <20190110124442.30132-4-kwolf@redhat.com> +Patchwork-id: 83951 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 03/12] rbd: Close image in qemu_rbd_open() error path +Bugzilla: 1644996 +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Eric Blake + +Commit e2b8247a322 introduced an error path in qemu_rbd_open() after +calling rbd_open(), but neglected to close the image again in this error +path. The error path should contain everything that the regular close +function qemu_rbd_close() contains. + +This adds the missing rbd_close() call. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit a51b9c4862c29f427931f45ee1d39ac1663ba859) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/rbd.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/block/rbd.c b/block/rbd.c +index ebe0701..dc369d0 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -780,6 +780,7 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + "automatically marking the image read-only."); + r = bdrv_set_read_only(bs, true, &local_err); + if (r < 0) { ++ rbd_close(s->image); + error_propagate(errp, local_err); + goto failed_open; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-rbd-Drop-deprecated-drive-parameter-filename.patch b/SOURCES/kvm-rbd-Drop-deprecated-drive-parameter-filename.patch new file mode 100644 index 0000000..f88dc4c --- /dev/null +++ b/SOURCES/kvm-rbd-Drop-deprecated-drive-parameter-filename.patch @@ -0,0 +1,59 @@ +From 0cfbcb2d3641f4826b39d641852b026e6c349f8a Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:13 +0200 +Subject: [PATCH 015/268] rbd: Drop deprecated -drive parameter "filename" + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-7-armbru@redhat.com> +Patchwork-id: 80735 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 06/23] rbd: Drop deprecated -drive parameter "filename" +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Parameter "filename" is deprecated since commit 91589d9e5ca, v2.10.0. +Time to get rid of it. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit bb9f762ff35459e34630d731a5a91c02602150d1) +Signed-off-by: Miroslav Rezanina +--- + block/rbd.c | 16 ---------------- + 1 file changed, 16 deletions(-) + +diff --git a/block/rbd.c b/block/rbd.c +index a14b42f..2842c0e 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -625,25 +625,9 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict *options, int flags, + QObject *crumpled = NULL; + const QDictEntry *e; + Error *local_err = NULL; +- const char *filename; + char *keypairs, *secretid; + int r; + +- /* If we are given a filename, parse the filename, with precedence given to +- * filename encoded options */ +- filename = qdict_get_try_str(options, "filename"); +- if (filename) { +- warn_report("'filename' option specified. " +- "This is an unsupported option, and may be deprecated " +- "in the future"); +- qemu_rbd_parse_filename(filename, options, &local_err); +- qdict_del(options, "filename"); +- if (local_err) { +- error_propagate(errp, local_err); +- return -EINVAL; +- } +- } +- + keypairs = g_strdup(qdict_get_try_str(options, "=keyvalue-pairs")); + if (keypairs) { + qdict_del(options, "=keyvalue-pairs"); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-rbd-New-parameter-auth-client-required.patch b/SOURCES/kvm-rbd-New-parameter-auth-client-required.patch new file mode 100644 index 0000000..5d53522 --- /dev/null +++ b/SOURCES/kvm-rbd-New-parameter-auth-client-required.patch @@ -0,0 +1,187 @@ +From 7aafba282f161865e4ffc4a2da7e06d59d3dfe9a Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:29 +0200 +Subject: [PATCH 031/268] rbd: New parameter auth-client-required + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-23-armbru@redhat.com> +Patchwork-id: 80731 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 22/23] rbd: New parameter auth-client-required +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Parameter auth-client-required lets you configure authentication +methods. We tried to provide that in v2.9.0, but backed out due to +interface design doubts (commit 464444fcc16). + +This commit is similar to what we backed out, but simpler: we use a +list of enumeration values instead of a list of objects with a member +of enumeration type. + +Let's review our reasons for backing out the first try, as stated in +the commit message: + + * The implementation uses deprecated rados_conf_set() key + "auth_supported". No biggie. + +Fixed: we use "auth-client-required". + + * The implementation makes -drive silently ignore invalid parameters + "auth" and "auth-supported.*.X" where X isn't "auth". Fixable (in + fact I'm going to fix similar bugs around parameter server), so + again no biggie. + +That fix is commit 2836284db60. This commit doesn't bring the bugs +back. + + * BlockdevOptionsRbd member @password-secret applies only to + authentication method cephx. Should it be a variant member of + RbdAuthMethod? + +We've had time to ponder, and we decided to stick to the way Ceph +configuration works: the key configured separately, and silently +ignored if the authentication method doesn't use it. + + * BlockdevOptionsRbd member @user could apply to both methods cephx + and none, but I'm not sure it's actually used with none. If it + isn't, should it be a variant member of RbdAuthMethod? + +Likewise. + + * The client offers a *set* of authentication methods, not a list. + Should the methods be optional members of BlockdevOptionsRbd instead + of members of list @auth-supported? The latter begs the question + what multiple entries for the same method mean. Trivial question + now that RbdAuthMethod contains nothing but @type, but less so when + RbdAuthMethod acquires other members, such the ones discussed above. + +Again, we decided to stick to the way Ceph configuration works, except +we make auth-client-required a list of enumeration values instead of a +string containing keywords separated by delimiters. + + * How BlockdevOptionsRbd member @auth-supported interacts with + settings from a configuration file specified with @conf is + undocumented. I suspect it's untested, too. + +Not actually true, the documentation for @conf says "Values in the +configuration file will be overridden by options specified via QAPI", +and we've tested this. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit a3699de4dde82bc76b33a83798a9da82c2336cce) +Signed-off-by: Miroslav Rezanina +--- + block/rbd.c | 42 ++++++++++++++++++++++++++++++++---------- + qapi/block-core.json | 13 +++++++++++++ + 2 files changed, 45 insertions(+), 10 deletions(-) + +diff --git a/block/rbd.c b/block/rbd.c +index c834d72..9c0903f 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -233,20 +233,42 @@ done: + + + static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, ++ BlockdevOptionsRbd *opts, + Error **errp) + { +- if (secretid == 0) { +- return 0; +- } ++ char *acr; ++ int r; ++ GString *accu; ++ RbdAuthModeList *auth; ++ ++ if (secretid) { ++ gchar *secret = qcrypto_secret_lookup_as_base64(secretid, ++ errp); ++ if (!secret) { ++ return -1; ++ } + +- gchar *secret = qcrypto_secret_lookup_as_base64(secretid, +- errp); +- if (!secret) { +- return -1; ++ rados_conf_set(cluster, "key", secret); ++ g_free(secret); + } + +- rados_conf_set(cluster, "key", secret); +- g_free(secret); ++ if (opts->has_auth_client_required) { ++ accu = g_string_new(""); ++ for (auth = opts->auth_client_required; auth; auth = auth->next) { ++ if (accu->str[0]) { ++ g_string_append_c(accu, ';'); ++ } ++ g_string_append(accu, RbdAuthMode_str(auth->value)); ++ } ++ acr = g_string_free(accu, FALSE); ++ r = rados_conf_set(cluster, "auth_client_required", acr); ++ g_free(acr); ++ if (r < 0) { ++ error_setg_errno(errp, -r, ++ "Could not set 'auth_client_required'"); ++ return r; ++ } ++ } + + return 0; + } +@@ -578,7 +600,7 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, + } + } + +- if (qemu_rbd_set_auth(*cluster, secretid, errp) < 0) { ++ if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) { + r = -EIO; + goto failed_shutdown; + } +diff --git a/qapi/block-core.json b/qapi/block-core.json +index c50517b..d1da7d1 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -3170,6 +3170,14 @@ + + + ## ++# @RbdAuthMode: ++# ++# Since: 3.0 ++## ++{ 'enum': 'RbdAuthMode', ++ 'data': [ 'cephx', 'none' ] } ++ ++## + # @BlockdevOptionsRbd: + # + # @pool: Ceph pool name. +@@ -3184,6 +3192,10 @@ + # + # @user: Ceph id name. + # ++# @auth-client-required: Acceptable authentication modes. ++# This maps to Ceph configuration option ++# "auth_client_required". (Since 3.0) ++# + # @server: Monitor host address and port. This maps + # to the "mon_host" Ceph option. + # +@@ -3195,6 +3207,7 @@ + '*conf': 'str', + '*snapshot': 'str', + '*user': 'str', ++ '*auth-client-required': ['RbdAuthMode'], + '*server': ['InetSocketAddressBase'] } } + + ## +-- +1.8.3.1 + diff --git a/SOURCES/kvm-rbd-New-parameter-key-secret.patch b/SOURCES/kvm-rbd-New-parameter-key-secret.patch new file mode 100644 index 0000000..3b1c31f --- /dev/null +++ b/SOURCES/kvm-rbd-New-parameter-key-secret.patch @@ -0,0 +1,162 @@ +From f560f687deba14702f4a8f6987168e2d51c5088a Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 18 Jun 2018 08:43:30 +0200 +Subject: [PATCH 032/268] rbd: New parameter key-secret + +RH-Author: Markus Armbruster +Message-id: <20180618084330.30009-24-armbru@redhat.com> +Patchwork-id: 80727 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 23/23] rbd: New parameter key-secret +Bugzilla: 1557995 +RH-Acked-by: Max Reitz +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Kevin Wolf + +Legacy -drive supports "password-secret" parameter that isn't +available with -blockdev / blockdev-add. That's because we backed out +our first try to provide it there due to interface design doubts, in +commit 577d8c9a811, v2.9.0. + +This is the second try. It brings back the parameter, except it's +named "key-secret" now. + +Let's review our reasons for backing out the first try, as stated in +the commit message: + + * BlockdevOptionsRbd member @password-secret isn't actually a + password, it's a key generated by Ceph. + +Addressed by the rename. + + * We're not sure where member @password-secret belongs (see the + previous commit). + +See previous commit. + + * How @password-secret interacts with settings from a configuration + file specified with @conf is undocumented. + +Not actually true, the documentation for @conf says "Values in the +configuration file will be overridden by options specified via QAPI", +and we've tested this. + +Signed-off-by: Markus Armbruster +Reviewed-by: Kevin Wolf +Signed-off-by: Kevin Wolf +(cherry picked from commit d083f954a95d37b460df0c2fbfe46ad7eb207b10) +[Conflict due to lack of commit e8e16d4baff "rbd: Switch to byte-based +callbacks" trivially resolved] + +Signed-off-by: Miroslav Rezanina +--- + block/rbd.c | 41 +++++++++++++++++++++++++---------------- + qapi/block-core.json | 6 ++++++ + 2 files changed, 31 insertions(+), 16 deletions(-) + +diff --git a/block/rbd.c b/block/rbd.c +index 9c0903f..3242bcd 100644 +--- a/block/rbd.c ++++ b/block/rbd.c +@@ -232,24 +232,25 @@ done: + } + + +-static int qemu_rbd_set_auth(rados_t cluster, const char *secretid, +- BlockdevOptionsRbd *opts, ++static int qemu_rbd_set_auth(rados_t cluster, BlockdevOptionsRbd *opts, + Error **errp) + { +- char *acr; ++ char *key, *acr; + int r; + GString *accu; + RbdAuthModeList *auth; + +- if (secretid) { +- gchar *secret = qcrypto_secret_lookup_as_base64(secretid, +- errp); +- if (!secret) { +- return -1; ++ if (opts->key_secret) { ++ key = qcrypto_secret_lookup_as_base64(opts->key_secret, errp); ++ if (!key) { ++ return -EIO; ++ } ++ r = rados_conf_set(cluster, "key", key); ++ g_free(key); ++ if (r < 0) { ++ error_setg_errno(errp, -r, "Could not set 'key'"); ++ return r; + } +- +- rados_conf_set(cluster, "key", secret); +- g_free(secret); + } + + if (opts->has_auth_client_required) { +@@ -360,9 +361,7 @@ static QemuOptsList runtime_opts = { + }, + }; + +-/* FIXME Deprecate and remove keypairs or make it available in QMP. +- * password_secret should eventually be configurable in opts->location. Support +- * for it in .bdrv_open will make it work here as well. */ ++/* FIXME Deprecate and remove keypairs or make it available in QMP. */ + static int qemu_rbd_do_create(BlockdevCreateOptions *options, + const char *keypairs, const char *password_secret, + Error **errp) +@@ -568,6 +567,16 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, + Error *local_err = NULL; + int r; + ++ if (secretid) { ++ if (opts->key_secret) { ++ error_setg(errp, ++ "Legacy 'password-secret' clashes with 'key-secret'"); ++ return -EINVAL; ++ } ++ opts->key_secret = g_strdup(secretid); ++ opts->has_key_secret = true; ++ } ++ + mon_host = qemu_rbd_mon_host(opts, &local_err); + if (local_err) { + error_propagate(errp, local_err); +@@ -600,8 +609,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx, + } + } + +- if (qemu_rbd_set_auth(*cluster, secretid, opts, errp) < 0) { +- r = -EIO; ++ r = qemu_rbd_set_auth(*cluster, opts, errp); ++ if (r < 0) { + goto failed_shutdown; + } + +diff --git a/qapi/block-core.json b/qapi/block-core.json +index d1da7d1..51eafdd 100644 +--- a/qapi/block-core.json ++++ b/qapi/block-core.json +@@ -3196,6 +3196,11 @@ + # This maps to Ceph configuration option + # "auth_client_required". (Since 3.0) + # ++# @key-secret: ID of a QCryptoSecret object providing a key ++# for cephx authentication. ++# This maps to Ceph configuration option ++# "key". (Since 3.0) ++# + # @server: Monitor host address and port. This maps + # to the "mon_host" Ceph option. + # +@@ -3208,6 +3213,7 @@ + '*snapshot': 'str', + '*user': 'str', + '*auth-client-required': ['RbdAuthMode'], ++ '*key-secret': 'str', + '*server': ['InetSocketAddressBase'] } } + + ## +-- +1.8.3.1 + diff --git a/SOURCES/kvm-redhat-enable-tpmdev-passthrough.patch b/SOURCES/kvm-redhat-enable-tpmdev-passthrough.patch new file mode 100644 index 0000000..b85df2c --- /dev/null +++ b/SOURCES/kvm-redhat-enable-tpmdev-passthrough.patch @@ -0,0 +1,61 @@ +From 63eeb9e6385bf7c43f2a8e1321abe0f24950f2cf Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Sun, 16 Dec 2018 17:18:22 +0000 +Subject: [PATCH] redhat: enable tpmdev passthrough +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20181216171822.5255-2-marcandre.lureau@redhat.com> +Patchwork-id: 83531 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] redhat: enable tpmdev passthrough +Bugzilla: 1654486 +RH-Acked-by: Ademar de Souza Reis Jr. +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Philippe Mathieu-Daudé + +https://bugzilla.redhat.com/show_bug.cgi?id=1654486 + +Signed-off-by: Marc-André Lureau +Signed-off-by: Danilo C. L. de Paula +--- + configure | 4 ++++ + redhat/qemu-kvm.spec.template | 2 +- + tests/Makefile.include | 5 +++-- + 3 files changed, 8 insertions(+), 3 deletions(-) + +diff --git a/configure b/configure +index 9446f49..139f3c8 100755 +--- a/configure ++++ b/configure +@@ -3723,6 +3723,10 @@ if test "$mingw32" != "yes"; then + else + tpm_emulator=no + fi ++ ++# RHEL8-specific, only passthrough for now, rhbz#1654486 ++tpm_emulator=no ++ + ########################################## + # attr probe + +diff --git a/tests/Makefile.include b/tests/Makefile.include +index 1177ca3..d200288 100644 +--- a/tests/Makefile.include ++++ b/tests/Makefile.include +@@ -301,8 +301,9 @@ check-qtest-i386-$(CONFIG_VHOST_USER_NET_TEST_i386) += tests/vhost-user-test$(EX + ifeq ($(CONFIG_VHOST_USER_NET_TEST_i386),) + check-qtest-x86_64-$(CONFIG_VHOST_USER_NET_TEST_x86_64) += tests/vhost-user-test$(EXESUF) + endif +-check-qtest-i386-$(CONFIG_TPM) += tests/tpm-crb-test$(EXESUF) +-check-qtest-i386-$(CONFIG_TPM) += tests/tpm-tis-test$(EXESUF) ++# RHEL8 - disable emulator tests, only passthrough for now ++#check-qtest-i386-$(CONFIG_TPM) += tests/tpm-crb-test$(EXESUF) ++#check-qtest-i386-$(CONFIG_TPM) += tests/tpm-tis-test$(EXESUF) + check-qtest-i386-$(CONFIG_SLIRP) += tests/test-netfilter$(EXESUF) + check-qtest-i386-$(CONFIG_POSIX) += tests/test-filter-mirror$(EXESUF) + check-qtest-i386-$(CONFIG_POSIX) += tests/test-filter-redirector$(EXESUF) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-rhel-Set-host-phys-bits-limit-48-on-rhel-machine-typ.patch b/SOURCES/kvm-rhel-Set-host-phys-bits-limit-48-on-rhel-machine-typ.patch new file mode 100644 index 0000000..e145cef --- /dev/null +++ b/SOURCES/kvm-rhel-Set-host-phys-bits-limit-48-on-rhel-machine-typ.patch @@ -0,0 +1,53 @@ +From 01a2ecb4c38fe4a35455ea706e76984ee8d5a769 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 13 Dec 2018 15:52:00 +0000 +Subject: [PATCH 2/5] rhel: Set host-phys-bits-limit=48 on rhel machine-types + +RH-Author: Eduardo Habkost +Message-id: <20181213155200.20300-3-ehabkost@redhat.com> +Patchwork-id: 83480 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/2] rhel: Set host-phys-bits-limit=48 on rhel machine-types +Bugzilla: 1598284 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Bandan Das +RH-Acked-by: Paolo Bonzini + +Upstream status: not applicable + +Currently we use the host physical address size by default on +VMs. This was a good default on most cases, but this is not the +case on host CPUs supporting 5-level EPT. On those cases, we +want VMs to use 4-level EPT by default. + +Ensure VMs will use 4-level EPT by default, by limiting physical +address bits to 48. + +Not applicable upstream because upstream doesn't set +host-phys-bits=on by default. + +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + include/hw/i386/pc.h | 5 +++++ + 1 file changed, 5 insertions(+) + +diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h +index 285e8df..c29176d 100644 +--- a/include/hw/i386/pc.h ++++ b/include/hw/i386/pc.h +@@ -963,6 +963,11 @@ extern void igd_passthrough_isa_bridge_create(PCIBus *bus, uint16_t gpu_dev_id); + .property = "host-phys-bits",\ + .value = "on",\ + },\ ++ { /* PC_RHEL_COMPAT */ \ ++ .driver = TYPE_X86_CPU,\ ++ .property = "host-phys-bits-limit",\ ++ .value = "48",\ ++ },\ + { /* PC_RHEL_COMPAT bz 1508330 */ \ + .driver = "vfio-pci",\ + .property = "x-no-geforce-quirks",\ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-rtl8139-fix-possible-out-of-bound-access.patch b/SOURCES/kvm-rtl8139-fix-possible-out-of-bound-access.patch new file mode 100644 index 0000000..2bc531c --- /dev/null +++ b/SOURCES/kvm-rtl8139-fix-possible-out-of-bound-access.patch @@ -0,0 +1,80 @@ +From b373cd31bd40fff153ecbeb4695b8667db28e4e0 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:58:57 +0000 +Subject: [PATCH 02/11] rtl8139: fix possible out of bound access +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-3-jasowang@redhat.com> +Patchwork-id: 83975 +O-Subject: [RHEL8 qemu-kvm PATCH 2/9] rtl8139: fix possible out of bound access +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +In rtl8139_do_receive(), we try to assign size_ to size which converts +from size_t to integer. This will cause troubles when size_ is greater +INT_MAX, this will lead a negative value in size and it can then pass +the check of size < MIN_BUF_SIZE which may lead out of bound access of +for both buf and buf1. + +Fixing by converting the type of size to size_t. + +CC: qemu-stable@nongnu.org +Reported-by: Daniel Shapira +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Jason Wang +(cherry picked from commit 1a326646fef38782e5542280040ec3ea23e4a730) +Signed-off-by: Danilo C. L. de Paula +--- + hw/net/rtl8139.c | 8 ++++---- + 1 file changed, 4 insertions(+), 4 deletions(-) + +diff --git a/hw/net/rtl8139.c b/hw/net/rtl8139.c +index 05453e7..0c916b7 100644 +--- a/hw/net/rtl8139.c ++++ b/hw/net/rtl8139.c +@@ -817,7 +817,7 @@ static ssize_t rtl8139_do_receive(NetClientState *nc, const uint8_t *buf, size_t + RTL8139State *s = qemu_get_nic_opaque(nc); + PCIDevice *d = PCI_DEVICE(s); + /* size is the length of the buffer passed to the driver */ +- int size = size_; ++ size_t size = size_; + const uint8_t *dot1q_buf = NULL; + + uint32_t packet_header = 0; +@@ -826,7 +826,7 @@ static ssize_t rtl8139_do_receive(NetClientState *nc, const uint8_t *buf, size_t + static const uint8_t broadcast_macaddr[6] = + { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff }; + +- DPRINTF(">>> received len=%d\n", size); ++ DPRINTF(">>> received len=%zu\n", size); + + /* test if board clock is stopped */ + if (!s->clock_enabled) +@@ -1035,7 +1035,7 @@ static ssize_t rtl8139_do_receive(NetClientState *nc, const uint8_t *buf, size_t + + if (size+4 > rx_space) + { +- DPRINTF("C+ Rx mode : descriptor %d size %d received %d + 4\n", ++ DPRINTF("C+ Rx mode : descriptor %d size %d received %zu + 4\n", + descriptor, rx_space, size); + + s->IntrStatus |= RxOverflow; +@@ -1148,7 +1148,7 @@ static ssize_t rtl8139_do_receive(NetClientState *nc, const uint8_t *buf, size_t + if (avail != 0 && RX_ALIGN(size + 8) >= avail) + { + DPRINTF("rx overflow: rx buffer length %d head 0x%04x " +- "read 0x%04x === available 0x%04x need 0x%04x\n", ++ "read 0x%04x === available 0x%04x need 0x%04zx\n", + s->RxBufferSize, s->RxBufAddr, s->RxBufPtr, avail, size + 8); + + s->IntrStatus |= RxOverflow; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390-doc-detailed-specifications-for-AP-virtualizati.patch b/SOURCES/kvm-s390-doc-detailed-specifications-for-AP-virtualizati.patch new file mode 100644 index 0000000..c8f7b3f --- /dev/null +++ b/SOURCES/kvm-s390-doc-detailed-specifications-for-AP-virtualizati.patch @@ -0,0 +1,889 @@ +From 3caa3a2cfbb83be5f52484a0542edc36cfac7b66 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Mon, 15 Oct 2018 10:19:31 +0100 +Subject: [PATCH 6/6] s390: doc: detailed specifications for AP virtualization + +RH-Author: Thomas Huth +Message-id: <1539598771-16223-7-git-send-email-thuth@redhat.com> +Patchwork-id: 82699 +O-Subject: [RHEL-8 qemu-kvm PATCH 6/6] s390: doc: detailed specifications for AP virtualization +Bugzilla: 1508142 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Tony Krowiak + +This patch provides documentation describing the AP architecture and +design concepts behind the virtualization of AP devices. It also +includes an example of how to configure AP devices for exclusive +use of KVM guests. + +Signed-off-by: Tony Krowiak +Reviewed-by: Pierre Morel +Tested-by: Pierre Morel +Tested-by: Christian Borntraeger +Message-Id: <20181010170309.12045-7-akrowiak@linux.ibm.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 694a8d703bfe06226a0574f5ec4af17a2b7060ef) +Signed-off-by: Danilo C. L. de Paula +--- + MAINTAINERS | 2 + + docs/vfio-ap.txt | 825 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 827 insertions(+) + create mode 100644 docs/vfio-ap.txt + +diff --git a/MAINTAINERS b/MAINTAINERS +index 99694d8..9b74756 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -96,6 +96,7 @@ F: include/hw/watchdog/wdt_diag288.h + F: pc-bios/s390-ccw/ + F: pc-bios/s390-ccw.img + F: target/s390x/ ++F: docs/vfio-ap.txt + K: ^Subject:.*(?i)s390x? + T: git git://github.com/cohuck/qemu.git s390-next + L: qemu-s390x@nongnu.org +@@ -1164,6 +1165,7 @@ F: hw/s390x/ap-bridge.c + F: include/hw/s390x/ap-device.h + F: include/hw/s390x/ap-bridge.h + F: hw/vfio/ap.c ++F: docs/vfio-ap.txt + L: qemu-s390x@nongnu.org + + vhost +diff --git a/docs/vfio-ap.txt b/docs/vfio-ap.txt +new file mode 100644 +index 0000000..1233968 +--- /dev/null ++++ b/docs/vfio-ap.txt +@@ -0,0 +1,825 @@ ++Adjunct Processor (AP) Device ++============================= ++ ++Contents: ++========= ++* Introduction ++* AP Architectural Overview ++* Start Interpretive Execution (SIE) Instruction ++* AP Matrix Configuration on Linux Host ++* Starting a Linux Guest Configured with an AP Matrix ++* Example: Configure AP Matrices for Three Linux Guests ++ ++Introduction: ++============ ++The IBM Adjunct Processor (AP) Cryptographic Facility is comprised ++of three AP instructions and from 1 to 256 PCIe cryptographic adapter cards. ++These AP devices provide cryptographic functions to all CPUs assigned to a ++linux system running in an IBM Z system LPAR. ++ ++On s390x, AP adapter cards are exposed via the AP bus. This document ++describes how those cards may be made available to KVM guests using the ++VFIO mediated device framework. ++ ++AP Architectural Overview: ++========================= ++In order understand the terminology used in the rest of this document, let's ++start with some definitions: ++ ++* AP adapter ++ ++ An AP adapter is an IBM Z adapter card that can perform cryptographic ++ functions. There can be from 0 to 256 adapters assigned to an LPAR depending ++ on the machine model. Adapters assigned to the LPAR in which a linux host is ++ running will be available to the linux host. Each adapter is identified by a ++ number from 0 to 255; however, the maximum adapter number allowed is ++ determined by machine model. When installed, an AP adapter is accessed by ++ AP instructions executed by any CPU. ++ ++* AP domain ++ ++ An adapter is partitioned into domains. Each domain can be thought of as ++ a set of hardware registers for processing AP instructions. An adapter can ++ hold up to 256 domains; however, the maximum domain number allowed is ++ determined by machine model. Each domain is identified by a number from 0 to ++ 255. Domains can be further classified into two types: ++ ++ * Usage domains are domains that can be accessed directly to process AP ++ commands ++ ++ * Control domains are domains that are accessed indirectly by AP ++ commands sent to a usage domain to control or change the domain; for ++ example, to set a secure private key for the domain. ++ ++* AP Queue ++ ++ An AP queue is the means by which an AP command-request message is sent to an ++ AP usage domain inside a specific AP. An AP queue is identified by a tuple ++ comprised of an AP adapter ID (APID) and an AP queue index (APQI). The ++ APQI corresponds to a given usage domain number within the adapter. This tuple ++ forms an AP Queue Number (APQN) uniquely identifying an AP queue. AP ++ instructions include a field containing the APQN to identify the AP queue to ++ which the AP command-request message is to be sent for processing. ++ ++* AP Instructions: ++ ++ There are three AP instructions: ++ ++ * NQAP: to enqueue an AP command-request message to a queue ++ * DQAP: to dequeue an AP command-reply message from a queue ++ * PQAP: to administer the queues ++ ++ AP instructions identify the domain that is targeted to process the AP ++ command; this must be one of the usage domains. An AP command may modify a ++ domain that is not one of the usage domains, but the modified domain ++ must be one of the control domains. ++ ++Start Interpretive Execution (SIE) Instruction ++============================================== ++A KVM guest is started by executing the Start Interpretive Execution (SIE) ++instruction. The SIE state description is a control block that contains the ++state information for a KVM guest and is supplied as input to the SIE ++instruction. The SIE state description contains a satellite control block called ++the Crypto Control Block (CRYCB). The CRYCB contains three fields to identify ++the adapters, usage domains and control domains assigned to the KVM guest: ++ ++* The AP Mask (APM) field is a bit mask that identifies the AP adapters assigned ++ to the KVM guest. Each bit in the mask, from left to right, corresponds to ++ an APID from 0-255. If a bit is set, the corresponding adapter is valid for ++ use by the KVM guest. ++ ++* The AP Queue Mask (AQM) field is a bit mask identifying the AP usage domains ++ assigned to the KVM guest. Each bit in the mask, from left to right, ++ corresponds to an AP queue index (APQI) from 0-255. If a bit is set, the ++ corresponding queue is valid for use by the KVM guest. ++ ++* The AP Domain Mask field is a bit mask that identifies the AP control domains ++ assigned to the KVM guest. The ADM bit mask controls which domains can be ++ changed by an AP command-request message sent to a usage domain from the ++ guest. Each bit in the mask, from left to right, corresponds to a domain from ++ 0-255. If a bit is set, the corresponding domain can be modified by an AP ++ command-request message sent to a usage domain. ++ ++If you recall from the description of an AP Queue, AP instructions include ++an APQN to identify the AP adapter and AP queue to which an AP command-request ++message is to be sent (NQAP and PQAP instructions), or from which a ++command-reply message is to be received (DQAP instruction). The validity of an ++APQN is defined by the matrix calculated from the APM and AQM; it is the ++cross product of all assigned adapter numbers (APM) with all assigned queue ++indexes (AQM). For example, if adapters 1 and 2 and usage domains 5 and 6 are ++assigned to a guest, the APQNs (1,5), (1,6), (2,5) and (2,6) will be valid for ++the guest. ++ ++The APQNs can provide secure key functionality - i.e., a private key is stored ++on the adapter card for each of its domains - so each APQN must be assigned to ++at most one guest or the linux host. ++ ++ Example 1: Valid configuration: ++ ------------------------------ ++ Guest1: adapters 1,2 domains 5,6 ++ Guest2: adapter 1,2 domain 7 ++ ++ This is valid because both guests have a unique set of APQNs: Guest1 has ++ APQNs (1,5), (1,6), (2,5) and (2,6); Guest2 has APQNs (1,7) and (2,7). ++ ++ Example 2: Valid configuration: ++ ------------------------------ ++ Guest1: adapters 1,2 domains 5,6 ++ Guest2: adapters 3,4 domains 5,6 ++ ++ This is also valid because both guests have a unique set of APQNs: ++ Guest1 has APQNs (1,5), (1,6), (2,5), (2,6); ++ Guest2 has APQNs (3,5), (3,6), (4,5), (4,6) ++ ++ Example 3: Invalid configuration: ++ -------------------------------- ++ Guest1: adapters 1,2 domains 5,6 ++ Guest2: adapter 1 domains 6,7 ++ ++ This is an invalid configuration because both guests have access to ++ APQN (1,6). ++ ++AP Matrix Configuration on Linux Host: ++===================================== ++A linux system is a guest of the LPAR in which it is running and has access to ++the AP resources configured for the LPAR. The LPAR's AP matrix is ++configured via its Activation Profile which can be edited on the HMC. When the ++linux system is started, the AP bus will detect the AP devices assigned to the ++LPAR and create the following in sysfs: ++ ++/sys/bus/ap ++... [devices] ++...... xx.yyyy ++...... ... ++...... cardxx ++...... ... ++ ++Where: ++ cardxx is AP adapter number xx (in hex) ++....xx.yyyy is an APQN with xx specifying the APID and yyyy specifying the ++ APQI ++ ++For example, if AP adapters 5 and 6 and domains 4, 71 (0x47), 171 (0xab) and ++255 (0xff) are configured for the LPAR, the sysfs representation on the linux ++host system would look like this: ++ ++/sys/bus/ap ++... [devices] ++...... 05.0004 ++...... 05.0047 ++...... 05.00ab ++...... 05.00ff ++...... 06.0004 ++...... 06.0047 ++...... 06.00ab ++...... 06.00ff ++...... card05 ++...... card06 ++ ++A set of default device drivers are also created to control each type of AP ++device that can be assigned to the LPAR on which a linux host is running: ++ ++/sys/bus/ap ++... [drivers] ++...... [cex2acard] for Crypto Express 2/3 accelerator cards ++...... [cex2aqueue] for AP queues served by Crypto Express 2/3 ++ accelerator cards ++...... [cex4card] for Crypto Express 4/5/6 accelerator and coprocessor ++ cards ++...... [cex4queue] for AP queues served by Crypto Express 4/5/6 ++ accelerator and coprocessor cards ++...... [pcixcccard] for Crypto Express 2/3 coprocessor cards ++...... [pcixccqueue] for AP queues served by Crypto Express 2/3 ++ coprocessor cards ++ ++Binding AP devices to device drivers ++------------------------------------ ++There are two sysfs files that specify bitmasks marking a subset of the APQN ++range as 'usable by the default AP queue device drivers' or 'not usable by the ++default device drivers' and thus available for use by the alternate device ++driver(s). The sysfs locations of the masks are: ++ ++ /sys/bus/ap/apmask ++ /sys/bus/ap/aqmask ++ ++ The 'apmask' is a 256-bit mask that identifies a set of AP adapter IDs ++ (APID). Each bit in the mask, from left to right (i.e., from most significant ++ to least significant bit in big endian order), corresponds to an APID from ++ 0-255. If a bit is set, the APID is marked as usable only by the default AP ++ queue device drivers; otherwise, the APID is usable by the vfio_ap ++ device driver. ++ ++ The 'aqmask' is a 256-bit mask that identifies a set of AP queue indexes ++ (APQI). Each bit in the mask, from left to right (i.e., from most significant ++ to least significant bit in big endian order), corresponds to an APQI from ++ 0-255. If a bit is set, the APQI is marked as usable only by the default AP ++ queue device drivers; otherwise, the APQI is usable by the vfio_ap device ++ driver. ++ ++ Take, for example, the following mask: ++ ++ 0x7dffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff ++ ++ It indicates: ++ ++ 1, 2, 3, 4, 5, and 7-255 belong to the default drivers' pool, and 0 and 6 ++ belong to the vfio_ap device driver's pool. ++ ++ The APQN of each AP queue device assigned to the linux host is checked by the ++ AP bus against the set of APQNs derived from the cross product of APIDs ++ and APQIs marked as usable only by the default AP queue device drivers. If a ++ match is detected, only the default AP queue device drivers will be probed; ++ otherwise, the vfio_ap device driver will be probed. ++ ++ By default, the two masks are set to reserve all APQNs for use by the default ++ AP queue device drivers. There are two ways the default masks can be changed: ++ ++ 1. The sysfs mask files can be edited by echoing a string into the ++ respective sysfs mask file in one of two formats: ++ ++ * An absolute hex string starting with 0x - like "0x12345678" - sets ++ the mask. If the given string is shorter than the mask, it is padded ++ with 0s on the right; for example, specifying a mask value of 0x41 is ++ the same as specifying: ++ ++ 0x4100000000000000000000000000000000000000000000000000000000000000 ++ ++ Keep in mind that the mask reads from left to right (i.e., most ++ significant to least significant bit in big endian order), so the mask ++ above identifies device numbers 1 and 7 (01000001). ++ ++ If the string is longer than the mask, the operation is terminated with ++ an error (EINVAL). ++ ++ * Individual bits in the mask can be switched on and off by specifying ++ each bit number to be switched in a comma separated list. Each bit ++ number string must be prepended with a ('+') or minus ('-') to indicate ++ the corresponding bit is to be switched on ('+') or off ('-'). Some ++ valid values are: ++ ++ "+0" switches bit 0 on ++ "-13" switches bit 13 off ++ "+0x41" switches bit 65 on ++ "-0xff" switches bit 255 off ++ ++ The following example: ++ +0,-6,+0x47,-0xf0 ++ ++ Switches bits 0 and 71 (0x47) on ++ Switches bits 6 and 240 (0xf0) off ++ ++ Note that the bits not specified in the list remain as they were before ++ the operation. ++ ++ 2. The masks can also be changed at boot time via parameters on the kernel ++ command line like this: ++ ++ ap.apmask=0xffff ap.aqmask=0x40 ++ ++ This would create the following masks: ++ ++ apmask: ++ 0xffff000000000000000000000000000000000000000000000000000000000000 ++ ++ aqmask: ++ 0x4000000000000000000000000000000000000000000000000000000000000000 ++ ++ Resulting in these two pools: ++ ++ default drivers pool: adapter 0-15, domain 1 ++ alternate drivers pool: adapter 16-255, domains 0, 2-255 ++ ++Configuring an AP matrix for a linux guest. ++------------------------------------------ ++The sysfs interfaces for configuring an AP matrix for a guest are built on the ++VFIO mediated device framework. To configure an AP matrix for a guest, a ++mediated matrix device must first be created for the /sys/devices/vfio_ap/matrix ++device. When the vfio_ap device driver is loaded, it registers with the VFIO ++mediated device framework. When the driver registers, the sysfs interfaces for ++creating mediated matrix devices is created: ++ ++/sys/devices ++... [vfio_ap] ++......[matrix] ++......... [mdev_supported_types] ++............ [vfio_ap-passthrough] ++............... create ++............... [devices] ++ ++A mediated AP matrix device is created by writing a UUID to the attribute file ++named 'create', for example: ++ ++ uuidgen > create ++ ++ or ++ ++ echo $uuid > create ++ ++When a mediated AP matrix device is created, a sysfs directory named after ++the UUID is created in the 'devices' subdirectory: ++ ++/sys/devices ++... [vfio_ap] ++......[matrix] ++......... [mdev_supported_types] ++............ [vfio_ap-passthrough] ++............... create ++............... [devices] ++.................. [$uuid] ++ ++There will also be three sets of attribute files created in the mediated ++matrix device's sysfs directory to configure an AP matrix for the ++KVM guest: ++ ++/sys/devices ++... [vfio_ap] ++......[matrix] ++......... [mdev_supported_types] ++............ [vfio_ap-passthrough] ++............... create ++............... [devices] ++.................. [$uuid] ++..................... assign_adapter ++..................... assign_control_domain ++..................... assign_domain ++..................... matrix ++..................... unassign_adapter ++..................... unassign_control_domain ++..................... unassign_domain ++ ++assign_adapter ++ To assign an AP adapter to the mediated matrix device, its APID is written ++ to the 'assign_adapter' file. This may be done multiple times to assign more ++ than one adapter. The APID may be specified using conventional semantics ++ as a decimal, hexadecimal, or octal number. For example, to assign adapters ++ 4, 5 and 16 to a mediated matrix device in decimal, hexadecimal and octal ++ respectively: ++ ++ echo 4 > assign_adapter ++ echo 0x5 > assign_adapter ++ echo 020 > assign_adapter ++ ++ In order to successfully assign an adapter: ++ ++ * The adapter number specified must represent a value from 0 up to the ++ maximum adapter number allowed by the machine model. If an adapter number ++ higher than the maximum is specified, the operation will terminate with ++ an error (ENODEV). ++ ++ * All APQNs that can be derived from the adapter ID being assigned and the ++ IDs of the previously assigned domains must be bound to the vfio_ap device ++ driver. If no domains have yet been assigned, then there must be at least ++ one APQN with the specified APID bound to the vfio_ap driver. If no such ++ APQNs are bound to the driver, the operation will terminate with an ++ error (EADDRNOTAVAIL). ++ ++ No APQN that can be derived from the adapter ID and the IDs of the ++ previously assigned domains can be assigned to another mediated matrix ++ device. If an APQN is assigned to another mediated matrix device, the ++ operation will terminate with an error (EADDRINUSE). ++ ++unassign_adapter ++ To unassign an AP adapter, its APID is written to the 'unassign_adapter' ++ file. This may also be done multiple times to unassign more than one adapter. ++ ++assign_domain ++ To assign a usage domain, the domain number is written into the ++ 'assign_domain' file. This may be done multiple times to assign more than one ++ usage domain. The domain number is specified using conventional semantics as ++ a decimal, hexadecimal, or octal number. For example, to assign usage domains ++ 4, 8, and 71 to a mediated matrix device in decimal, hexadecimal and octal ++ respectively: ++ ++ echo 4 > assign_domain ++ echo 0x8 > assign_domain ++ echo 0107 > assign_domain ++ ++ In order to successfully assign a domain: ++ ++ * The domain number specified must represent a value from 0 up to the ++ maximum domain number allowed by the machine model. If a domain number ++ higher than the maximum is specified, the operation will terminate with ++ an error (ENODEV). ++ ++ * All APQNs that can be derived from the domain ID being assigned and the IDs ++ of the previously assigned adapters must be bound to the vfio_ap device ++ driver. If no domains have yet been assigned, then there must be at least ++ one APQN with the specified APQI bound to the vfio_ap driver. If no such ++ APQNs are bound to the driver, the operation will terminate with an ++ error (EADDRNOTAVAIL). ++ ++ No APQN that can be derived from the domain ID being assigned and the IDs ++ of the previously assigned adapters can be assigned to another mediated ++ matrix device. If an APQN is assigned to another mediated matrix device, ++ the operation will terminate with an error (EADDRINUSE). ++ ++unassign_domain ++ To unassign a usage domain, the domain number is written into the ++ 'unassign_domain' file. This may be done multiple times to unassign more than ++ one usage domain. ++ ++assign_control_domain ++ To assign a control domain, the domain number is written into the ++ 'assign_control_domain' file. This may be done multiple times to ++ assign more than one control domain. The domain number may be specified using ++ conventional semantics as a decimal, hexadecimal, or octal number. For ++ example, to assign control domains 4, 8, and 71 to a mediated matrix device ++ in decimal, hexadecimal and octal respectively: ++ ++ echo 4 > assign_domain ++ echo 0x8 > assign_domain ++ echo 0107 > assign_domain ++ ++ In order to successfully assign a control domain, the domain number ++ specified must represent a value from 0 up to the maximum domain number ++ allowed by the machine model. If a control domain number higher than the ++ maximum is specified, the operation will terminate with an error (ENODEV). ++ ++unassign_control_domain ++ To unassign a control domain, the domain number is written into the ++ 'unassign_domain' file. This may be done multiple times to unassign more than ++ one control domain. ++ ++Notes: Hot plug/unplug is not currently supported for mediated AP matrix ++devices, so no changes to the AP matrix will be allowed while a guest using ++the mediated matrix device is running. Attempts to assign an adapter, ++domain or control domain will be rejected and an error (EBUSY) returned. ++ ++Starting a Linux Guest Configured with an AP Matrix: ++=================================================== ++To provide a mediated matrix device for use by a guest, the following option ++must be specified on the QEMU command line: ++ ++ -device vfio_ap,sysfsdev=$path-to-mdev ++ ++The sysfsdev parameter specifies the path to the mediated matrix device. ++There are a number of ways to specify this path: ++ ++/sys/devices/vfio_ap/matrix/$uuid ++/sys/bus/mdev/devices/$uuid ++/sys/bus/mdev/drivers/vfio_mdev/$uuid ++/sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/devices/$uuid ++ ++When the linux guest is started, the guest will open the mediated ++matrix device's file descriptor to get information about the mediated matrix ++device. The vfio_ap device driver will update the APM, AQM, and ADM fields in ++the guest's CRYCB with the adapter, usage domain and control domains assigned ++via the mediated matrix device's sysfs attribute files. Programs running on the ++linux guest will then: ++ ++1. Have direct access to the APQNs derived from the cross product of the AP ++ adapter numbers (APID) and queue indexes (APQI) specified in the APM and AQM ++ fields of the guests's CRYCB respectively. These APQNs identify the AP queues ++ that are valid for use by the guest; meaning, AP commands can be sent by the ++ guest to any of these queues for processing. ++ ++2. Have authorization to process AP commands to change a control domain ++ identified in the ADM field of the guest's CRYCB. The AP command must be sent ++ to a valid APQN (see 1 above). ++ ++CPU model features: ++ ++Three CPU model features are available for controlling guest access to AP ++facilities: ++ ++1. AP facilities feature ++ ++ The AP facilities feature indicates that AP facilities are installed on the ++ guest. This feature will be exposed for use only if the AP facilities ++ are installed on the host system. The feature is s390-specific and is ++ represented as a parameter of the -cpu option on the QEMU command line: ++ ++ qemu-system-s390x -cpu $model,ap=on|off ++ ++ Where: ++ ++ $model is the CPU model defined for the guest (defaults to the model of ++ the host system if not specified). ++ ++ ap=on|off indicates whether AP facilities are installed (on) or not ++ (off). The default for CPU models zEC12 or newer ++ is ap=on. AP facilities must be installed on the guest if a ++ vfio-ap device (-device vfio-ap,sysfsdev=$path) is configured ++ for the guest, or the guest will fail to start. ++ ++2. Query Configuration Information (QCI) facility ++ ++ The QCI facility is used by the AP bus running on the guest to query the ++ configuration of the AP facilities. This facility will be available ++ only if the QCI facility is installed on the host system. The feature is ++ s390-specific and is represented as a parameter of the -cpu option on the ++ QEMU command line: ++ ++ qemu-system-s390x -cpu $model,apqci=on|off ++ ++ Where: ++ ++ $model is the CPU model defined for the guest ++ ++ apqci=on|off indicates whether the QCI facility is installed (on) or ++ not (off). The default for CPU models zEC12 or newer ++ is apqci=on; for older models, QCI will not be installed. ++ ++ If QCI is installed (apqci=on) but AP facilities are not ++ (ap=off), an error message will be logged, but the guest ++ will be allowed to start. It makes no sense to have QCI ++ installed if the AP facilities are not; this is considered ++ an invalid configuration. ++ ++ If the QCI facility is not installed, APQNs with an APQI ++ greater than 15 will not be detected by the AP bus ++ running on the guest. ++ ++3. Adjunct Process Facility Test (APFT) facility ++ ++ The APFT facility is used by the AP bus running on the guest to test the ++ AP facilities available for a given AP queue. This facility will be available ++ only if the APFT facility is installed on the host system. The feature is ++ s390-specific and is represented as a parameter of the -cpu option on the ++ QEMU command line: ++ ++ qemu-system-s390x -cpu $model,apft=on|off ++ ++ Where: ++ ++ $model is the CPU model defined for the guest (defaults to the model of ++ the host system if not specified). ++ ++ apft=on|off indicates whether the APFT facility is installed (on) or ++ not (off). The default for CPU models zEC12 and ++ newer is apft=on for older models, APFT will not be ++ installed. ++ ++ If APFT is installed (apft=on) but AP facilities are not ++ (ap=off), an error message will be logged, but the guest ++ will be allowed to start. It makes no sense to have APFT ++ installed if the AP facilities are not; this is considered ++ an invalid configuration. ++ ++ It also makes no sense to turn APFT off because the AP bus ++ running on the guest will not detect CEX4 and newer devices ++ without it. Since only CEX4 and newer devices are supported ++ for guest usage, no AP devices can be made accessible to a ++ guest started without APFT installed. ++ ++Example: Configure AP Matrixes for Three Linux Guests: ++===================================================== ++Let's now provide an example to illustrate how KVM guests may be given ++access to AP facilities. For this example, we will show how to configure ++three guests such that executing the lszcrypt command on the guests would ++look like this: ++ ++Guest1 ++------ ++CARD.DOMAIN TYPE MODE ++------------------------------ ++05 CEX5C CCA-Coproc ++05.0004 CEX5C CCA-Coproc ++05.00ab CEX5C CCA-Coproc ++06 CEX5A Accelerator ++06.0004 CEX5A Accelerator ++06.00ab CEX5C CCA-Coproc ++ ++Guest2 ++------ ++CARD.DOMAIN TYPE MODE ++------------------------------ ++05 CEX5A Accelerator ++05.0047 CEX5A Accelerator ++05.00ff CEX5A Accelerator (5,4), (5,171), (6,4), (6,171), ++ ++Guest3 ++------ ++CARD.DOMAIN TYPE MODE ++------------------------------ ++06 CEX5A Accelerator ++06.0047 CEX5A Accelerator ++06.00ff CEX5A Accelerator ++ ++These are the steps: ++ ++1. Install the vfio_ap module on the linux host. The dependency chain for the ++ vfio_ap module is: ++ * iommu ++ * s390 ++ * zcrypt ++ * vfio ++ * vfio_mdev ++ * vfio_mdev_device ++ * KVM ++ ++ To build the vfio_ap module, the kernel build must be configured with the ++ following Kconfig elements selected: ++ * IOMMU_SUPPORT ++ * S390 ++ * ZCRYPT ++ * S390_AP_IOMMU ++ * VFIO ++ * VFIO_MDEV ++ * VFIO_MDEV_DEVICE ++ * KVM ++ ++ If using make menuconfig select the following to build the vfio_ap module: ++ -> Device Drivers ++ -> IOMMU Hardware Support ++ select S390 AP IOMMU Support ++ -> VFIO Non-Privileged userspace driver framework ++ -> Mediated device driver frramework ++ -> VFIO driver for Mediated devices ++ -> I/O subsystem ++ -> VFIO support for AP devices ++ ++2. Secure the AP queues to be used by the three guests so that the host can not ++ access them. To secure the AP queues 05.0004, 05.0047, 05.00ab, 05.00ff, ++ 06.0004, 06.0047, 06.00ab, and 06.00ff for use by the vfio_ap device driver, ++ the corresponding APQNs must be removed from the default queue drivers pool ++ as follows: ++ ++ echo -5,-6 > /sys/bus/ap/apmask ++ ++ echo -4,-0x47,-0xab,-0xff > /sys/bus/ap/aqmask ++ ++ This will result in AP queues 05.0004, 05.0047, 05.00ab, 05.00ff, 06.0004, ++ 06.0047, 06.00ab, and 06.00ff getting bound to the vfio_ap device driver. The ++ sysfs directory for the vfio_ap device driver will now contain symbolic links ++ to the AP queue devices bound to it: ++ ++ /sys/bus/ap ++ ... [drivers] ++ ...... [vfio_ap] ++ ......... [05.0004] ++ ......... [05.0047] ++ ......... [05.00ab] ++ ......... [05.00ff] ++ ......... [06.0004] ++ ......... [06.0047] ++ ......... [06.00ab] ++ ......... [06.00ff] ++ ++ Keep in mind that only type 10 and newer adapters (i.e., CEX4 and later) ++ can be bound to the vfio_ap device driver. The reason for this is to ++ simplify the implementation by not needlessly complicating the design by ++ supporting older devices that will go out of service in the relatively near ++ future, and for which there are few older systems on which to test. ++ ++ The administrator, therefore, must take care to secure only AP queues that ++ can be bound to the vfio_ap device driver. The device type for a given AP ++ queue device can be read from the parent card's sysfs directory. For example, ++ to see the hardware type of the queue 05.0004: ++ ++ cat /sys/bus/ap/devices/card05/hwtype ++ ++ The hwtype must be 10 or higher (CEX4 or newer) in order to be bound to the ++ vfio_ap device driver. ++ ++3. Create the mediated devices needed to configure the AP matrixes for the ++ three guests and to provide an interface to the vfio_ap driver for ++ use by the guests: ++ ++ /sys/devices/vfio_ap/matrix/ ++ --- [mdev_supported_types] ++ ------ [vfio_ap-passthrough] (passthrough mediated matrix device type) ++ --------- create ++ --------- [devices] ++ ++ To create the mediated devices for the three guests: ++ ++ uuidgen > create ++ uuidgen > create ++ uuidgen > create ++ ++ or ++ ++ echo $uuid1 > create ++ echo $uuid2 > create ++ echo $uuid3 > create ++ ++ This will create three mediated devices in the [devices] subdirectory named ++ after the UUID used to create the mediated device. We'll call them $uuid1, ++ $uuid2 and $uuid3 and this is the sysfs directory structure after creation: ++ ++ /sys/devices/vfio_ap/matrix/ ++ --- [mdev_supported_types] ++ ------ [vfio_ap-passthrough] ++ --------- [devices] ++ ------------ [$uuid1] ++ --------------- assign_adapter ++ --------------- assign_control_domain ++ --------------- assign_domain ++ --------------- matrix ++ --------------- unassign_adapter ++ --------------- unassign_control_domain ++ --------------- unassign_domain ++ ++ ------------ [$uuid2] ++ --------------- assign_adapter ++ --------------- assign_control_domain ++ --------------- assign_domain ++ --------------- matrix ++ --------------- unassign_adapter ++ ----------------unassign_control_domain ++ ----------------unassign_domain ++ ++ ------------ [$uuid3] ++ --------------- assign_adapter ++ --------------- assign_control_domain ++ --------------- assign_domain ++ --------------- matrix ++ --------------- unassign_adapter ++ ----------------unassign_control_domain ++ ----------------unassign_domain ++ ++4. The administrator now needs to configure the matrixes for the mediated ++ devices $uuid1 (for Guest1), $uuid2 (for Guest2) and $uuid3 (for Guest3). ++ ++ This is how the matrix is configured for Guest1: ++ ++ echo 5 > assign_adapter ++ echo 6 > assign_adapter ++ echo 4 > assign_domain ++ echo 0xab > assign_domain ++ ++ Control domains can similarly be assigned using the assign_control_domain ++ sysfs file. ++ ++ If a mistake is made configuring an adapter, domain or control domain, ++ you can use the unassign_xxx interfaces to unassign the adapter, domain or ++ control domain. ++ ++ To display the matrix configuration for Guest1: ++ ++ cat matrix ++ ++ The output will display the APQNs in the format xx.yyyy, where xx is ++ the adapter number and yyyy is the domain number. The output for Guest1 ++ will look like this: ++ ++ 05.0004 ++ 05.00ab ++ 06.0004 ++ 06.00ab ++ ++ This is how the matrix is configured for Guest2: ++ ++ echo 5 > assign_adapter ++ echo 0x47 > assign_domain ++ echo 0xff > assign_domain ++ ++ This is how the matrix is configured for Guest3: ++ ++ echo 6 > assign_adapter ++ echo 0x47 > assign_domain ++ echo 0xff > assign_domain ++ ++5. Start Guest1: ++ ++ /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ ++ -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid1 ... ++ ++7. Start Guest2: ++ ++ /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ ++ -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid2 ... ++ ++7. Start Guest3: ++ ++ /usr/bin/qemu-system-s390x ... -cpu host,ap=on,apqci=on,apft=on \ ++ -device vfio-ap,sysfsdev=/sys/devices/vfio_ap/matrix/$uuid3 ... ++ ++When the guest is shut down, the mediated matrix devices may be removed. ++ ++Using our example again, to remove the mediated matrix device $uuid1: ++ ++ /sys/devices/vfio_ap/matrix/ ++ --- [mdev_supported_types] ++ ------ [vfio_ap-passthrough] ++ --------- [devices] ++ ------------ [$uuid1] ++ --------------- remove ++ ++ ++ echo 1 > remove ++ ++ This will remove all of the mdev matrix device's sysfs structures including ++ the mdev device itself. To recreate and reconfigure the mdev matrix device, ++ all of the steps starting with step 3 will have to be performed again. Note ++ that the remove will fail if a guest using the mdev is still running. ++ ++ It is not necessary to remove an mdev matrix device, but one may want to ++ remove it if no guest will use it during the remaining lifetime of the linux ++ host. If the mdev matrix device is removed, one may want to also reconfigure ++ the pool of adapters and queues reserved for use by the default drivers. ++ ++Limitations ++=========== ++* The KVM/kernel interfaces do not provide a way to prevent restoring an APQN ++ to the default drivers pool of a queue that is still assigned to a mediated ++ device in use by a guest. It is incumbent upon the administrator to ++ ensure there is no mediated device in use by a guest to which the APQN is ++ assigned lest the host be given access to the private data of the AP queue ++ device, such as a private key configured specifically for the guest. ++ ++* Dynamically modifying the AP matrix for a running guest (which would amount to ++ hot(un)plug of AP devices for the guest) is currently not supported ++ ++* Live guest migration is not supported for guests using AP devices. +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-Enable-KVM-huge-page-backing-support.patch b/SOURCES/kvm-s390x-Enable-KVM-huge-page-backing-support.patch new file mode 100644 index 0000000..6a814f9 --- /dev/null +++ b/SOURCES/kvm-s390x-Enable-KVM-huge-page-backing-support.patch @@ -0,0 +1,114 @@ +From c9ff4b704e6be10e4309b5cdb8c2d3e7fc0d3d0f Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Mon, 6 Aug 2018 14:18:41 +0100 +Subject: [PATCH 2/3] s390x: Enable KVM huge page backing support + +RH-Author: David Hildenbrand +Message-id: <20180806141842.23963-3-david@redhat.com> +Patchwork-id: 81645 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 2/3] s390x: Enable KVM huge page backing support +Bugzilla: 1610906 +RH-Acked-by: Thomas Huth +RH-Acked-by: Cornelia Huck +RH-Acked-by: Paolo Bonzini + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1610906 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=17624600 +Upstream: N/A + +Kernel part is in kvm/next, scheduled for 4.19. Patch has been reviewed +upstream but cannot get picked up yet due to the outstanding linux +header sync. Conflict to upstream patch: We have no units.h, therefore +we have to unfold "4*KiB" and "1*MiB". + +QEMU has had huge page support for a longer time already, but KVM +memory management under s390x needed some changes to work with huge +backings. + +Now that we have support, let's enable it if requested and +available. Otherwise we now properly tell the user if there is no +support and back out instead of failing to run the VM later on. + +Signed-off-by: Janosch Frank +Reviewed-by: David Hildenbrand +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/kvm.c | 34 ++++++++++++++++++++++++++++++++-- + 1 file changed, 32 insertions(+), 2 deletions(-) + +diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c +index fbccceb..3474310a9 100644 +--- a/target/s390x/kvm.c ++++ b/target/s390x/kvm.c +@@ -34,6 +34,7 @@ + #include "qapi/error.h" + #include "qemu/error-report.h" + #include "qemu/timer.h" ++#include "qemu/mmap-alloc.h" + #include "sysemu/sysemu.h" + #include "sysemu/hw_accel.h" + #include "hw/hw.h" +@@ -140,6 +141,7 @@ static int cap_mem_op; + static int cap_s390_irq; + static int cap_ri; + static int cap_gs; ++static int cap_hpage_1m; + + static int active_cmma; + +@@ -221,9 +223,9 @@ static void kvm_s390_enable_cmma(void) + .attr = KVM_S390_VM_MEM_ENABLE_CMMA, + }; + +- if (mem_path) { ++ if (cap_hpage_1m) { + warn_report("CMM will not be enabled because it is not " +- "compatible with hugetlbfs."); ++ "compatible with huge memory backings."); + return; + } + rc = kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr); +@@ -282,10 +284,38 @@ void kvm_s390_crypto_reset(void) + } + } + ++static int kvm_s390_configure_mempath_backing(KVMState *s) ++{ ++ size_t path_psize = qemu_mempath_getpagesize(mem_path); ++ ++ if (path_psize == 4 * 1024) { ++ return 0; ++ } ++ ++ if (path_psize != 1024 * 1024) { ++ error_report("Memory backing with 2G pages was specified, " ++ "but KVM does not support this memory backing"); ++ return -EINVAL; ++ } ++ ++ if (kvm_vm_enable_cap(s, KVM_CAP_S390_HPAGE_1M, 0)) { ++ error_report("Memory backing with 1M pages was specified, " ++ "but KVM does not support this memory backing"); ++ return -EINVAL; ++ } ++ ++ cap_hpage_1m = 1; ++ return 0; ++} ++ + int kvm_arch_init(MachineState *ms, KVMState *s) + { + MachineClass *mc = MACHINE_GET_CLASS(ms); + ++ if (mem_path && kvm_s390_configure_mempath_backing(s)) { ++ return -EINVAL; ++ } ++ + mc->default_cpu_type = S390_CPU_TYPE_NAME("host"); + cap_sync_regs = kvm_check_extension(s, KVM_CAP_SYNC_REGS); + cap_async_pf = kvm_check_extension(s, KVM_CAP_ASYNC_PF); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-Return-specification-exception-for-unimplement.patch b/SOURCES/kvm-s390x-Return-specification-exception-for-unimplement.patch new file mode 100644 index 0000000..4b18a00 --- /dev/null +++ b/SOURCES/kvm-s390x-Return-specification-exception-for-unimplement.patch @@ -0,0 +1,54 @@ +From 35f8992f043687db739ccaf363b3f3686eeba398 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Tue, 22 Jan 2019 17:40:28 +0000 +Subject: [PATCH 10/11] s390x: Return specification exception for unimplemented + diag 308 subcodes + +RH-Author: Thomas Huth +Message-id: <1548178828-21117-2-git-send-email-thuth@redhat.com> +Patchwork-id: 84086 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] s390x: Return specification exception for unimplemented diag 308 subcodes +Bugzilla: 1668261 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Janosch Frank + +The architecture specifies specification exceptions for all +unavailable subcodes. + +The presence of subcodes is indicated by checking some query subcode. +For example 6 will indicate that 3-6 are available. So future systems +might call new subcodes to check for new features. This should not +trigger a hw error, instead we return the architectured specification +exception. + +Signed-off-by: Janosch Frank +Cc: qemu-stable@nongnu.org +Message-Id: <20190111113657.66195-3-frankja@linux.ibm.com> +Reviewed-by: Christian Borntraeger +Reviewed-by: David Hildenbrand +Signed-off-by: Cornelia Huck +(cherry picked from commit 37dbd1f4d4805edcd18d94eb202bb3461b3cd52d) +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/diag.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/target/s390x/diag.c b/target/s390x/diag.c +index a755837..50b58df 100644 +--- a/target/s390x/diag.c ++++ b/target/s390x/diag.c +@@ -182,7 +182,7 @@ out: + } + return; + default: +- hw_error("Unhandled diag308 subcode %" PRIx64, subcode); ++ s390_program_interrupt(env, PGM_SPECIFICATION, ILEN_AUTO, ra); + break; + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-add-RHEL-7.6-machine-type-for-ccw.patch b/SOURCES/kvm-s390x-add-RHEL-7.6-machine-type-for-ccw.patch new file mode 100644 index 0000000..b28af04 --- /dev/null +++ b/SOURCES/kvm-s390x-add-RHEL-7.6-machine-type-for-ccw.patch @@ -0,0 +1,92 @@ +From 50dd601eeaa0d0aa6c082085b48b03351d2f0afa Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Tue, 7 Aug 2018 09:05:53 +0000 +Subject: [PATCH 18/21] s390x: add RHEL 7.6 machine type for ccw + +RH-Author: Cornelia Huck +Message-id: <20180807100554.29643-2-cohuck@redhat.com> +Patchwork-id: 81661 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 1/2] s390x: add RHEL 7.6 machine type for ccw +Bugzilla: 1595718 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Thomas Huth +RH-Acked-by: Jens Freimann + +Straight port of the s390-ccw-virtio-rhel7.6.0 machine from 7.6. +Generic compat defines are already present. + +Signed-off-by: Cornelia Huck +--- + hw/s390x/s390-virtio-ccw.c | 35 ++++++++++++++++++++++++++++++++--- + 1 file changed, 32 insertions(+), 3 deletions(-) + +diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c +index 3956ac3..64e2a3b 100644 +--- a/hw/s390x/s390-virtio-ccw.c ++++ b/hw/s390x/s390-virtio-ccw.c +@@ -671,6 +671,8 @@ bool css_migration_enabled(void) + } \ + type_init(ccw_machine_register_##suffix) + ++#if 0 /* Disabled for Red Hat Enterprise Linux */ ++ + #define CCW_COMPAT_2_11 \ + HW_COMPAT_2_11 \ + {\ +@@ -682,8 +684,6 @@ bool css_migration_enabled(void) + #define CCW_COMPAT_2_10 \ + HW_COMPAT_2_10 + +-#if 0 /* Disabled for Red Hat Enterprise Linux */ +- + #define CCW_COMPAT_2_9 \ + HW_COMPAT_2_9 \ + {\ +@@ -883,14 +883,43 @@ DEFINE_CCW_MACHINE(2_4, "2.4", false); + + #else + ++/* ++ * like CCW_COMPAT_2_11, but includes HW_COMPAT_RHEL7_5 (derived from ++ * HW_COMPAT_2_11 and HW_COMPAT_2_10) instead of HW_COMPAT_2_11 ++ */ ++#define CCW_COMPAT_RHEL7_5 \ ++ HW_COMPAT_RHEL7_5 \ ++ {\ ++ .driver = TYPE_SCLP_EVENT_FACILITY,\ ++ .property = "allow_all_mask_sizes",\ ++ .value = "off",\ ++ }, ++ ++static void ccw_machine_rhel760_instance_options(MachineState *machine) ++{ ++} ++ ++static void ccw_machine_rhel760_class_options(MachineClass *mc) ++{ ++} ++DEFINE_CCW_MACHINE(rhel760, "rhel7.6.0", true); ++ + static void ccw_machine_rhel750_instance_options(MachineState *machine) + { ++ static const S390FeatInit qemu_cpu_feat = { S390_FEAT_LIST_QEMU_V2_11 }; ++ ccw_machine_rhel760_instance_options(machine); ++ ++ /* before 2.12 we emulated the very first z900, and RHEL 7.5 is ++ based on 2.10 */ ++ s390_set_qemu_cpu_model(0x2064, 7, 1, qemu_cpu_feat); + } + + static void ccw_machine_rhel750_class_options(MachineClass *mc) + { ++ ccw_machine_rhel760_class_options(mc); ++ SET_MACHINE_COMPAT(mc, CCW_COMPAT_RHEL7_5); + } +-DEFINE_CCW_MACHINE(rhel750, "rhel7.5.0", true); ++DEFINE_CCW_MACHINE(rhel750, "rhel7.5.0", false); + + #endif + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-ap-base-Adjunct-Processor-AP-object-model.patch b/SOURCES/kvm-s390x-ap-base-Adjunct-Processor-AP-object-model.patch new file mode 100644 index 0000000..d3e8f22 --- /dev/null +++ b/SOURCES/kvm-s390x-ap-base-Adjunct-Processor-AP-object-model.patch @@ -0,0 +1,281 @@ +From 375c8f51c7c5489f81cfd21cb289e44b4965d75d Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Mon, 15 Oct 2018 10:19:29 +0100 +Subject: [PATCH 4/6] s390x/ap: base Adjunct Processor (AP) object model + +RH-Author: Thomas Huth +Message-id: <1539598771-16223-5-git-send-email-thuth@redhat.com> +Patchwork-id: 82695 +O-Subject: [RHEL-8 qemu-kvm PATCH 4/6] s390x/ap: base Adjunct Processor (AP) object model +Bugzilla: 1508142 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Tony Krowiak + +Introduces the base object model for virtualizing AP devices. + +Signed-off-by: Tony Krowiak +Tested-by: Pierre Morel +Acked-by: David Hildenbrand +Reviewed-by: Thomas Huth +Reviewed-by: Halil Pasic +Tested-by: Christian Borntraeger +Message-Id: <20181010170309.12045-5-akrowiak@linux.ibm.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit a51b31535a8ec13997de29b357f7cc1dcd8a7f9c) +Signed-off-by: Danilo C. L. de Paula +--- + MAINTAINERS | 12 +++++++ + hw/s390x/Makefile.objs | 2 ++ + hw/s390x/ap-bridge.c | 78 ++++++++++++++++++++++++++++++++++++++++++++ + hw/s390x/ap-device.c | 38 +++++++++++++++++++++ + hw/s390x/s390-virtio-ccw.c | 4 +++ + include/hw/s390x/ap-bridge.h | 19 +++++++++++ + include/hw/s390x/ap-device.h | 22 +++++++++++++ + 7 files changed, 175 insertions(+) + create mode 100644 hw/s390x/ap-bridge.c + create mode 100644 hw/s390x/ap-device.c + create mode 100644 include/hw/s390x/ap-bridge.h + create mode 100644 include/hw/s390x/ap-device.h + +diff --git a/MAINTAINERS b/MAINTAINERS +index 1e32116..31cf6ff 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1152,6 +1152,18 @@ F: include/hw/s390x/s390-ccw.h + T: git git://github.com/cohuck/qemu.git s390-next + L: qemu-s390x@nongnu.org + ++vfio-ap ++M: Christian Borntraeger ++M: Tony Krowiak ++M: Halil Pasic ++M: Pierre Morel ++S: Supported ++F: hw/s390x/ap-device.c ++F: hw/s390x/ap-bridge.c ++F: include/hw/s390x/ap-device.h ++F: include/hw/s390x/ap-bridge.h ++L: qemu-s390x@nongnu.org ++ + vhost + M: Michael S. Tsirkin + S: Supported +diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs +index dc704b5..655d1ac 100644 +--- a/hw/s390x/Makefile.objs ++++ b/hw/s390x/Makefile.objs +@@ -17,3 +17,5 @@ obj-y += s390-stattrib.o + obj-$(CONFIG_KVM) += s390-skeys-kvm.o + obj-$(CONFIG_KVM) += s390-stattrib-kvm.o + obj-y += s390-ccw.o ++obj-y += ap-device.o ++obj-y += ap-bridge.o +diff --git a/hw/s390x/ap-bridge.c b/hw/s390x/ap-bridge.c +new file mode 100644 +index 0000000..3795d30 +--- /dev/null ++++ b/hw/s390x/ap-bridge.c +@@ -0,0 +1,78 @@ ++/* ++ * ap bridge ++ * ++ * Copyright 2018 IBM Corp. ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or (at ++ * your option) any later version. See the COPYING file in the top-level ++ * directory. ++ */ ++#include "qemu/osdep.h" ++#include "qapi/error.h" ++#include "hw/sysbus.h" ++#include "qemu/bitops.h" ++#include "hw/s390x/ap-bridge.h" ++#include "cpu.h" ++ ++static char *ap_bus_get_dev_path(DeviceState *dev) ++{ ++ /* at most one */ ++ return g_strdup_printf("/1"); ++} ++ ++static void ap_bus_class_init(ObjectClass *oc, void *data) ++{ ++ BusClass *k = BUS_CLASS(oc); ++ ++ k->get_dev_path = ap_bus_get_dev_path; ++ /* More than one ap device does not make sense */ ++ k->max_dev = 1; ++} ++ ++static const TypeInfo ap_bus_info = { ++ .name = TYPE_AP_BUS, ++ .parent = TYPE_BUS, ++ .instance_size = 0, ++ .class_init = ap_bus_class_init, ++}; ++ ++void s390_init_ap(void) ++{ ++ DeviceState *dev; ++ ++ /* If no AP instructions then no need for AP bridge */ ++ if (!s390_has_feat(S390_FEAT_AP)) { ++ return; ++ } ++ ++ /* Create bridge device */ ++ dev = qdev_create(NULL, TYPE_AP_BRIDGE); ++ object_property_add_child(qdev_get_machine(), TYPE_AP_BRIDGE, ++ OBJECT(dev), NULL); ++ qdev_init_nofail(dev); ++ ++ /* Create bus on bridge device */ ++ qbus_create(TYPE_AP_BUS, dev, TYPE_AP_BUS); ++ } ++ ++static void ap_bridge_class_init(ObjectClass *oc, void *data) ++{ ++ DeviceClass *dc = DEVICE_CLASS(oc); ++ ++ set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories); ++} ++ ++static const TypeInfo ap_bridge_info = { ++ .name = TYPE_AP_BRIDGE, ++ .parent = TYPE_SYS_BUS_DEVICE, ++ .instance_size = 0, ++ .class_init = ap_bridge_class_init, ++}; ++ ++static void ap_register(void) ++{ ++ type_register_static(&ap_bridge_info); ++ type_register_static(&ap_bus_info); ++} ++ ++type_init(ap_register) +diff --git a/hw/s390x/ap-device.c b/hw/s390x/ap-device.c +new file mode 100644 +index 0000000..f5ac8db +--- /dev/null ++++ b/hw/s390x/ap-device.c +@@ -0,0 +1,38 @@ ++/* ++ * Adjunct Processor (AP) matrix device ++ * ++ * Copyright 2018 IBM Corp. ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or (at ++ * your option) any later version. See the COPYING file in the top-level ++ * directory. ++ */ ++#include "qemu/osdep.h" ++#include "qemu/module.h" ++#include "qapi/error.h" ++#include "hw/qdev.h" ++#include "hw/s390x/ap-device.h" ++ ++static void ap_class_init(ObjectClass *klass, void *data) ++{ ++ DeviceClass *dc = DEVICE_CLASS(klass); ++ ++ dc->desc = "AP device class"; ++ dc->hotpluggable = false; ++} ++ ++static const TypeInfo ap_device_info = { ++ .name = AP_DEVICE_TYPE, ++ .parent = TYPE_DEVICE, ++ .instance_size = sizeof(APDevice), ++ .class_size = sizeof(DeviceClass), ++ .class_init = ap_class_init, ++ .abstract = true, ++}; ++ ++static void ap_device_register(void) ++{ ++ type_register_static(&ap_device_info); ++} ++ ++type_init(ap_device_register) +diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c +index bf039a1..43a8213 100644 +--- a/hw/s390x/s390-virtio-ccw.c ++++ b/hw/s390x/s390-virtio-ccw.c +@@ -32,6 +32,7 @@ + #include "ipl.h" + #include "hw/s390x/s390-virtio-ccw.h" + #include "hw/s390x/css-bridge.h" ++#include "hw/s390x/ap-bridge.h" + #include "migration/register.h" + #include "cpu_models.h" + #include "hw/nmi.h" +@@ -305,6 +306,9 @@ static void ccw_init(MachineState *machine) + /* init the SIGP facility */ + s390_init_sigp(); + ++ /* create AP bridge and bus(es) */ ++ s390_init_ap(); ++ + /* get a BUS */ + css_bus = virtual_css_bus_init(); + s390_init_ipl_dev(machine->kernel_filename, machine->kernel_cmdline, +diff --git a/include/hw/s390x/ap-bridge.h b/include/hw/s390x/ap-bridge.h +new file mode 100644 +index 0000000..470e439 +--- /dev/null ++++ b/include/hw/s390x/ap-bridge.h +@@ -0,0 +1,19 @@ ++/* ++ * ap bridge ++ * ++ * Copyright 2018 IBM Corp. ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or (at ++ * your option) any later version. See the COPYING file in the top-level ++ * directory. ++ */ ++ ++#ifndef HW_S390X_AP_BRIDGE_H ++#define HW_S390X_AP_BRIDGE_H ++ ++#define TYPE_AP_BRIDGE "ap-bridge" ++#define TYPE_AP_BUS "ap-bus" ++ ++void s390_init_ap(void); ++ ++#endif +diff --git a/include/hw/s390x/ap-device.h b/include/hw/s390x/ap-device.h +new file mode 100644 +index 0000000..765e908 +--- /dev/null ++++ b/include/hw/s390x/ap-device.h +@@ -0,0 +1,22 @@ ++/* ++ * Adjunct Processor (AP) matrix device interfaces ++ * ++ * Copyright 2018 IBM Corp. ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or (at ++ * your option) any later version. See the COPYING file in the top-level ++ * directory. ++ */ ++#ifndef HW_S390X_AP_DEVICE_H ++#define HW_S390X_AP_DEVICE_H ++ ++#define AP_DEVICE_TYPE "ap-device" ++ ++typedef struct APDevice { ++ DeviceState parent_obj; ++} APDevice; ++ ++#define AP_DEVICE(obj) \ ++ OBJECT_CHECK(APDevice, (obj), AP_DEVICE_TYPE) ++ ++#endif /* HW_S390X_AP_DEVICE_H */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-cpumodel-Set-up-CPU-model-for-AP-device-suppor.patch b/SOURCES/kvm-s390x-cpumodel-Set-up-CPU-model-for-AP-device-suppor.patch new file mode 100644 index 0000000..8b6c2b4 --- /dev/null +++ b/SOURCES/kvm-s390x-cpumodel-Set-up-CPU-model-for-AP-device-suppor.patch @@ -0,0 +1,148 @@ +From f52debc0934c9f1fa9373876bbbad3c6b314bac1 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Mon, 15 Oct 2018 10:19:27 +0100 +Subject: [PATCH 2/6] s390x/cpumodel: Set up CPU model for AP device support + +RH-Author: Thomas Huth +Message-id: <1539598771-16223-3-git-send-email-thuth@redhat.com> +Patchwork-id: 82694 +O-Subject: [RHEL-8 qemu-kvm PATCH 2/6] s390x/cpumodel: Set up CPU model for AP device support +Bugzilla: 1508142 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Tony Krowiak + +A new CPU model feature and two new CPU model facilities are +introduced to support AP devices for a KVM guest. + +CPU model features: + +1. The S390_FEAT_AP CPU model feature indicates whether AP + instructions are available to the guest. This feature will + be enabled only if the AP instructions are available on the + linux host as determined by the availability of the + KVM_S390_VM_CRYPTO_ENABLE_APIE VM attribute which is exposed + by KVM only if the AP instructions are available on the + host. + + This feature must be turned on from userspace to execute AP + instructions on the KVM guest. The QEMU command line to turn + this feature on looks something like this: + + qemu-system-s390x ... -cpu xxx,ap=on ... + + This feature will be supported for zEC12 and newer CPU models. + The feature will not be supported for older models because + there are few older systems on which to test and the older + crypto cards will be going out of service in the relatively + near future. + +CPU model facilities: + +1. The S390_FEAT_AP_QUERY_CONFIG_INFO feature indicates whether the + AP Query Configuration Information (QCI) facility is available + to the guest as determined by whether the facility is available + on the host. This feature will be exposed by KVM only if the + QCI facility is installed on the host. + +2. The S390_FEAT_AP_FACILITY_TEST feature indicates whether the AP + Facility Test (APFT) facility is available to the guest as + determined by whether the facility is available on the host. + This feature will be exposed by KVM only if APFT is installed + on the host. + +Signed-off-by: Tony Krowiak +Tested-by: Pierre Morel +Reviewed-by: David Hildenbrand +Reviewed-by: Halil Pasic +Reviewed-by: Christian Borntraeger +Tested-by: Christian Borntraeger +Message-Id: <20181010170309.12045-3-akrowiak@linux.ibm.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit c5cd17afddda89376712b315a41ede96b034e4c2) +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/cpu_features.c | 3 +++ + target/s390x/cpu_features_def.h | 3 +++ + target/s390x/cpu_models.c | 2 ++ + target/s390x/gen-features.c | 3 +++ + 4 files changed, 11 insertions(+) + +diff --git a/target/s390x/cpu_features.c b/target/s390x/cpu_features.c +index e05e6aa..0fbee27 100644 +--- a/target/s390x/cpu_features.c ++++ b/target/s390x/cpu_features.c +@@ -40,8 +40,10 @@ static const S390FeatDef s390_features[] = { + FEAT_INIT("srs", S390_FEAT_TYPE_STFL, 9, "Sense-running-status facility"), + FEAT_INIT("csske", S390_FEAT_TYPE_STFL, 10, "Conditional-SSKE facility"), + FEAT_INIT("ctop", S390_FEAT_TYPE_STFL, 11, "Configuration-topology facility"), ++ FEAT_INIT("apqci", S390_FEAT_TYPE_STFL, 12, "Query AP Configuration Information facility"), + FEAT_INIT("ipter", S390_FEAT_TYPE_STFL, 13, "IPTE-range facility"), + FEAT_INIT("nonqks", S390_FEAT_TYPE_STFL, 14, "Nonquiescing key-setting facility"), ++ FEAT_INIT("apft", S390_FEAT_TYPE_STFL, 15, "AP Facilities Test facility"), + FEAT_INIT("etf2", S390_FEAT_TYPE_STFL, 16, "Extended-translation facility 2"), + FEAT_INIT("msa-base", S390_FEAT_TYPE_STFL, 17, "Message-security-assist facility (excluding subfunctions)"), + FEAT_INIT("ldisp", S390_FEAT_TYPE_STFL, 18, "Long-displacement facility"), +@@ -130,6 +132,7 @@ static const S390FeatDef s390_features[] = { + + FEAT_INIT_MISC("dateh2", "DAT-enhancement facility 2"), + FEAT_INIT_MISC("cmm", "Collaborative-memory-management facility"), ++ FEAT_INIT_MISC("ap", "AP instructions installed"), + + FEAT_INIT("plo-cl", S390_FEAT_TYPE_PLO, 0, "PLO Compare and load (32 bit in general registers)"), + FEAT_INIT("plo-clg", S390_FEAT_TYPE_PLO, 1, "PLO Compare and load (64 bit in parameter list)"), +diff --git a/target/s390x/cpu_features_def.h b/target/s390x/cpu_features_def.h +index ac2c947..5fc7e7b 100644 +--- a/target/s390x/cpu_features_def.h ++++ b/target/s390x/cpu_features_def.h +@@ -27,8 +27,10 @@ typedef enum { + S390_FEAT_SENSE_RUNNING_STATUS, + S390_FEAT_CONDITIONAL_SSKE, + S390_FEAT_CONFIGURATION_TOPOLOGY, ++ S390_FEAT_AP_QUERY_CONFIG_INFO, + S390_FEAT_IPTE_RANGE, + S390_FEAT_NONQ_KEY_SETTING, ++ S390_FEAT_AP_FACILITIES_TEST, + S390_FEAT_EXTENDED_TRANSLATION_2, + S390_FEAT_MSA, + S390_FEAT_LONG_DISPLACEMENT, +@@ -119,6 +121,7 @@ typedef enum { + /* Misc */ + S390_FEAT_DAT_ENH_2, + S390_FEAT_CMM, ++ S390_FEAT_AP, + + /* PLO */ + S390_FEAT_PLO_CL, +diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c +index 0b5d271..3856104 100644 +--- a/target/s390x/cpu_models.c ++++ b/target/s390x/cpu_models.c +@@ -774,6 +774,8 @@ static void check_consistency(const S390CPUModel *model) + { S390_FEAT_PRNO_TRNG_QRTCR, S390_FEAT_MSA_EXT_5 }, + { S390_FEAT_PRNO_TRNG, S390_FEAT_MSA_EXT_5 }, + { S390_FEAT_SIE_KSS, S390_FEAT_SIE_F2 }, ++ { S390_FEAT_AP_QUERY_CONFIG_INFO, S390_FEAT_AP }, ++ { S390_FEAT_AP_FACILITIES_TEST, S390_FEAT_AP }, + }; + int i; + +diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c +index 5af042c..7302269 100644 +--- a/target/s390x/gen-features.c ++++ b/target/s390x/gen-features.c +@@ -447,6 +447,9 @@ static uint16_t full_GEN12_GA1[] = { + S390_FEAT_ADAPTER_INT_SUPPRESSION, + S390_FEAT_EDAT_2, + S390_FEAT_SIDE_EFFECT_ACCESS_ESOP2, ++ S390_FEAT_AP_QUERY_CONFIG_INFO, ++ S390_FEAT_AP_FACILITIES_TEST, ++ S390_FEAT_AP, + }; + + static uint16_t full_GEN12_GA2[] = { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-cpumodel-default-enable-bpb-and-ppa15-for-z196.patch b/SOURCES/kvm-s390x-cpumodel-default-enable-bpb-and-ppa15-for-z196.patch new file mode 100644 index 0000000..d41fcf0 --- /dev/null +++ b/SOURCES/kvm-s390x-cpumodel-default-enable-bpb-and-ppa15-for-z196.patch @@ -0,0 +1,66 @@ +From 9269bf16c4ca06496840473ec842dfcbc2d9020f Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Tue, 7 Aug 2018 09:05:54 +0000 +Subject: [PATCH 19/21] s390x/cpumodel: default enable bpb and ppa15 for z196 + and later + +RH-Author: Cornelia Huck +Message-id: <20180807100554.29643-3-cohuck@redhat.com> +Patchwork-id: 81660 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 2/2] s390x/cpumodel: default enable bpb and ppa15 for z196 and later +Bugzilla: 1595718 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Thomas Huth +RH-Acked-by: Jens Freimann + +Upstream: downstream version of 8727315111 ("s390x/cpumodel: default + enable bpb and ppa15 for z196 and later"); downstream does + not have the upstream machine types, instead we need to + turn off the bits for the RHEL 7.5 machine + +Most systems and host kernels provide the necessary building blocks for +bpb and ppa15. We can reverse the logic and default enable those +features, while still allowing to disable it via cpu model. + +So let us add bpb and ppa15 to z196 and later default CPU model for the +qemu rhel7.6.0 machine. (like -cpu z13). Older machine types (i.e. +s390-ccw-virtio-rhel7.5.0) will retain the old value and not provide those +bits in the default model. + +Signed-off-by: Cornelia Huck +--- + hw/s390x/s390-virtio-ccw.c | 4 ++++ + target/s390x/gen-features.c | 2 ++ + 2 files changed, 6 insertions(+) + +diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c +index 64e2a3b..bf039a1 100644 +--- a/hw/s390x/s390-virtio-ccw.c ++++ b/hw/s390x/s390-virtio-ccw.c +@@ -912,6 +912,10 @@ static void ccw_machine_rhel750_instance_options(MachineState *machine) + /* before 2.12 we emulated the very first z900, and RHEL 7.5 is + based on 2.10 */ + s390_set_qemu_cpu_model(0x2064, 7, 1, qemu_cpu_feat); ++ ++ /* bpb and ppa15 were only in the full model in RHEL 7.5 */ ++ s390_cpudef_featoff_greater(11, 1, S390_FEAT_PPA15); ++ s390_cpudef_featoff_greater(11, 1, S390_FEAT_BPB); + } + + static void ccw_machine_rhel750_class_options(MachineClass *mc) +diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c +index 0cdbc15..6626b6f 100644 +--- a/target/s390x/gen-features.c ++++ b/target/s390x/gen-features.c +@@ -512,6 +512,8 @@ static uint16_t default_GEN11_GA1[] = { + S390_FEAT_IPTE_RANGE, + S390_FEAT_ACCESS_EXCEPTION_FS_INDICATION, + S390_FEAT_GROUP_MSA_EXT_4, ++ S390_FEAT_PPA15, ++ S390_FEAT_BPB, + }; + + #define default_GEN11_GA2 EmptyFeat +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-kvm-add-etoken-facility.patch b/SOURCES/kvm-s390x-kvm-add-etoken-facility.patch new file mode 100644 index 0000000..daf9cba --- /dev/null +++ b/SOURCES/kvm-s390x-kvm-add-etoken-facility.patch @@ -0,0 +1,190 @@ +From 65d1d181ba7236d4d537faa545afc6d72c3db091 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Thu, 9 Aug 2018 10:15:09 +0000 +Subject: [PATCH 21/21] s390x/kvm: add etoken facility + +RH-Author: Thomas Huth +Message-id: <1533813309-9643-3-git-send-email-thuth@redhat.com> +Patchwork-id: 81687 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 2/2] s390x/kvm: add etoken facility +Bugzilla: 1612938 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +Provide the etoken facility. We need to handle cpu model, migration and +clear reset. + +Signed-off-by: Christian Borntraeger +Acked-by: Janosch Frank +Signed-off-by: Thomas Huth +--- + target/s390x/cpu.h | 3 +++ + target/s390x/cpu_features.c | 3 ++- + target/s390x/cpu_features_def.h | 3 ++- + target/s390x/gen-features.c | 3 ++- + target/s390x/kvm.c | 11 +++++++++++ + target/s390x/machine.c | 20 +++++++++++++++++++- + 6 files changed, 39 insertions(+), 4 deletions(-) + +diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h +index 3ee40f0..86d08fa 100644 +--- a/target/s390x/cpu.h ++++ b/target/s390x/cpu.h +@@ -2,6 +2,7 @@ + * S/390 virtual CPU header + * + * Copyright (c) 2009 Ulrich Hecht ++ * Copyright IBM Corp. 2012, 2018 + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public +@@ -68,6 +69,8 @@ struct CPUS390XState { + uint32_t aregs[16]; /* access registers */ + uint8_t riccb[64]; /* runtime instrumentation control */ + uint64_t gscb[4]; /* guarded storage control */ ++ uint64_t etoken; /* etoken */ ++ uint64_t etoken_extension; /* etoken extension */ + + /* Fields up to this point are not cleared by initial CPU reset */ + struct {} start_initial_reset_fields; +diff --git a/target/s390x/cpu_features.c b/target/s390x/cpu_features.c +index 3b9e274..e05e6aa 100644 +--- a/target/s390x/cpu_features.c ++++ b/target/s390x/cpu_features.c +@@ -1,7 +1,7 @@ + /* + * CPU features/facilities for s390x + * +- * Copyright 2016 IBM Corp. ++ * Copyright IBM Corp. 2016, 2018 + * + * Author(s): David Hildenbrand + * +@@ -106,6 +106,7 @@ static const S390FeatDef s390_features[] = { + FEAT_INIT("irbm", S390_FEAT_TYPE_STFL, 145, "Insert-reference-bits-multiple facility"), + FEAT_INIT("msa8-base", S390_FEAT_TYPE_STFL, 146, "Message-security-assist-extension-8 facility (excluding subfunctions)"), + FEAT_INIT("cmmnt", S390_FEAT_TYPE_STFL, 147, "CMM: ESSA-enhancement (no translate) facility"), ++ FEAT_INIT("etoken", S390_FEAT_TYPE_STFL, 156, "Etoken facility"), + + /* SCLP SCCB Byte 80 - 98 (bit numbers relative to byte-80) */ + FEAT_INIT("gsls", S390_FEAT_TYPE_SCLP_CONF_CHAR, 40, "SIE: Guest-storage-limit-suppression facility"), +diff --git a/target/s390x/cpu_features_def.h b/target/s390x/cpu_features_def.h +index 7c5915c..ac2c947 100644 +--- a/target/s390x/cpu_features_def.h ++++ b/target/s390x/cpu_features_def.h +@@ -1,7 +1,7 @@ + /* + * CPU features/facilities for s390 + * +- * Copyright 2016 IBM Corp. ++ * Copyright IBM Corp. 2016, 2018 + * + * Author(s): Michael Mueller + * David Hildenbrand +@@ -93,6 +93,7 @@ typedef enum { + S390_FEAT_INSERT_REFERENCE_BITS_MULT, + S390_FEAT_MSA_EXT_8, + S390_FEAT_CMM_NT, ++ S390_FEAT_ETOKEN, + + /* Sclp Conf Char */ + S390_FEAT_SIE_GSLS, +diff --git a/target/s390x/gen-features.c b/target/s390x/gen-features.c +index 6626b6f..5af042c 100644 +--- a/target/s390x/gen-features.c ++++ b/target/s390x/gen-features.c +@@ -1,7 +1,7 @@ + /* + * S390 feature list generator + * +- * Copyright 2016 IBM Corp. ++ * Copyright IBM Corp. 2016, 2018 + * + * Author(s): Michael Mueller + * David Hildenbrand +@@ -471,6 +471,7 @@ static uint16_t full_GEN14_GA1[] = { + S390_FEAT_GROUP_MSA_EXT_7, + S390_FEAT_GROUP_MSA_EXT_8, + S390_FEAT_CMM_NT, ++ S390_FEAT_ETOKEN, + S390_FEAT_HPMA2, + S390_FEAT_SIE_KSS, + S390_FEAT_GROUP_MULTIPLE_EPOCH_PTFF, +diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c +index 3474310a9..2a63499 100644 +--- a/target/s390x/kvm.c ++++ b/target/s390x/kvm.c +@@ -524,6 +524,12 @@ int kvm_arch_put_registers(CPUState *cs, int level) + cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_BPBC; + } + ++ if (can_sync_regs(cs, KVM_SYNC_ETOKEN)) { ++ cs->kvm_run->s.regs.etoken = env->etoken; ++ cs->kvm_run->s.regs.etoken_extension = env->etoken_extension; ++ cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_ETOKEN; ++ } ++ + /* Finally the prefix */ + if (can_sync_regs(cs, KVM_SYNC_PREFIX)) { + cs->kvm_run->s.regs.prefix = env->psa; +@@ -638,6 +644,11 @@ int kvm_arch_get_registers(CPUState *cs) + env->bpbc = cs->kvm_run->s.regs.bpbc; + } + ++ if (can_sync_regs(cs, KVM_SYNC_ETOKEN)) { ++ env->etoken = cs->kvm_run->s.regs.etoken; ++ env->etoken_extension = cs->kvm_run->s.regs.etoken_extension; ++ } ++ + /* pfault parameters */ + if (can_sync_regs(cs, KVM_SYNC_PFAULT)) { + env->pfault_token = cs->kvm_run->s.regs.pft; +diff --git a/target/s390x/machine.c b/target/s390x/machine.c +index 84b4928..8421deb 100644 +--- a/target/s390x/machine.c ++++ b/target/s390x/machine.c +@@ -1,7 +1,7 @@ + /* + * S390x machine definitions and functions + * +- * Copyright IBM Corp. 2014 ++ * Copyright IBM Corp. 2014, 2018 + * + * Authors: + * Thomas Huth +@@ -210,6 +210,23 @@ const VMStateDescription vmstate_bpbc = { + } + }; + ++static bool etoken_needed(void *opaque) ++{ ++ return s390_has_feat(S390_FEAT_ETOKEN); ++} ++ ++const VMStateDescription vmstate_etoken = { ++ .name = "cpu/etoken", ++ .version_id = 1, ++ .minimum_version_id = 1, ++ .needed = etoken_needed, ++ .fields = (VMStateField[]) { ++ VMSTATE_UINT64(env.etoken, S390CPU), ++ VMSTATE_UINT64(env.etoken_extension, S390CPU), ++ VMSTATE_END_OF_LIST() ++ } ++}; ++ + const VMStateDescription vmstate_s390_cpu = { + .name = "cpu", + .post_load = cpu_post_load, +@@ -245,6 +262,7 @@ const VMStateDescription vmstate_s390_cpu = { + &vmstate_exval, + &vmstate_gscb, + &vmstate_bpbc, ++ &vmstate_etoken, + NULL + }, + }; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-kvm-enable-AP-instruction-interpretation-for-g.patch b/SOURCES/kvm-s390x-kvm-enable-AP-instruction-interpretation-for-g.patch new file mode 100644 index 0000000..d6ddbe9 --- /dev/null +++ b/SOURCES/kvm-s390x-kvm-enable-AP-instruction-interpretation-for-g.patch @@ -0,0 +1,89 @@ +From 89c813343ab05381d7def3b67120a4480490add7 Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Mon, 15 Oct 2018 10:19:28 +0100 +Subject: [PATCH 3/6] s390x/kvm: enable AP instruction interpretation for guest + +RH-Author: Thomas Huth +Message-id: <1539598771-16223-4-git-send-email-thuth@redhat.com> +Patchwork-id: 82697 +O-Subject: [RHEL-8 qemu-kvm PATCH 3/6] s390x/kvm: enable AP instruction interpretation for guest +Bugzilla: 1508142 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Tony Krowiak + +Let's use the KVM_SET_DEVICE_ATTR ioctl to enable hardware +interpretation of AP instructions executed on the guest. +If the S390_FEAT_AP feature is switched on for the guest, +AP instructions must be interpreted by default; otherwise, +they will be intercepted. + +This attribute setting may be overridden by a device. For example, +a device may want to provide AP instructions to the guest (i.e., +S390_FEAT_AP turned on), but it may want to emulate them. In this +case, the AP instructions executed on the guest must be +intercepted; so when the device is realized, it must disable +interpretation. + +Signed-off-by: Tony Krowiak +Tested-by: Pierre Morel +Reviewed-by: David Hildenbrand +Reviewed-by: Thomas Huth +Reviewed-by: Christian Borntraeger +Acked-by: Halil Pasic +Tested-by: Christian Borntraeger +Message-Id: <20181010170309.12045-4-akrowiak@linux.ibm.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 1d7db85b61cb9888b8ed8c8923343b468405b7a0) +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/kvm.c | 19 +++++++++++++++++++ + 1 file changed, 19 insertions(+) + +diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c +index 2a63499..4cb3499 100644 +--- a/target/s390x/kvm.c ++++ b/target/s390x/kvm.c +@@ -2298,11 +2298,26 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) + error_setg(errp, "KVM: host CPU model could not be identified"); + return; + } ++ /* for now, we can only provide the AP feature with HW support */ ++ if (kvm_vm_check_attr(kvm_state, KVM_S390_VM_CRYPTO, ++ KVM_S390_VM_CRYPTO_ENABLE_APIE)) { ++ set_bit(S390_FEAT_AP, model->features); ++ } + /* strip of features that are not part of the maximum model */ + bitmap_and(model->features, model->features, model->def->full_feat, + S390_FEAT_MAX); + } + ++static void kvm_s390_configure_apie(bool interpret) ++{ ++ uint64_t attr = interpret ? KVM_S390_VM_CRYPTO_ENABLE_APIE : ++ KVM_S390_VM_CRYPTO_DISABLE_APIE; ++ ++ if (kvm_vm_check_attr(kvm_state, KVM_S390_VM_CRYPTO, attr)) { ++ kvm_s390_set_attr(attr); ++ } ++} ++ + void kvm_s390_apply_cpu_model(const S390CPUModel *model, Error **errp) + { + struct kvm_s390_vm_cpu_processor prop = { +@@ -2360,6 +2375,10 @@ void kvm_s390_apply_cpu_model(const S390CPUModel *model, Error **errp) + if (test_bit(S390_FEAT_CMM, model->features)) { + kvm_s390_enable_cmma(); + } ++ ++ if (test_bit(S390_FEAT_AP, model->features)) { ++ kvm_s390_configure_apie(true); ++ } + } + + void kvm_s390_restart_interrupt(S390CPU *cpu) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-kvm-pass-values-instead-of-pointers-to-kvm_s39.patch b/SOURCES/kvm-s390x-kvm-pass-values-instead-of-pointers-to-kvm_s39.patch new file mode 100644 index 0000000..c35d2ad --- /dev/null +++ b/SOURCES/kvm-s390x-kvm-pass-values-instead-of-pointers-to-kvm_s39.patch @@ -0,0 +1,129 @@ +From 3aa8ecd6ba8715d04ce13568c609f023fea1b638 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:04 +0000 +Subject: [PATCH 02/22] s390x/kvm: pass values instead of pointers to + kvm_s390_set_clock_*() + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-3-david@redhat.com> +Patchwork-id: 83746 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 02/12] s390x/kvm: pass values instead of pointers to kvm_s390_set_clock_*() +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +We are going to factor out the TOD into a separate device and use const +pointers for device class functions where possible. We are passing right +now ordinary pointers that should never be touched when setting the TOD. +Let's just pass the values directly. + +Note that s390_set_clock() will be removed in a follow-on patch and +therefore its calling convention is not changed. + +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-3-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 4ab6a1feac0a142045d3b7bdbb8182a99c0b8980) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/cpu.c | 4 ++-- + target/s390x/kvm-stub.c | 4 ++-- + target/s390x/kvm.c | 12 ++++++------ + target/s390x/kvm_s390x.h | 4 ++-- + 4 files changed, 12 insertions(+), 12 deletions(-) + +diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c +index c2b775f..1f590d1 100644 +--- a/target/s390x/cpu.c ++++ b/target/s390x/cpu.c +@@ -414,9 +414,9 @@ int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) + int r = 0; + + if (kvm_enabled()) { +- r = kvm_s390_set_clock_ext(tod_high, tod_low); ++ r = kvm_s390_set_clock_ext(*tod_high, *tod_low); + if (r == -ENXIO) { +- return kvm_s390_set_clock(tod_high, tod_low); ++ return kvm_s390_set_clock(*tod_high, *tod_low); + } + } + /* Fixme TCG */ +diff --git a/target/s390x/kvm-stub.c b/target/s390x/kvm-stub.c +index 29b1054..bf7795e 100644 +--- a/target/s390x/kvm-stub.c ++++ b/target/s390x/kvm-stub.c +@@ -60,12 +60,12 @@ int kvm_s390_get_clock_ext(uint8_t *tod_high, uint64_t *tod_low) + return -ENOSYS; + } + +-int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) ++int kvm_s390_set_clock(uint8_t tod_high, uint64_t tod_low) + { + return -ENOSYS; + } + +-int kvm_s390_set_clock_ext(uint8_t *tod_high, uint64_t *tod_low) ++int kvm_s390_set_clock_ext(uint8_t tod_high, uint64_t tod_low) + { + return -ENOSYS; + } +diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c +index 4cb3499..1cf117b 100644 +--- a/target/s390x/kvm.c ++++ b/target/s390x/kvm.c +@@ -708,13 +708,13 @@ int kvm_s390_get_clock_ext(uint8_t *tod_high, uint64_t *tod_low) + return r; + } + +-int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) ++int kvm_s390_set_clock(uint8_t tod_high, uint64_t tod_low) + { + int r; + struct kvm_device_attr attr = { + .group = KVM_S390_VM_TOD, + .attr = KVM_S390_VM_TOD_LOW, +- .addr = (uint64_t)tod_low, ++ .addr = (uint64_t)&tod_low, + }; + + r = kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr); +@@ -723,15 +723,15 @@ int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) + } + + attr.attr = KVM_S390_VM_TOD_HIGH; +- attr.addr = (uint64_t)tod_high; ++ attr.addr = (uint64_t)&tod_high; + return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attr); + } + +-int kvm_s390_set_clock_ext(uint8_t *tod_high, uint64_t *tod_low) ++int kvm_s390_set_clock_ext(uint8_t tod_high, uint64_t tod_low) + { + struct kvm_s390_vm_tod_clock gtod = { +- .epoch_idx = *tod_high, +- .tod = *tod_low, ++ .epoch_idx = tod_high, ++ .tod = tod_low, + }; + struct kvm_device_attr attr = { + .group = KVM_S390_VM_TOD, +diff --git a/target/s390x/kvm_s390x.h b/target/s390x/kvm_s390x.h +index c383bf4..36eb34b 100644 +--- a/target/s390x/kvm_s390x.h ++++ b/target/s390x/kvm_s390x.h +@@ -25,8 +25,8 @@ int kvm_s390_get_ri(void); + int kvm_s390_get_gs(void); + int kvm_s390_get_clock(uint8_t *tod_high, uint64_t *tod_clock); + int kvm_s390_get_clock_ext(uint8_t *tod_high, uint64_t *tod_clock); +-int kvm_s390_set_clock(uint8_t *tod_high, uint64_t *tod_clock); +-int kvm_s390_set_clock_ext(uint8_t *tod_high, uint64_t *tod_clock); ++int kvm_s390_set_clock(uint8_t tod_high, uint64_t tod_clock); ++int kvm_s390_set_clock_ext(uint8_t tod_high, uint64_t tod_clock); + void kvm_s390_enable_css_support(S390CPU *cpu); + int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch, + int vq, bool assign); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-sclp-fix-maxram-calculation.patch b/SOURCES/kvm-s390x-sclp-fix-maxram-calculation.patch new file mode 100644 index 0000000..bacc1cf --- /dev/null +++ b/SOURCES/kvm-s390x-sclp-fix-maxram-calculation.patch @@ -0,0 +1,51 @@ +From 619064cd5e5f275a0d61fcd05855b1598c2d654b Mon Sep 17 00:00:00 2001 +From: Cornelia Huck +Date: Wed, 1 Aug 2018 11:09:14 +0200 +Subject: [PATCH 267/268] s390x/sclp: fix maxram calculation + +RH-Author: Cornelia Huck +Message-id: <20180801110914.17729-1-cohuck@redhat.com> +Patchwork-id: 81564 +O-Subject: [RHEL-7.6 qemu-kvm-ma PATCH] s390x/sclp: fix maxram calculation +Bugzilla: 1595740 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Thomas Huth +RH-Acked-by: Miroslav Rezanina + +From: Christian Borntraeger + +We clamp down ram_size to match the sclp increment size. We do +not do the same for maxram_size, which means for large guests +with some sizes (e.g. -m 50000) maxram_size differs from ram_size. +This can break other code (e.g. CMMA migration) which uses maxram_size +to calculate the number of pages and then throws some errors. + +Fixes: 82fab5c5b90e468f3e9d54c ("s390x/sclp: remove memory hotplug support") +Signed-off-by: Christian Borntraeger +CC: qemu-stable@nongnu.org +CC: David Hildenbrand +Message-Id: <1532959766-53343-1-git-send-email-borntraeger@de.ibm.com> +Reviewed-by: David Hildenbrand +Signed-off-by: Cornelia Huck +(cherry picked from commit 408e5ace517ff18c9c7cd918fc93358162e6e26d) +Signed-off-by: Cornelia Huck +Signed-off-by: Miroslav Rezanina +--- + hw/s390x/sclp.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c +index 047d577..2abdb62 100644 +--- a/hw/s390x/sclp.c ++++ b/hw/s390x/sclp.c +@@ -319,6 +319,7 @@ static void sclp_memory_init(SCLPDevice *sclp) + initial_mem = initial_mem >> increment_size << increment_size; + + machine->ram_size = initial_mem; ++ machine->maxram_size = initial_mem; + /* let's propagate the changed ram size into the global variable. */ + ram_size = initial_mem; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-SET-CLOCK-COMPARATOR-can-clear-CKC-interru.patch b/SOURCES/kvm-s390x-tcg-SET-CLOCK-COMPARATOR-can-clear-CKC-interru.patch new file mode 100644 index 0000000..1311935 --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-SET-CLOCK-COMPARATOR-can-clear-CKC-interru.patch @@ -0,0 +1,52 @@ +From 91995f02cd4c100d26edd6df3dae0813b8685c02 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:08 +0000 +Subject: [PATCH 06/22] s390x/tcg: SET CLOCK COMPARATOR can clear CKC + interrupts + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-7-david@redhat.com> +Patchwork-id: 83750 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 06/12] s390x/tcg: SET CLOCK COMPARATOR can clear CKC interrupts +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +Let's stop the timer and delete any pending CKC IRQ before doing +anything else. + +While at it, add a comment why the check for ckc == -1ULL is needed. + +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-7-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 345f1ab96e8279a537f32ae7447296d23308c7d1) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/misc_helper.c | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c +index 4f675c7..cada7af 100644 +--- a/target/s390x/misc_helper.c ++++ b/target/s390x/misc_helper.c +@@ -155,6 +155,13 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time) + { + S390TODState *td = s390_get_todstate(); + ++ /* stop the timer and remove pending CKC IRQs */ ++ timer_del(env->tod_timer); ++ qemu_mutex_lock_iothread(); ++ env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR; ++ qemu_mutex_unlock_iothread(); ++ ++ /* the tod has to exceed the ckc, this can never happen if ckc is all 1's */ + if (time == -1ULL) { + return; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-avoid-overflows-in-time2tod-tod2time.patch b/SOURCES/kvm-s390x-tcg-avoid-overflows-in-time2tod-tod2time.patch new file mode 100644 index 0000000..1ab3207 --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-avoid-overflows-in-time2tod-tod2time.patch @@ -0,0 +1,57 @@ +From 45383e6748287c088d09d7f94dfbac76c38e7329 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:03 +0000 +Subject: [PATCH 01/22] s390x/tcg: avoid overflows in time2tod/tod2time + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-2-david@redhat.com> +Patchwork-id: 83744 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 01/12] s390x/tcg: avoid overflows in time2tod/tod2time +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +Big values for the TOD/ns clock can result in some overflows that can be +avoided. Not all overflows can be handled however, as the conversion either +multiplies by 4.096 or divided by 4.096. + +Apply the trick used in the Linux kernel in arch/s390/include/asm/timex.h +for tod_to_ns() and use the same trick also for the conversion in the +other direction. + +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-2-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 14055ce53c2d901d826ffad7fb7d6bb8ab46bdfd) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/internal.h | 5 +++-- + 1 file changed, 3 insertions(+), 2 deletions(-) + +diff --git a/target/s390x/internal.h b/target/s390x/internal.h +index d911e84..d1ed06f 100644 +--- a/target/s390x/internal.h ++++ b/target/s390x/internal.h +@@ -243,13 +243,14 @@ enum cc_op { + /* Converts ns to s390's clock format */ + static inline uint64_t time2tod(uint64_t ns) + { +- return (ns << 9) / 125; ++ return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9); ++ + } + + /* Converts s390's clock format to ns */ + static inline uint64_t tod2time(uint64_t t) + { +- return (t * 125) >> 9; ++ return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9); + } + + static inline hwaddr decode_basedisp_s(CPUS390XState *env, uint32_t ipb, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-drop-tod_basetime.patch b/SOURCES/kvm-s390x-tcg-drop-tod_basetime.patch new file mode 100644 index 0000000..c344efa --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-drop-tod_basetime.patch @@ -0,0 +1,78 @@ +From 76dfca00588aeb71d85bac7ee0e2cf89df0d5b15 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:06 +0000 +Subject: [PATCH 04/22] s390x/tcg: drop tod_basetime + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-5-david@redhat.com> +Patchwork-id: 83747 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 04/12] s390x/tcg: drop tod_basetime +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +Never set to anything but 0. + +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-5-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit f777b20544fe5db3de179a83374cbf9f1e454427) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/cpu.c | 1 - + target/s390x/cpu.h | 1 - + target/s390x/misc_helper.c | 4 ++-- + 3 files changed, 2 insertions(+), 4 deletions(-) + +diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c +index 167d089..5570741 100644 +--- a/target/s390x/cpu.c ++++ b/target/s390x/cpu.c +@@ -290,7 +290,6 @@ static void s390_cpu_initfn(Object *obj) + qemu_get_timedate(&tm, 0); + env->tod_offset = TOD_UNIX_EPOCH + + (time2tod(mktimegm(&tm)) * 1000000000ULL); +- env->tod_basetime = 0; + env->tod_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu); + env->cpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu); + s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu); +diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h +index 67118c8..04f9adf 100644 +--- a/target/s390x/cpu.h ++++ b/target/s390x/cpu.h +@@ -134,7 +134,6 @@ struct CPUS390XState { + #endif + + uint64_t tod_offset; +- uint64_t tod_basetime; + QEMUTimer *tod_timer; + + QEMUTimer *cpu_timer; +diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c +index e0b23c1..8b3b040 100644 +--- a/target/s390x/misc_helper.c ++++ b/target/s390x/misc_helper.c +@@ -142,7 +142,7 @@ uint64_t HELPER(stck)(CPUS390XState *env) + uint64_t time; + + time = env->tod_offset + +- time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) - env->tod_basetime); ++ time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)); + + return time; + } +@@ -162,7 +162,7 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time) + /* nanoseconds */ + time = tod2time(time); + +- timer_mod(env->tod_timer, env->tod_basetime + time); ++ timer_mod(env->tod_timer, time); + } + + /* Set Tod Programmable Field */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-fix-locking-problem-with-tcg_s390_tod_upda.patch b/SOURCES/kvm-s390x-tcg-fix-locking-problem-with-tcg_s390_tod_upda.patch new file mode 100644 index 0000000..52722df --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-fix-locking-problem-with-tcg_s390_tod_upda.patch @@ -0,0 +1,100 @@ +From 4013e52e76cb56a657c58fe03334d72d1ab1824b Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:11 +0000 +Subject: [PATCH 09/22] s390x/tcg: fix locking problem with + tcg_s390_tod_updated + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-10-david@redhat.com> +Patchwork-id: 83754 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 09/12] s390x/tcg: fix locking problem with tcg_s390_tod_updated +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +tcg_s390_tod_updated() is always called with the iothread being locked +(e.g. from S390TODClass->set() e.g. via HELPER(sck) or on incoming +migration). The helper we call takes the lock itself - bad. + +Let's change that by factoring out updating the ckc timer. This now looks +much nicer than having to call a helper from another function. + +While touching it we also make sure that env->ckc is updated even if the +new value is -1ULL, for now it would not have been modified in that case. + +Reported-by: Christian Borntraeger +Signed-off-by: David Hildenbrand +Message-Id: <20180629170520.13671-1-david@redhat.com> +Reviewed-by: Richard Henderson +Signed-off-by: Cornelia Huck +(cherry picked from commit 30c8db0e219a3c1d8b39c19e8b858830cb141738) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/misc_helper.c | 26 ++++++++++++++++---------- + 1 file changed, 16 insertions(+), 10 deletions(-) + +diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c +index d629b2f..ffb9f6c 100644 +--- a/target/s390x/misc_helper.c ++++ b/target/s390x/misc_helper.c +@@ -150,26 +150,23 @@ uint64_t HELPER(stck)(CPUS390XState *env) + return tod.low; + } + +-/* Set Clock Comparator */ +-void HELPER(sckc)(CPUS390XState *env, uint64_t time) ++static void update_ckc_timer(CPUS390XState *env) + { + S390TODState *td = s390_get_todstate(); ++ uint64_t time; + + /* stop the timer and remove pending CKC IRQs */ + timer_del(env->tod_timer); +- qemu_mutex_lock_iothread(); ++ g_assert(qemu_mutex_iothread_locked()); + env->pending_int &= ~INTERRUPT_EXT_CLOCK_COMPARATOR; +- qemu_mutex_unlock_iothread(); + + /* the tod has to exceed the ckc, this can never happen if ckc is all 1's */ +- if (time == -1ULL) { ++ if (env->ckc == -1ULL) { + return; + } + +- env->ckc = time; +- + /* difference between origins */ +- time -= td->base.low; ++ time = env->ckc - td->base.low; + + /* nanoseconds */ + time = tod2time(time); +@@ -177,12 +174,21 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time) + timer_mod(env->tod_timer, time); + } + ++/* Set Clock Comparator */ ++void HELPER(sckc)(CPUS390XState *env, uint64_t ckc) ++{ ++ env->ckc = ckc; ++ ++ qemu_mutex_lock_iothread(); ++ update_ckc_timer(env); ++ qemu_mutex_unlock_iothread(); ++} ++ + void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque) + { + S390CPU *cpu = S390_CPU(cs); +- CPUS390XState *env = &cpu->env; + +- helper_sckc(env, env->ckc); ++ update_ckc_timer(&cpu->env); + } + + /* Set Clock */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-implement-SET-CLOCK.patch b/SOURCES/kvm-s390x-tcg-implement-SET-CLOCK.patch new file mode 100644 index 0000000..17c5cf7 --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-implement-SET-CLOCK.patch @@ -0,0 +1,114 @@ +From cfc375b3dcc391e8895884ff0fe58bdd1652b2d7 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:09 +0000 +Subject: [PATCH 07/22] s390x/tcg: implement SET CLOCK + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-8-david@redhat.com> +Patchwork-id: 83748 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 07/12] s390x/tcg: implement SET CLOCK +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + target/s390x/translate.c: ExitStatus vs. DisasJumpType + +This allows a guest to change its TOD. We already take care of updating +all CKC timers from within S390TODClass. + +Use MO_ALIGN to load the operand manually - this will properly trigger a +SPECIFICATION exception. + +Acked-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-8-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 9dc6753718d4c0fe327729fea94e4d9f3f5a3d17) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/helper.h | 1 + + target/s390x/insn-data.def | 3 +-- + target/s390x/misc_helper.c | 16 ++++++++++++++++ + target/s390x/translate.c | 9 +++++++++ + 4 files changed, 27 insertions(+), 2 deletions(-) + +diff --git a/target/s390x/helper.h b/target/s390x/helper.h +index 59cba86..97c60ca 100644 +--- a/target/s390x/helper.h ++++ b/target/s390x/helper.h +@@ -127,6 +127,7 @@ DEF_HELPER_4(diag, void, env, i32, i32, i32) + DEF_HELPER_3(load_psw, noreturn, env, i64, i64) + DEF_HELPER_FLAGS_2(spx, TCG_CALL_NO_RWG, void, env, i64) + DEF_HELPER_FLAGS_1(stck, TCG_CALL_NO_RWG_SE, i64, env) ++DEF_HELPER_FLAGS_2(sck, TCG_CALL_NO_RWG, i32, env, i64) + DEF_HELPER_FLAGS_2(sckc, TCG_CALL_NO_RWG, void, env, i64) + DEF_HELPER_FLAGS_2(sckpf, TCG_CALL_NO_RWG, void, env, i64) + DEF_HELPER_FLAGS_1(stckc, TCG_CALL_NO_RWG, i64, env) +diff --git a/target/s390x/insn-data.def b/target/s390x/insn-data.def +index 1576194..5c6f33e 100644 +--- a/target/s390x/insn-data.def ++++ b/target/s390x/insn-data.def +@@ -997,8 +997,7 @@ + /* SET ADDRESS SPACE CONTROL FAST */ + C(0xb279, SACF, S, Z, 0, a2, 0, 0, sacf, 0) + /* SET CLOCK */ +- /* ??? Not implemented - is it necessary? */ +- C(0xb204, SCK, S, Z, 0, 0, 0, 0, 0, 0) ++ C(0xb204, SCK, S, Z, la2, 0, 0, 0, sck, 0) + /* SET CLOCK COMPARATOR */ + C(0xb206, SCKC, S, Z, 0, m2_64a, 0, 0, sckc, 0) + /* SET CLOCK PROGRAMMABLE FIELD */ +diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c +index cada7af..d629b2f 100644 +--- a/target/s390x/misc_helper.c ++++ b/target/s390x/misc_helper.c +@@ -185,6 +185,22 @@ void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque) + helper_sckc(env, env->ckc); + } + ++/* Set Clock */ ++uint32_t HELPER(sck)(CPUS390XState *env, uint64_t tod_low) ++{ ++ S390TODState *td = s390_get_todstate(); ++ S390TODClass *tdc = S390_TOD_GET_CLASS(td); ++ S390TOD tod = { ++ .high = 0, ++ .low = tod_low, ++ }; ++ ++ qemu_mutex_lock_iothread(); ++ tdc->set(td, &tod, &error_abort); ++ qemu_mutex_unlock_iothread(); ++ return 0; ++} ++ + /* Set Tod Programmable Field */ + void HELPER(sckpf)(CPUS390XState *env, uint64_t r0) + { +diff --git a/target/s390x/translate.c b/target/s390x/translate.c +index 7d39ab3..ed9aff4 100644 +--- a/target/s390x/translate.c ++++ b/target/s390x/translate.c +@@ -4015,6 +4015,15 @@ static ExitStatus op_stcke(DisasContext *s, DisasOps *o) + return NO_EXIT; + } + ++static ExitStatus op_sck(DisasContext *s, DisasOps *o) ++{ ++ check_privileged(s); ++ tcg_gen_qemu_ld_i64(o->in1, o->addr1, get_mem_index(s), MO_TEQ | MO_ALIGN); ++ gen_helper_sck(cc_op, cpu_env, o->in1); ++ set_cc_static(s); ++ return NO_EXIT; ++} ++ + static ExitStatus op_sckc(DisasContext *s, DisasOps *o) + { + check_privileged(s); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-properly-implement-the-TOD.patch b/SOURCES/kvm-s390x-tcg-properly-implement-the-TOD.patch new file mode 100644 index 0000000..445ad8a --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-properly-implement-the-TOD.patch @@ -0,0 +1,350 @@ +From 15af83ccafc175c7a61d39f998ac7eb43829e53c Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:07 +0000 +Subject: [PATCH 05/22] s390x/tcg: properly implement the TOD + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-6-david@redhat.com> +Patchwork-id: 83751 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 05/12] s390x/tcg: properly implement the TOD +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +Right now, each CPU has its own TOD. Especially, the TOD will differ +based on creation time of a CPU - e.g. when hotplugging a CPU the times +will differ quite a lot, resulting in stall warnings in the guest. + +Let's use a single TOD by implementing our new TOD device. Prepare it +for TOD-clock epoch extension. + +Most importantly, whenever we set the TOD, we have to update the CKC +timer. + +Introduce "tcg_s390x.h" just like "kvm_s390x.h" for tcg specific +function declarations that should not go into cpu.h. + +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-6-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 7de3b1cdc67dcb572c1761c2051252e91a438b22) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + hw/s390x/tod-qemu.c | 46 ++++++++++++++++++++++++++++++++++++++++++---- + hw/s390x/tod.c | 11 +++++++++++ + include/hw/s390x/tod.h | 19 +++++++++++++++++++ + target/s390x/cpu.c | 7 ------- + target/s390x/cpu.h | 1 - + target/s390x/internal.h | 16 ---------------- + target/s390x/misc_helper.c | 25 +++++++++++++++++++------ + target/s390x/tcg_s390x.h | 18 ++++++++++++++++++ + 8 files changed, 109 insertions(+), 34 deletions(-) + create mode 100644 target/s390x/tcg_s390x.h + +diff --git a/hw/s390x/tod-qemu.c b/hw/s390x/tod-qemu.c +index 03ea1ce..59c015c 100644 +--- a/hw/s390x/tod-qemu.c ++++ b/hw/s390x/tod-qemu.c +@@ -11,19 +11,43 @@ + #include "qemu/osdep.h" + #include "qapi/error.h" + #include "hw/s390x/tod.h" ++#include "qemu/timer.h" ++#include "qemu/cutils.h" ++#include "cpu.h" ++#include "tcg_s390x.h" + + static void qemu_s390_tod_get(const S390TODState *td, S390TOD *tod, + Error **errp) + { +- /* FIXME */ +- tod->high = 0; +- tod->low = 0; ++ *tod = td->base; ++ ++ tod->low += time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)); ++ if (tod->low < td->base.low) { ++ tod->high++; ++ } + } + + static void qemu_s390_tod_set(S390TODState *td, const S390TOD *tod, + Error **errp) + { +- /* FIXME */ ++ CPUState *cpu; ++ ++ td->base = *tod; ++ ++ td->base.low -= time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)); ++ if (td->base.low > tod->low) { ++ td->base.high--; ++ } ++ ++ /* ++ * The TOD has been changed and we have to recalculate the CKC values ++ * for all CPUs. We do this asynchronously, as "SET CLOCK should be ++ * issued only while all other activity on all CPUs .. has been ++ * suspended". ++ */ ++ CPU_FOREACH(cpu) { ++ async_run_on_cpu(cpu, tcg_s390_tod_updated, RUN_ON_CPU_NULL); ++ } + } + + static void qemu_s390_tod_class_init(ObjectClass *oc, void *data) +@@ -34,10 +58,24 @@ static void qemu_s390_tod_class_init(ObjectClass *oc, void *data) + tdc->set = qemu_s390_tod_set; + } + ++static void qemu_s390_tod_init(Object *obj) ++{ ++ S390TODState *td = S390_TOD(obj); ++ struct tm tm; ++ ++ qemu_get_timedate(&tm, 0); ++ td->base.high = 0; ++ td->base.low = TOD_UNIX_EPOCH + (time2tod(mktimegm(&tm)) * 1000000000ULL); ++ if (td->base.low < TOD_UNIX_EPOCH) { ++ td->base.high += 1; ++ } ++} ++ + static TypeInfo qemu_s390_tod_info = { + .name = TYPE_QEMU_S390_TOD, + .parent = TYPE_S390_TOD, + .instance_size = sizeof(S390TODState), ++ .instance_init = qemu_s390_tod_init, + .class_init = qemu_s390_tod_class_init, + .class_size = sizeof(S390TODClass), + }; +diff --git a/hw/s390x/tod.c b/hw/s390x/tod.c +index 0501aff..1c63f41 100644 +--- a/hw/s390x/tod.c ++++ b/hw/s390x/tod.c +@@ -30,6 +30,17 @@ void s390_init_tod(void) + qdev_init_nofail(DEVICE(obj)); + } + ++S390TODState *s390_get_todstate(void) ++{ ++ static S390TODState *ts; ++ ++ if (!ts) { ++ ts = S390_TOD(object_resolve_path_type("", TYPE_S390_TOD, NULL)); ++ } ++ ++ return ts; ++} ++ + #define S390_TOD_CLOCK_VALUE_MISSING 0x00 + #define S390_TOD_CLOCK_VALUE_PRESENT 0x01 + +diff --git a/include/hw/s390x/tod.h b/include/hw/s390x/tod.h +index 7096b57..413c0d7 100644 +--- a/include/hw/s390x/tod.h ++++ b/include/hw/s390x/tod.h +@@ -30,6 +30,9 @@ typedef struct S390TOD { + typedef struct S390TODState { + /* private */ + DeviceState parent_obj; ++ ++ /* unused by KVM implementation */ ++ S390TOD base; + } S390TODState; + + typedef struct S390TODClass { +@@ -41,6 +44,22 @@ typedef struct S390TODClass { + void (*set)(S390TODState *td, const S390TOD *tod, Error **errp); + } S390TODClass; + ++/* The value of the TOD clock for 1.1.1970. */ ++#define TOD_UNIX_EPOCH 0x7d91048bca000000ULL ++ ++/* Converts ns to s390's clock format */ ++static inline uint64_t time2tod(uint64_t ns) ++{ ++ return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9); ++} ++ ++/* Converts s390's clock format to ns */ ++static inline uint64_t tod2time(uint64_t t) ++{ ++ return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9); ++} ++ + void s390_init_tod(void); ++S390TODState *s390_get_todstate(void); + + #endif +diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c +index 5570741..f74d2a3 100644 +--- a/target/s390x/cpu.c ++++ b/target/s390x/cpu.c +@@ -30,7 +30,6 @@ + #include "kvm_s390x.h" + #include "sysemu/kvm.h" + #include "qemu-common.h" +-#include "qemu/cutils.h" + #include "qemu/timer.h" + #include "qemu/error-report.h" + #include "trace.h" +@@ -276,9 +275,6 @@ static void s390_cpu_initfn(Object *obj) + CPUState *cs = CPU(obj); + S390CPU *cpu = S390_CPU(obj); + CPUS390XState *env = &cpu->env; +-#if !defined(CONFIG_USER_ONLY) +- struct tm tm; +-#endif + + cs->env_ptr = env; + cs->halted = 1; +@@ -287,9 +283,6 @@ static void s390_cpu_initfn(Object *obj) + s390_cpu_get_crash_info_qom, NULL, NULL, NULL, NULL); + s390_cpu_model_register_props(obj); + #if !defined(CONFIG_USER_ONLY) +- qemu_get_timedate(&tm, 0); +- env->tod_offset = TOD_UNIX_EPOCH + +- (time2tod(mktimegm(&tm)) * 1000000000ULL); + env->tod_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu); + env->cpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu); + s390_cpu_set_state(S390_CPU_STATE_STOPPED, cpu); +diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h +index 04f9adf..6500f42 100644 +--- a/target/s390x/cpu.h ++++ b/target/s390x/cpu.h +@@ -133,7 +133,6 @@ struct CPUS390XState { + uint64_t cpuid; + #endif + +- uint64_t tod_offset; + QEMUTimer *tod_timer; + + QEMUTimer *cpu_timer; +diff --git a/target/s390x/internal.h b/target/s390x/internal.h +index d1ed06f..61a509d 100644 +--- a/target/s390x/internal.h ++++ b/target/s390x/internal.h +@@ -237,22 +237,6 @@ enum cc_op { + CC_OP_MAX + }; + +-/* The value of the TOD clock for 1.1.1970. */ +-#define TOD_UNIX_EPOCH 0x7d91048bca000000ULL +- +-/* Converts ns to s390's clock format */ +-static inline uint64_t time2tod(uint64_t ns) +-{ +- return (ns << 9) / 125 + (((ns & 0xff10000000000000ull) / 125) << 9); +- +-} +- +-/* Converts s390's clock format to ns */ +-static inline uint64_t tod2time(uint64_t t) +-{ +- return ((t >> 9) * 125) + (((t & 0x1ff) * 125) >> 9); +-} +- + static inline hwaddr decode_basedisp_s(CPUS390XState *env, uint32_t ipb, + uint8_t *ar) + { +diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c +index 8b3b040..4f675c7 100644 +--- a/target/s390x/misc_helper.c ++++ b/target/s390x/misc_helper.c +@@ -29,6 +29,8 @@ + #include "exec/address-spaces.h" + #include "exec/exec-all.h" + #include "exec/cpu_ldst.h" ++#include "qapi/error.h" ++#include "tcg_s390x.h" + + #if !defined(CONFIG_USER_ONLY) + #include "sysemu/cpus.h" +@@ -40,6 +42,7 @@ + #include "hw/s390x/ioinst.h" + #include "hw/s390x/s390-pci-inst.h" + #include "hw/boards.h" ++#include "hw/s390x/tod.h" + #endif + + /* #define DEBUG_HELPER */ +@@ -139,17 +142,19 @@ void HELPER(spx)(CPUS390XState *env, uint64_t a1) + /* Store Clock */ + uint64_t HELPER(stck)(CPUS390XState *env) + { +- uint64_t time; ++ S390TODState *td = s390_get_todstate(); ++ S390TODClass *tdc = S390_TOD_GET_CLASS(td); ++ S390TOD tod; + +- time = env->tod_offset + +- time2tod(qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL)); +- +- return time; ++ tdc->get(td, &tod, &error_abort); ++ return tod.low; + } + + /* Set Clock Comparator */ + void HELPER(sckc)(CPUS390XState *env, uint64_t time) + { ++ S390TODState *td = s390_get_todstate(); ++ + if (time == -1ULL) { + return; + } +@@ -157,7 +162,7 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time) + env->ckc = time; + + /* difference between origins */ +- time -= env->tod_offset; ++ time -= td->base.low; + + /* nanoseconds */ + time = tod2time(time); +@@ -165,6 +170,14 @@ void HELPER(sckc)(CPUS390XState *env, uint64_t time) + timer_mod(env->tod_timer, time); + } + ++void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque) ++{ ++ S390CPU *cpu = S390_CPU(cs); ++ CPUS390XState *env = &cpu->env; ++ ++ helper_sckc(env, env->ckc); ++} ++ + /* Set Tod Programmable Field */ + void HELPER(sckpf)(CPUS390XState *env, uint64_t r0) + { +diff --git a/target/s390x/tcg_s390x.h b/target/s390x/tcg_s390x.h +new file mode 100644 +index 0000000..4e308aa +--- /dev/null ++++ b/target/s390x/tcg_s390x.h +@@ -0,0 +1,18 @@ ++/* ++ * QEMU TCG support -- s390x specific functions. ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * ++ * Authors: ++ * David Hildenbrand ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#ifndef TCG_S390X_H ++#define TCG_S390X_H ++ ++void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque); ++ ++#endif /* TCG_S390X_H */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tcg-rearm-the-CKC-timer-during-migration.patch b/SOURCES/kvm-s390x-tcg-rearm-the-CKC-timer-during-migration.patch new file mode 100644 index 0000000..fd1c7ea --- /dev/null +++ b/SOURCES/kvm-s390x-tcg-rearm-the-CKC-timer-during-migration.patch @@ -0,0 +1,98 @@ +From 2c2a89b2c366969ef32d9ede7ff0af92d22eee10 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:10 +0000 +Subject: [PATCH 08/22] s390x/tcg: rearm the CKC timer during migration + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-9-david@redhat.com> +Patchwork-id: 83752 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 08/12] s390x/tcg: rearm the CKC timer during migration +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +If the CPU data is migrated after the TOD clock, the CKC timer of a CPU +is not rearmed. Let's rearm it when loading the CPU state. + +Introduce tcg-stub.c just like kvm-stub.c for tcg specific stubs. + +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-9-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 7c12f710bad60dc7e509da4e80c77e952ef0490c) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + target/s390x/Makefile.objs | 1 + + target/s390x/machine.c | 6 ++++++ + target/s390x/tcg-stub.c | 20 ++++++++++++++++++++ + 3 files changed, 27 insertions(+) + create mode 100644 target/s390x/tcg-stub.c + +diff --git a/target/s390x/Makefile.objs b/target/s390x/Makefile.objs +index 31932de..22a9a99 100644 +--- a/target/s390x/Makefile.objs ++++ b/target/s390x/Makefile.objs +@@ -5,6 +5,7 @@ obj-$(CONFIG_SOFTMMU) += machine.o ioinst.o arch_dump.o mmu_helper.o diag.o + obj-$(CONFIG_SOFTMMU) += sigp.o + obj-$(CONFIG_KVM) += kvm.o + obj-$(call lnot,$(CONFIG_KVM)) += kvm-stub.o ++obj-$(call lnot,$(CONFIG_TCG)) += tcg-stub.o + + # build and run feature list generator + feat-src = $(SRC_PATH)/target/$(TARGET_BASE_ARCH)/ +diff --git a/target/s390x/machine.c b/target/s390x/machine.c +index 8421deb..cb792aa 100644 +--- a/target/s390x/machine.c ++++ b/target/s390x/machine.c +@@ -19,6 +19,7 @@ + #include "cpu.h" + #include "internal.h" + #include "kvm_s390x.h" ++#include "tcg_s390x.h" + #include "sysemu/kvm.h" + + static int cpu_post_load(void *opaque, int version_id) +@@ -34,6 +35,11 @@ static int cpu_post_load(void *opaque, int version_id) + return kvm_s390_vcpu_interrupt_post_load(cpu); + } + ++ if (tcg_enabled()) { ++ /* Rearm the CKC timer if necessary */ ++ tcg_s390_tod_updated(CPU(cpu), RUN_ON_CPU_NULL); ++ } ++ + return 0; + } + +diff --git a/target/s390x/tcg-stub.c b/target/s390x/tcg-stub.c +new file mode 100644 +index 0000000..c93501d +--- /dev/null ++++ b/target/s390x/tcg-stub.c +@@ -0,0 +1,20 @@ ++/* ++ * QEMU TCG support -- s390x specific function stubs. ++ * ++ * Copyright (C) 2018 Red Hat Inc ++ * ++ * Authors: ++ * David Hildenbrand ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#include "qemu/osdep.h" ++#include "qemu-common.h" ++#include "cpu.h" ++#include "tcg_s390x.h" ++ ++void tcg_s390_tod_updated(CPUState *cs, run_on_cpu_data opaque) ++{ ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tod-Properly-stop-the-KVM-TOD-while-the-guest-.patch b/SOURCES/kvm-s390x-tod-Properly-stop-the-KVM-TOD-while-the-guest-.patch new file mode 100644 index 0000000..69b29d0 --- /dev/null +++ b/SOURCES/kvm-s390x-tod-Properly-stop-the-KVM-TOD-while-the-guest-.patch @@ -0,0 +1,250 @@ +From 9552c25dc788925f211daaa55518e5144f8f0cc7 Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:13 +0000 +Subject: [PATCH 11/22] s390x/tod: Properly stop the KVM TOD while the guest is + not running + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-12-david@redhat.com> +Patchwork-id: 83755 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 11/12] s390x/tod: Properly stop the KVM TOD while the guest is not running +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier + +Just like on other architectures, we should stop the clock while the guest +is not running. This is already properly done for TCG. Right now, doing an +offline migration (stop, migrate, cont) can easily trigger stalls in the +guest. + +Even doing a + (hmp) stop + ... wait 2 minutes ... + (hmp) cont +will already trigger stalls. + +So whenever the guest stops, backup the KVM TOD. When continuing to run +the guest, restore the KVM TOD. + +One special case is starting a simple VM: Reading the TOD from KVM to +stop it right away until the guest is actually started means that the +time of any simple VM will already differ to the host time. We can +simply leave the TOD running and the guest won't be able to recognize +it. + +For migration, we actually want to keep the TOD stopped until really +starting the guest. To be able to catch most errors, we should however +try to set the TOD in addition to simply storing it. So we can still +catch basic migration problems. + +If anything goes wrong while backing up/restoring the TOD, we have to +ignore it (but print a warning). This is then basically a fallback to +old behavior (TOD remains running). + +I tested this very basically with an initrd: + 1. Start a simple VM. Observed that the TOD is kept running. Old + behavior. + 2. Ordinary live migration. Observed that the TOD is temporarily + stopped on the destination when setting the new value and + correctly started when finally starting the guest. + 3. Offline live migration. (stop, migrate, cont). Observed that the + TOD will be stopped on the source with the "stop" command. On the + destination, the TOD is temporarily stopped when setting the new + value and correctly started when finally starting the guest via + "cont". + 4. Simple stop/cont correctly stops/starts the TOD. (multiple stops + or conts in a row have no effect, so works as expected) + +In the future, we might want to send the guest a special kind of time sync +interrupt under some conditions, so it can synchronize its tod to the +host tod. This is interesting for migration scenarios but also when we +get time sync interrupts ourselves. This however will most probably have +to be handled in KVM (e.g. when the tods differ too much) and is not +desired e.g. when debugging the guest (single stepping should not +result in permanent time syncs). I consider something like that an add-on +on top of this basic "don't break the guest" handling. + +Signed-off-by: David Hildenbrand +Message-Id: <20181130094957.4121-1-david@redhat.com> +Acked-by: Christian Borntraeger +Reviewed-by: Thomas Huth +Signed-off-by: Cornelia Huck +(cherry picked from commit 9bc9d3d1ae3bcd1caaad1946494726b52f58b291) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + hw/s390x/tod-kvm.c | 102 ++++++++++++++++++++++++++++++++++++++++++++++++- + include/hw/s390x/tod.h | 8 +++- + 2 files changed, 107 insertions(+), 3 deletions(-) + +diff --git a/hw/s390x/tod-kvm.c b/hw/s390x/tod-kvm.c +index df564ab..2456bf7 100644 +--- a/hw/s390x/tod-kvm.c ++++ b/hw/s390x/tod-kvm.c +@@ -10,10 +10,11 @@ + + #include "qemu/osdep.h" + #include "qapi/error.h" ++#include "sysemu/sysemu.h" + #include "hw/s390x/tod.h" + #include "kvm_s390x.h" + +-static void kvm_s390_tod_get(const S390TODState *td, S390TOD *tod, Error **errp) ++static void kvm_s390_get_tod_raw(S390TOD *tod, Error **errp) + { + int r; + +@@ -27,7 +28,17 @@ static void kvm_s390_tod_get(const S390TODState *td, S390TOD *tod, Error **errp) + } + } + +-static void kvm_s390_tod_set(S390TODState *td, const S390TOD *tod, Error **errp) ++static void kvm_s390_tod_get(const S390TODState *td, S390TOD *tod, Error **errp) ++{ ++ if (td->stopped) { ++ *tod = td->base; ++ return; ++ } ++ ++ kvm_s390_get_tod_raw(tod, errp); ++} ++ ++static void kvm_s390_set_tod_raw(const S390TOD *tod, Error **errp) + { + int r; + +@@ -41,18 +52,105 @@ static void kvm_s390_tod_set(S390TODState *td, const S390TOD *tod, Error **errp) + } + } + ++static void kvm_s390_tod_set(S390TODState *td, const S390TOD *tod, Error **errp) ++{ ++ Error *local_err = NULL; ++ ++ /* ++ * Somebody (e.g. migration) set the TOD. We'll store it into KVM to ++ * properly detect errors now but take a look at the runstate to decide ++ * whether really to keep the tod running. E.g. during migration, this ++ * is the point where we want to stop the initially running TOD to fire ++ * it back up when actually starting the migrated guest. ++ */ ++ kvm_s390_set_tod_raw(tod, &local_err); ++ if (local_err) { ++ error_propagate(errp, local_err); ++ return; ++ } ++ ++ if (runstate_is_running()) { ++ td->stopped = false; ++ } else { ++ td->stopped = true; ++ td->base = *tod; ++ } ++} ++ ++static void kvm_s390_tod_vm_state_change(void *opaque, int running, ++ RunState state) ++{ ++ S390TODState *td = opaque; ++ Error *local_err = NULL; ++ ++ if (running && td->stopped) { ++ /* Set the old TOD when running the VM - start the TOD clock. */ ++ kvm_s390_set_tod_raw(&td->base, &local_err); ++ if (local_err) { ++ warn_report_err(local_err); ++ } ++ /* Treat errors like the TOD was running all the time. */ ++ td->stopped = false; ++ } else if (!running && !td->stopped) { ++ /* Store the TOD when stopping the VM - stop the TOD clock. */ ++ kvm_s390_get_tod_raw(&td->base, &local_err); ++ if (local_err) { ++ /* Keep the TOD running in case we could not back it up. */ ++ warn_report_err(local_err); ++ } else { ++ td->stopped = true; ++ } ++ } ++} ++ ++static void kvm_s390_tod_realize(DeviceState *dev, Error **errp) ++{ ++ S390TODState *td = S390_TOD(dev); ++ S390TODClass *tdc = S390_TOD_GET_CLASS(td); ++ Error *local_err = NULL; ++ ++ tdc->parent_realize(dev, &local_err); ++ if (local_err) { ++ error_propagate(errp, local_err); ++ return; ++ } ++ ++ /* ++ * We need to know when the VM gets started/stopped to start/stop the TOD. ++ * As we can never have more than one TOD instance (and that will never be ++ * removed), registering here and never unregistering is good enough. ++ */ ++ qemu_add_vm_change_state_handler(kvm_s390_tod_vm_state_change, td); ++} ++ + static void kvm_s390_tod_class_init(ObjectClass *oc, void *data) + { + S390TODClass *tdc = S390_TOD_CLASS(oc); + ++ device_class_set_parent_realize(DEVICE_CLASS(oc), kvm_s390_tod_realize, ++ &tdc->parent_realize); + tdc->get = kvm_s390_tod_get; + tdc->set = kvm_s390_tod_set; + } + ++static void kvm_s390_tod_init(Object *obj) ++{ ++ S390TODState *td = S390_TOD(obj); ++ ++ /* ++ * The TOD is initially running (value stored in KVM). Avoid needless ++ * loading/storing of the TOD when starting a simple VM, so let it ++ * run although the (never started) VM is stopped. For migration, we ++ * will properly set the TOD later. ++ */ ++ td->stopped = false; ++} ++ + static TypeInfo kvm_s390_tod_info = { + .name = TYPE_KVM_S390_TOD, + .parent = TYPE_S390_TOD, + .instance_size = sizeof(S390TODState), ++ .instance_init = kvm_s390_tod_init, + .class_init = kvm_s390_tod_class_init, + .class_size = sizeof(S390TODClass), + }; +diff --git a/include/hw/s390x/tod.h b/include/hw/s390x/tod.h +index 413c0d7..cbd7552 100644 +--- a/include/hw/s390x/tod.h ++++ b/include/hw/s390x/tod.h +@@ -31,13 +31,19 @@ typedef struct S390TODState { + /* private */ + DeviceState parent_obj; + +- /* unused by KVM implementation */ ++ /* ++ * Used by TCG to remember the time base. Used by KVM to backup the TOD ++ * while the TOD is stopped. ++ */ + S390TOD base; ++ /* Used by KVM to remember if the TOD is stopped and base is valid. */ ++ bool stopped; + } S390TODState; + + typedef struct S390TODClass { + /* private */ + DeviceClass parent_class; ++ void (*parent_realize)(DeviceState *dev, Error **errp); + + /* public */ + void (*get)(const S390TODState *td, S390TOD *tod, Error **errp); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-tod-factor-out-TOD-into-separate-device.patch b/SOURCES/kvm-s390x-tod-factor-out-TOD-into-separate-device.patch new file mode 100644 index 0000000..9136eba --- /dev/null +++ b/SOURCES/kvm-s390x-tod-factor-out-TOD-into-separate-device.patch @@ -0,0 +1,517 @@ +From cd498a086077cab11fbc695336db52ba819201bd Mon Sep 17 00:00:00 2001 +From: David Hildenbrand +Date: Fri, 21 Dec 2018 15:36:05 +0000 +Subject: [PATCH 03/22] s390x/tod: factor out TOD into separate device + +RH-Author: David Hildenbrand +Message-id: <20181221153614.27961-4-david@redhat.com> +Patchwork-id: 83749 +O-Subject: [RHEL-8.0 qemu-kvm v2 PATCH 03/12] s390x/tod: factor out TOD into separate device +Bugzilla: 1653569 +RH-Acked-by: Cornelia Huck +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + hw/s390x/s390-virtio-ccw.c: contextual conflict (console init + rework) + +Let's treat this like a separate device. TCG will have to store the +actual state/time later on. + +Include cpu-qom.h in kvm_s390x.h (due to S390CPU) to compile tod-kvm.c. + +Reviewed-by: Thomas Huth +Signed-off-by: David Hildenbrand +Message-Id: <20180627134410.4901-4-david@redhat.com> +Signed-off-by: Cornelia Huck +(cherry picked from commit 8046f374a64b81fdf4f71f7a433bf4035d501521) +Signed-off-by: David Hildenbrand +Signed-off-by: Danilo C. L. de Paula +--- + hw/s390x/Makefile.objs | 3 ++ + hw/s390x/s390-virtio-ccw.c | 57 ++-------------------- + hw/s390x/tod-kvm.c | 64 ++++++++++++++++++++++++ + hw/s390x/tod-qemu.c | 49 +++++++++++++++++++ + hw/s390x/tod.c | 119 +++++++++++++++++++++++++++++++++++++++++++++ + include/hw/s390x/tod.h | 46 ++++++++++++++++++ + target/s390x/cpu.c | 32 ------------ + target/s390x/cpu.h | 2 - + target/s390x/kvm_s390x.h | 2 + + 9 files changed, 286 insertions(+), 88 deletions(-) + create mode 100644 hw/s390x/tod-kvm.c + create mode 100644 hw/s390x/tod-qemu.c + create mode 100644 hw/s390x/tod.c + create mode 100644 include/hw/s390x/tod.h + +diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs +index 655d1ac..add89b1 100644 +--- a/hw/s390x/Makefile.objs ++++ b/hw/s390x/Makefile.objs +@@ -14,6 +14,9 @@ obj-$(CONFIG_PCI) += s390-pci-bus.o s390-pci-inst.o + obj-$(call lnot,$(CONFIG_PCI)) += s390-pci-stub.o + obj-y += s390-skeys.o + obj-y += s390-stattrib.o ++obj-y += tod.o ++obj-$(CONFIG_KVM) += tod-kvm.o ++obj-$(CONFIG_TCG) += tod-qemu.o + obj-$(CONFIG_KVM) += s390-skeys-kvm.o + obj-$(CONFIG_KVM) += s390-stattrib-kvm.o + obj-y += s390-ccw.o +diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c +index 43a8213..8f93edc 100644 +--- a/hw/s390x/s390-virtio-ccw.c ++++ b/hw/s390x/s390-virtio-ccw.c +@@ -36,6 +36,7 @@ + #include "migration/register.h" + #include "cpu_models.h" + #include "hw/nmi.h" ++#include "hw/s390x/tod.h" + + S390CPU *s390_cpu_addr2state(uint16_t cpu_addr) + { +@@ -188,58 +189,6 @@ static void s390_memory_init(ram_addr_t mem_size) + s390_stattrib_init(); + } + +-#define S390_TOD_CLOCK_VALUE_MISSING 0x00 +-#define S390_TOD_CLOCK_VALUE_PRESENT 0x01 +- +-static void gtod_save(QEMUFile *f, void *opaque) +-{ +- uint64_t tod_low; +- uint8_t tod_high; +- int r; +- +- r = s390_get_clock(&tod_high, &tod_low); +- if (r) { +- warn_report("Unable to get guest clock for migration: %s", +- strerror(-r)); +- error_printf("Guest clock will not be migrated " +- "which could cause the guest to hang."); +- qemu_put_byte(f, S390_TOD_CLOCK_VALUE_MISSING); +- return; +- } +- +- qemu_put_byte(f, S390_TOD_CLOCK_VALUE_PRESENT); +- qemu_put_byte(f, tod_high); +- qemu_put_be64(f, tod_low); +-} +- +-static int gtod_load(QEMUFile *f, void *opaque, int version_id) +-{ +- uint64_t tod_low; +- uint8_t tod_high; +- int r; +- +- if (qemu_get_byte(f) == S390_TOD_CLOCK_VALUE_MISSING) { +- warn_report("Guest clock was not migrated. This could " +- "cause the guest to hang."); +- return 0; +- } +- +- tod_high = qemu_get_byte(f); +- tod_low = qemu_get_be64(f); +- +- r = s390_set_clock(&tod_high, &tod_low); +- if (r) { +- error_report("Unable to set KVM guest TOD clock: %s", strerror(-r)); +- } +- +- return r; +-} +- +-static SaveVMHandlers savevm_gtod = { +- .save_state = gtod_save, +- .load_state = gtod_load, +-}; +- + static void s390_init_ipl_dev(const char *kernel_filename, + const char *kernel_cmdline, + const char *initrd_filename, const char *firmware, +@@ -350,8 +299,8 @@ static void ccw_init(MachineState *machine) + /* Create VirtIO network adapters */ + s390_create_virtio_net(BUS(css_bus), "virtio-net-ccw"); + +- /* Register savevm handler for guest TOD clock */ +- register_savevm_live(NULL, "todclock", 0, 1, &savevm_gtod, NULL); ++ /* init the TOD clock */ ++ s390_init_tod(); + } + + static void s390_cpu_plug(HotplugHandler *hotplug_dev, +diff --git a/hw/s390x/tod-kvm.c b/hw/s390x/tod-kvm.c +new file mode 100644 +index 0000000..df564ab +--- /dev/null ++++ b/hw/s390x/tod-kvm.c +@@ -0,0 +1,64 @@ ++/* ++ * TOD (Time Of Day) clock - KVM implementation ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * Author(s): David Hildenbrand ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#include "qemu/osdep.h" ++#include "qapi/error.h" ++#include "hw/s390x/tod.h" ++#include "kvm_s390x.h" ++ ++static void kvm_s390_tod_get(const S390TODState *td, S390TOD *tod, Error **errp) ++{ ++ int r; ++ ++ r = kvm_s390_get_clock_ext(&tod->high, &tod->low); ++ if (r == -ENXIO) { ++ r = kvm_s390_get_clock(&tod->high, &tod->low); ++ } ++ if (r) { ++ error_setg(errp, "Unable to get KVM guest TOD clock: %s", ++ strerror(-r)); ++ } ++} ++ ++static void kvm_s390_tod_set(S390TODState *td, const S390TOD *tod, Error **errp) ++{ ++ int r; ++ ++ r = kvm_s390_set_clock_ext(tod->high, tod->low); ++ if (r == -ENXIO) { ++ r = kvm_s390_set_clock(tod->high, tod->low); ++ } ++ if (r) { ++ error_setg(errp, "Unable to set KVM guest TOD clock: %s", ++ strerror(-r)); ++ } ++} ++ ++static void kvm_s390_tod_class_init(ObjectClass *oc, void *data) ++{ ++ S390TODClass *tdc = S390_TOD_CLASS(oc); ++ ++ tdc->get = kvm_s390_tod_get; ++ tdc->set = kvm_s390_tod_set; ++} ++ ++static TypeInfo kvm_s390_tod_info = { ++ .name = TYPE_KVM_S390_TOD, ++ .parent = TYPE_S390_TOD, ++ .instance_size = sizeof(S390TODState), ++ .class_init = kvm_s390_tod_class_init, ++ .class_size = sizeof(S390TODClass), ++}; ++ ++static void register_types(void) ++{ ++ type_register_static(&kvm_s390_tod_info); ++} ++type_init(register_types); +diff --git a/hw/s390x/tod-qemu.c b/hw/s390x/tod-qemu.c +new file mode 100644 +index 0000000..03ea1ce +--- /dev/null ++++ b/hw/s390x/tod-qemu.c +@@ -0,0 +1,49 @@ ++/* ++ * TOD (Time Of Day) clock - QEMU implementation ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * Author(s): David Hildenbrand ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#include "qemu/osdep.h" ++#include "qapi/error.h" ++#include "hw/s390x/tod.h" ++ ++static void qemu_s390_tod_get(const S390TODState *td, S390TOD *tod, ++ Error **errp) ++{ ++ /* FIXME */ ++ tod->high = 0; ++ tod->low = 0; ++} ++ ++static void qemu_s390_tod_set(S390TODState *td, const S390TOD *tod, ++ Error **errp) ++{ ++ /* FIXME */ ++} ++ ++static void qemu_s390_tod_class_init(ObjectClass *oc, void *data) ++{ ++ S390TODClass *tdc = S390_TOD_CLASS(oc); ++ ++ tdc->get = qemu_s390_tod_get; ++ tdc->set = qemu_s390_tod_set; ++} ++ ++static TypeInfo qemu_s390_tod_info = { ++ .name = TYPE_QEMU_S390_TOD, ++ .parent = TYPE_S390_TOD, ++ .instance_size = sizeof(S390TODState), ++ .class_init = qemu_s390_tod_class_init, ++ .class_size = sizeof(S390TODClass), ++}; ++ ++static void register_types(void) ++{ ++ type_register_static(&qemu_s390_tod_info); ++} ++type_init(register_types); +diff --git a/hw/s390x/tod.c b/hw/s390x/tod.c +new file mode 100644 +index 0000000..0501aff +--- /dev/null ++++ b/hw/s390x/tod.c +@@ -0,0 +1,119 @@ ++/* ++ * TOD (Time Of Day) clock ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * Author(s): David Hildenbrand ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#include "qemu/osdep.h" ++#include "hw/s390x/tod.h" ++#include "qapi/error.h" ++#include "qemu/error-report.h" ++#include "sysemu/kvm.h" ++#include "migration/register.h" ++ ++void s390_init_tod(void) ++{ ++ Object *obj; ++ ++ if (kvm_enabled()) { ++ obj = object_new(TYPE_KVM_S390_TOD); ++ } else { ++ obj = object_new(TYPE_QEMU_S390_TOD); ++ } ++ object_property_add_child(qdev_get_machine(), TYPE_S390_TOD, obj, NULL); ++ object_unref(obj); ++ ++ qdev_init_nofail(DEVICE(obj)); ++} ++ ++#define S390_TOD_CLOCK_VALUE_MISSING 0x00 ++#define S390_TOD_CLOCK_VALUE_PRESENT 0x01 ++ ++static void s390_tod_save(QEMUFile *f, void *opaque) ++{ ++ S390TODState *td = opaque; ++ S390TODClass *tdc = S390_TOD_GET_CLASS(td); ++ Error *err = NULL; ++ S390TOD tod; ++ ++ tdc->get(td, &tod, &err); ++ if (err) { ++ warn_report_err(err); ++ error_printf("Guest clock will not be migrated " ++ "which could cause the guest to hang."); ++ qemu_put_byte(f, S390_TOD_CLOCK_VALUE_MISSING); ++ return; ++ } ++ ++ qemu_put_byte(f, S390_TOD_CLOCK_VALUE_PRESENT); ++ qemu_put_byte(f, tod.high); ++ qemu_put_be64(f, tod.low); ++} ++ ++static int s390_tod_load(QEMUFile *f, void *opaque, int version_id) ++{ ++ S390TODState *td = opaque; ++ S390TODClass *tdc = S390_TOD_GET_CLASS(td); ++ Error *err = NULL; ++ S390TOD tod; ++ ++ if (qemu_get_byte(f) == S390_TOD_CLOCK_VALUE_MISSING) { ++ warn_report("Guest clock was not migrated. This could " ++ "cause the guest to hang."); ++ return 0; ++ } ++ ++ tod.high = qemu_get_byte(f); ++ tod.low = qemu_get_be64(f); ++ ++ tdc->set(td, &tod, &err); ++ if (err) { ++ error_report_err(err); ++ return -1; ++ } ++ return 0; ++} ++ ++static SaveVMHandlers savevm_tod = { ++ .save_state = s390_tod_save, ++ .load_state = s390_tod_load, ++}; ++ ++static void s390_tod_realize(DeviceState *dev, Error **errp) ++{ ++ S390TODState *td = S390_TOD(dev); ++ ++ /* Legacy migration interface */ ++ register_savevm_live(NULL, "todclock", 0, 1, &savevm_tod, td); ++} ++ ++static void s390_tod_class_init(ObjectClass *oc, void *data) ++{ ++ DeviceClass *dc = DEVICE_CLASS(oc); ++ ++ dc->desc = "TOD (Time Of Day) Clock"; ++ dc->realize = s390_tod_realize; ++ set_bit(DEVICE_CATEGORY_MISC, dc->categories); ++ ++ /* We only have one TOD clock in the system attached to the machine */ ++ dc->user_creatable = false; ++} ++ ++static TypeInfo s390_tod_info = { ++ .name = TYPE_S390_TOD, ++ .parent = TYPE_DEVICE, ++ .instance_size = sizeof(S390TODState), ++ .class_init = s390_tod_class_init, ++ .class_size = sizeof(S390TODClass), ++ .abstract = true, ++}; ++ ++static void register_types(void) ++{ ++ type_register_static(&s390_tod_info); ++} ++type_init(register_types); +diff --git a/include/hw/s390x/tod.h b/include/hw/s390x/tod.h +new file mode 100644 +index 0000000..7096b57 +--- /dev/null ++++ b/include/hw/s390x/tod.h +@@ -0,0 +1,46 @@ ++/* ++ * TOD (Time Of Day) clock ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * Author(s): David Hildenbrand ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#ifndef HW_S390_TOD_H ++#define HW_S390_TOD_H ++ ++#include "hw/qdev.h" ++ ++typedef struct S390TOD { ++ uint8_t high; ++ uint64_t low; ++} S390TOD; ++ ++#define TYPE_S390_TOD "s390-tod" ++#define S390_TOD(obj) OBJECT_CHECK(S390TODState, (obj), TYPE_S390_TOD) ++#define S390_TOD_CLASS(oc) OBJECT_CLASS_CHECK(S390TODClass, (oc), \ ++ TYPE_S390_TOD) ++#define S390_TOD_GET_CLASS(obj) OBJECT_GET_CLASS(S390TODClass, (obj), \ ++ TYPE_S390_TOD) ++#define TYPE_KVM_S390_TOD TYPE_S390_TOD "-kvm" ++#define TYPE_QEMU_S390_TOD TYPE_S390_TOD "-qemu" ++ ++typedef struct S390TODState { ++ /* private */ ++ DeviceState parent_obj; ++} S390TODState; ++ ++typedef struct S390TODClass { ++ /* private */ ++ DeviceClass parent_class; ++ ++ /* public */ ++ void (*get)(const S390TODState *td, S390TOD *tod, Error **errp); ++ void (*set)(S390TODState *td, const S390TOD *tod, Error **errp); ++} S390TODClass; ++ ++void s390_init_tod(void); ++ ++#endif +diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c +index 1f590d1..167d089 100644 +--- a/target/s390x/cpu.c ++++ b/target/s390x/cpu.c +@@ -391,38 +391,6 @@ unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu) + return s390_count_running_cpus(); + } + +-int s390_get_clock(uint8_t *tod_high, uint64_t *tod_low) +-{ +- int r = 0; +- +- if (kvm_enabled()) { +- r = kvm_s390_get_clock_ext(tod_high, tod_low); +- if (r == -ENXIO) { +- return kvm_s390_get_clock(tod_high, tod_low); +- } +- } else { +- /* Fixme TCG */ +- *tod_high = 0; +- *tod_low = 0; +- } +- +- return r; +-} +- +-int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low) +-{ +- int r = 0; +- +- if (kvm_enabled()) { +- r = kvm_s390_set_clock_ext(*tod_high, *tod_low); +- if (r == -ENXIO) { +- return kvm_s390_set_clock(*tod_high, *tod_low); +- } +- } +- /* Fixme TCG */ +- return r; +-} +- + int s390_set_memory_limit(uint64_t new_limit, uint64_t *hw_limit) + { + if (kvm_enabled()) { +diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h +index 86d08fa..67118c8 100644 +--- a/target/s390x/cpu.h ++++ b/target/s390x/cpu.h +@@ -691,8 +691,6 @@ static inline uint64_t s390_build_validity_mcic(void) + + + /* cpu.c */ +-int s390_get_clock(uint8_t *tod_high, uint64_t *tod_low); +-int s390_set_clock(uint8_t *tod_high, uint64_t *tod_low); + void s390_crypto_reset(void); + bool s390_get_squash_mcss(void); + int s390_set_memory_limit(uint64_t new_limit, uint64_t *hw_limit); +diff --git a/target/s390x/kvm_s390x.h b/target/s390x/kvm_s390x.h +index 36eb34b..6e52287 100644 +--- a/target/s390x/kvm_s390x.h ++++ b/target/s390x/kvm_s390x.h +@@ -10,6 +10,8 @@ + #ifndef KVM_S390X_H + #define KVM_S390X_H + ++#include "cpu-qom.h" ++ + struct kvm_s390_irq; + + void kvm_s390_floating_interrupt_legacy(struct kvm_s390_irq *irq); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x-vfio-ap-Introduce-VFIO-AP-device.patch b/SOURCES/kvm-s390x-vfio-ap-Introduce-VFIO-AP-device.patch new file mode 100644 index 0000000..01464ce --- /dev/null +++ b/SOURCES/kvm-s390x-vfio-ap-Introduce-VFIO-AP-device.patch @@ -0,0 +1,305 @@ +From 95bd55e26e97832288c088f2c0bd3618d9f9f7ff Mon Sep 17 00:00:00 2001 +From: Thomas Huth +Date: Mon, 15 Oct 2018 10:19:30 +0100 +Subject: [PATCH 5/6] s390x/vfio: ap: Introduce VFIO AP device + +RH-Author: Thomas Huth +Message-id: <1539598771-16223-6-git-send-email-thuth@redhat.com> +Patchwork-id: 82700 +O-Subject: [RHEL-8 qemu-kvm PATCH 5/6] s390x/vfio: ap: Introduce VFIO AP device +Bugzilla: 1508142 +RH-Acked-by: David Hildenbrand +RH-Acked-by: Cornelia Huck +RH-Acked-by: Jens Freimann + +From: Tony Krowiak + +Introduces a VFIO based AP device. The device is defined via +the QEMU command line by specifying: + + -device vfio-ap,sysfsdev= + +There may be only one vfio-ap device configured for a guest. + +The mediated matrix device is created by the VFIO AP device +driver by writing a UUID to a sysfs attribute file (see +docs/vfio-ap.txt). The mediated matrix device will be named +after the UUID. Symbolic links to the $uuid are created in +many places, so the path to the mediated matrix device $uuid +can be specified in any of the following ways: + +/sys/devices/vfio_ap/matrix/$uuid +/sys/devices/vfio_ap/matrix/mdev_supported_types/vfio_ap-passthrough/devices/$uuid +/sys/bus/mdev/devices/$uuid +/sys/bus/mdev/drivers/vfio_mdev/$uuid + +When the vfio-ap device is realized, it acquires and opens the +VFIO iommu group to which the mediated matrix device is +bound. This causes a VFIO group notification event to be +signaled. The vfio_ap device driver's group notification +handler will get called at which time the device driver +will configure the the AP devices to which the guest will +be granted access. + +Signed-off-by: Tony Krowiak +Tested-by: Pierre Morel +Acked-by: Halil Pasic +Tested-by: Pierre Morel +Tested-by: Christian Borntraeger +Message-Id: <20181010170309.12045-6-akrowiak@linux.ibm.com> +Reviewed-by: Thomas Huth +[CH: added missing g_free and device category] +Signed-off-by: Cornelia Huck + +(cherry picked from commit 2fe2942cd6ddad8ddd40fe5d16d67599c28959d7) +Signed-off-by: Danilo C. L. de Paula +--- + MAINTAINERS | 2 + + default-configs/s390x-softmmu.mak | 1 + + hw/vfio/Makefile.objs | 1 + + hw/vfio/ap.c | 181 ++++++++++++++++++++++++++++++++++++++ + include/hw/vfio/vfio-common.h | 1 + + 5 files changed, 186 insertions(+) + create mode 100644 hw/vfio/ap.c + +diff --git a/MAINTAINERS b/MAINTAINERS +index 31cf6ff..99694d8 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -88,6 +88,7 @@ F: hw/char/terminal3270.c + F: hw/intc/s390_flic.c + F: hw/intc/s390_flic_kvm.c + F: hw/s390x/ ++F: hw/vfio/ap.c + F: hw/vfio/ccw.c + F: hw/watchdog/wdt_diag288.c + F: include/hw/s390x/ +@@ -1162,6 +1163,7 @@ F: hw/s390x/ap-device.c + F: hw/s390x/ap-bridge.c + F: include/hw/s390x/ap-device.h + F: include/hw/s390x/ap-bridge.h ++F: hw/vfio/ap.c + L: qemu-s390x@nongnu.org + + vhost +diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak +index 17e871a..d6ab059 100644 +--- a/default-configs/s390x-softmmu.mak ++++ b/default-configs/s390x-softmmu.mak +@@ -10,3 +10,4 @@ CONFIG_S390_FLIC_KVM=$(CONFIG_KVM) + # Disabled for Red Hat Enterprise Linux: + # CONFIG_VFIO_CCW=$(CONFIG_LINUX) + CONFIG_WDT_DIAG288=y ++CONFIG_VFIO_AP=$(CONFIG_LINUX) +diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs +index b25ca64..a6b6039 100644 +--- a/hw/vfio/Makefile.objs ++++ b/hw/vfio/Makefile.objs +@@ -3,4 +3,5 @@ obj-$(CONFIG_SOFTMMU) += common.o + obj-$(CONFIG_PCI) += pci.o pci-quirks.o display.o + obj-$(CONFIG_VFIO_CCW) += ccw.o + obj-$(CONFIG_SOFTMMU) += spapr.o ++obj-$(CONFIG_VFIO_AP) += ap.o + endif +diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c +new file mode 100644 +index 0000000..3962bb7 +--- /dev/null ++++ b/hw/vfio/ap.c +@@ -0,0 +1,181 @@ ++/* ++ * VFIO based AP matrix device assignment ++ * ++ * Copyright 2018 IBM Corp. ++ * Author(s): Tony Krowiak ++ * Halil Pasic ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or (at ++ * your option) any later version. See the COPYING file in the top-level ++ * directory. ++ */ ++ ++#include ++#include ++#include "qemu/osdep.h" ++#include "qapi/error.h" ++#include "hw/sysbus.h" ++#include "hw/vfio/vfio.h" ++#include "hw/vfio/vfio-common.h" ++#include "hw/s390x/ap-device.h" ++#include "qemu/error-report.h" ++#include "qemu/queue.h" ++#include "qemu/option.h" ++#include "qemu/config-file.h" ++#include "cpu.h" ++#include "kvm_s390x.h" ++#include "sysemu/sysemu.h" ++#include "hw/s390x/ap-bridge.h" ++#include "exec/address-spaces.h" ++ ++#define VFIO_AP_DEVICE_TYPE "vfio-ap" ++ ++typedef struct VFIOAPDevice { ++ APDevice apdev; ++ VFIODevice vdev; ++} VFIOAPDevice; ++ ++#define VFIO_AP_DEVICE(obj) \ ++ OBJECT_CHECK(VFIOAPDevice, (obj), VFIO_AP_DEVICE_TYPE) ++ ++static void vfio_ap_compute_needs_reset(VFIODevice *vdev) ++{ ++ vdev->needs_reset = false; ++} ++ ++/* ++ * We don't need vfio_hot_reset_multi and vfio_eoi operations for ++ * vfio-ap device now. ++ */ ++struct VFIODeviceOps vfio_ap_ops = { ++ .vfio_compute_needs_reset = vfio_ap_compute_needs_reset, ++}; ++ ++static void vfio_ap_put_device(VFIOAPDevice *vapdev) ++{ ++ g_free(vapdev->vdev.name); ++ vfio_put_base_device(&vapdev->vdev); ++} ++ ++static VFIOGroup *vfio_ap_get_group(VFIOAPDevice *vapdev, Error **errp) ++{ ++ GError *gerror = NULL; ++ char *symlink, *group_path; ++ int groupid; ++ ++ symlink = g_strdup_printf("%s/iommu_group", vapdev->vdev.sysfsdev); ++ group_path = g_file_read_link(symlink, &gerror); ++ g_free(symlink); ++ ++ if (!group_path) { ++ error_setg(errp, "%s: no iommu_group found for %s: %s", ++ VFIO_AP_DEVICE_TYPE, vapdev->vdev.sysfsdev, gerror->message); ++ return NULL; ++ } ++ ++ if (sscanf(basename(group_path), "%d", &groupid) != 1) { ++ error_setg(errp, "vfio: failed to read %s", group_path); ++ g_free(group_path); ++ return NULL; ++ } ++ ++ g_free(group_path); ++ ++ return vfio_get_group(groupid, &address_space_memory, errp); ++} ++ ++static void vfio_ap_realize(DeviceState *dev, Error **errp) ++{ ++ int ret; ++ char *mdevid; ++ Error *local_err = NULL; ++ VFIOGroup *vfio_group; ++ APDevice *apdev = AP_DEVICE(dev); ++ VFIOAPDevice *vapdev = VFIO_AP_DEVICE(apdev); ++ ++ vfio_group = vfio_ap_get_group(vapdev, &local_err); ++ if (!vfio_group) { ++ goto out_err; ++ } ++ ++ vapdev->vdev.ops = &vfio_ap_ops; ++ vapdev->vdev.type = VFIO_DEVICE_TYPE_AP; ++ mdevid = basename(vapdev->vdev.sysfsdev); ++ vapdev->vdev.name = g_strdup_printf("%s", mdevid); ++ vapdev->vdev.dev = dev; ++ ++ ret = vfio_get_device(vfio_group, mdevid, &vapdev->vdev, &local_err); ++ if (ret) { ++ goto out_get_dev_err; ++ } ++ ++ return; ++ ++out_get_dev_err: ++ vfio_ap_put_device(vapdev); ++ vfio_put_group(vfio_group); ++out_err: ++ error_propagate(errp, local_err); ++} ++ ++static void vfio_ap_unrealize(DeviceState *dev, Error **errp) ++{ ++ APDevice *apdev = AP_DEVICE(dev); ++ VFIOAPDevice *vapdev = VFIO_AP_DEVICE(apdev); ++ VFIOGroup *group = vapdev->vdev.group; ++ ++ vfio_ap_put_device(vapdev); ++ vfio_put_group(group); ++} ++ ++static Property vfio_ap_properties[] = { ++ DEFINE_PROP_STRING("sysfsdev", VFIOAPDevice, vdev.sysfsdev), ++ DEFINE_PROP_END_OF_LIST(), ++}; ++ ++static void vfio_ap_reset(DeviceState *dev) ++{ ++ int ret; ++ APDevice *apdev = AP_DEVICE(dev); ++ VFIOAPDevice *vapdev = VFIO_AP_DEVICE(apdev); ++ ++ ret = ioctl(vapdev->vdev.fd, VFIO_DEVICE_RESET); ++ if (ret) { ++ error_report("%s: failed to reset %s device: %s", __func__, ++ vapdev->vdev.name, strerror(ret)); ++ } ++} ++ ++static const VMStateDescription vfio_ap_vmstate = { ++ .name = VFIO_AP_DEVICE_TYPE, ++ .unmigratable = 1, ++}; ++ ++static void vfio_ap_class_init(ObjectClass *klass, void *data) ++{ ++ DeviceClass *dc = DEVICE_CLASS(klass); ++ ++ dc->props = vfio_ap_properties; ++ dc->vmsd = &vfio_ap_vmstate; ++ dc->desc = "VFIO-based AP device assignment"; ++ set_bit(DEVICE_CATEGORY_MISC, dc->categories); ++ dc->realize = vfio_ap_realize; ++ dc->unrealize = vfio_ap_unrealize; ++ dc->hotpluggable = false; ++ dc->reset = vfio_ap_reset; ++ dc->bus_type = TYPE_AP_BUS; ++} ++ ++static const TypeInfo vfio_ap_info = { ++ .name = VFIO_AP_DEVICE_TYPE, ++ .parent = AP_DEVICE_TYPE, ++ .instance_size = sizeof(VFIOAPDevice), ++ .class_init = vfio_ap_class_init, ++}; ++ ++static void vfio_ap_type_init(void) ++{ ++ type_register_static(&vfio_ap_info); ++} ++ ++type_init(vfio_ap_type_init) +diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h +index d936014..f29df6e 100644 +--- a/include/hw/vfio/vfio-common.h ++++ b/include/hw/vfio/vfio-common.h +@@ -47,6 +47,7 @@ enum { + VFIO_DEVICE_TYPE_PCI = 0, + VFIO_DEVICE_TYPE_PLATFORM = 1, + VFIO_DEVICE_TYPE_CCW = 2, ++ VFIO_DEVICE_TYPE_AP = 3, + }; + + typedef struct VFIOMmap { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-s390x.conf b/SOURCES/kvm-s390x.conf new file mode 100644 index 0000000..d82b818 --- /dev/null +++ b/SOURCES/kvm-s390x.conf @@ -0,0 +1,19 @@ +# User changes in this file are preserved across upgrades. +# +# Setting "modprobe kvm nested=1" only enables Nested Virtualization until +# the next reboot or module reload. Uncomment the option below to enable +# the feature permanently. +# +#options kvm nested=1 +# +# +# Setting "modprobe kvm hpage=1" only enables Huge Page Backing (1MB) +# support until the next reboot or module reload. Uncomment the option +# below to enable the feature permanently. +# +# Note: - Incompatible with "nested=1". Loading the module will fail. +# - Dirty page logging will be performed on a 1MB (not 4KB) basis, +# which can result in a lot of data having to be transferred during +# migration, and therefore taking very long to converge. +# +#options kvm hpage=1 diff --git a/SOURCES/kvm-scripts-qemu.py-allow-adding-to-the-list-of-extra-ar.patch b/SOURCES/kvm-scripts-qemu.py-allow-adding-to-the-list-of-extra-ar.patch new file mode 100644 index 0000000..5e034bf --- /dev/null +++ b/SOURCES/kvm-scripts-qemu.py-allow-adding-to-the-list-of-extra-ar.patch @@ -0,0 +1,53 @@ +From 1b7816394c8475dce6033b6f6cd4b57785898345 Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:36 +0000 +Subject: [PATCH 08/13] scripts/qemu.py: allow adding to the list of extra + arguments +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: +Patchwork-id: 83438 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 2/7] scripts/qemu.py: allow adding to the list of extra arguments +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +Tests will often need to add extra arguments to QEMU command +line arguments. + +Signed-off-by: Cleber Rosa +Reviewed-by: Stefan Hajnoczi +Message-Id: <20180530184156.15634-3-crosa@redhat.com> +Reviewed-by: Philippe Mathieu-Daudé +Tested-by: Philippe Mathieu-Daudé +Signed-off-by: Eduardo Habkost +(cherry picked from commit 572a82438308216fbfc615c924b6e94e4b68dfaa) +Signed-off-by: Yash Mankad +Signed-off-by: Danilo C. L. de Paula +--- + scripts/qemu.py | 6 ++++++ + 1 file changed, 6 insertions(+) + +diff --git a/scripts/qemu.py b/scripts/qemu.py +index 08a3e9a..7cd8193 100644 +--- a/scripts/qemu.py ++++ b/scripts/qemu.py +@@ -359,3 +359,9 @@ class QEMUMachine(object): + of the qemu process. + ''' + return self._iolog ++ ++ def add_args(self, *args): ++ ''' ++ Adds to the list of extra arguments to be given to the QEMU binary ++ ''' ++ self._args.extend(args) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scripts-qemu.py-introduce-set_console-method.patch b/SOURCES/kvm-scripts-qemu.py-introduce-set_console-method.patch new file mode 100644 index 0000000..9a5a684 --- /dev/null +++ b/SOURCES/kvm-scripts-qemu.py-introduce-set_console-method.patch @@ -0,0 +1,197 @@ +From 99213f4ec15e0f5d1c5cae562e1010091da4a66e Mon Sep 17 00:00:00 2001 +From: Yash Mankad +Date: Wed, 12 Dec 2018 00:14:38 +0000 +Subject: [PATCH 10/13] scripts/qemu.py: introduce set_console() method +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Yash Mankad +Message-id: <76ce1001806fed23760f7c02b8e8472d793ef8b1.1544573601.git.ymankad@redhat.com> +Patchwork-id: 83435 +O-Subject: [RHEL-8.0 qemu-kvm PATCH v2 4/7] scripts/qemu.py: introduce set_console() method +Bugzilla: 1655807 +RH-Acked-by: Eduardo Habkost +RH-Acked-by: John Snow +RH-Acked-by: Philippe Mathieu-Daudé + +From: Cleber Rosa + +The set_console() method is intended to ease higher level use cases +that require a console device. + +The amount of intelligence is limited on purpose, requiring either the +device type explicitly, or the existence of a machine (pattern) +definition. + +Because of the console device type selection criteria (by machine +type), users should also be able to define that. It'll then be used +for both '-machine' and for the console device type selection. + +Users of the set_console() method will certainly be interested in +accessing the console device, and for that a console_socket property +has been added. + +Signed-off-by: Cleber Rosa +Message-Id: <20180530184156.15634-5-crosa@redhat.com> +Tested-by: Philippe Mathieu-Daudé +Reviewed-by: Stefan Hajnoczi +Signed-off-by: Eduardo Habkost +(cherry picked from commit 22dea9db2baf72a79782c748c57e2d87b06234d5) +Signed-off-by: Yash Mankad +Signed-off-by: Danilo C. L. de Paula +--- + scripts/qemu.py | 97 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++- + 1 file changed, 96 insertions(+), 1 deletion(-) + +diff --git a/scripts/qemu.py b/scripts/qemu.py +index 7cd8193..f099ce7 100644 +--- a/scripts/qemu.py ++++ b/scripts/qemu.py +@@ -17,19 +17,41 @@ import logging + import os + import subprocess + import qmp.qmp ++import re + import shutil ++import socket + import tempfile + + + LOG = logging.getLogger(__name__) + + ++#: Maps machine types to the preferred console device types ++CONSOLE_DEV_TYPES = { ++ r'^clipper$': 'isa-serial', ++ r'^malta': 'isa-serial', ++ r'^(pc.*|q35.*|isapc)$': 'isa-serial', ++ r'^(40p|powernv|prep)$': 'isa-serial', ++ r'^pseries.*': 'spapr-vty', ++ r'^s390-ccw-virtio.*': 'sclpconsole', ++ } ++ ++ + class QEMUMachineError(Exception): + """ + Exception called when an error in QEMUMachine happens. + """ + + ++class QEMUMachineAddDeviceError(QEMUMachineError): ++ """ ++ Exception raised when a request to add a device can not be fulfilled ++ ++ The failures are caused by limitations, lack of information or conflicting ++ requests on the QEMUMachine methods. This exception does not represent ++ failures reported by the QEMU binary itself. ++ """ ++ + class MonitorResponseError(qmp.qmp.QMPError): + ''' + Represents erroneous QMP monitor reply +@@ -91,6 +113,10 @@ class QEMUMachine(object): + self._test_dir = test_dir + self._temp_dir = None + self._launched = False ++ self._machine = None ++ self._console_device_type = None ++ self._console_address = None ++ self._console_socket = None + + # just in case logging wasn't configured by the main script: + logging.basicConfig() +@@ -175,9 +201,19 @@ class QEMUMachine(object): + self._monitor_address[1]) + else: + moncdev = 'socket,id=mon,path=%s' % self._vm_monitor +- return ['-chardev', moncdev, ++ args = ['-chardev', moncdev, + '-mon', 'chardev=mon,mode=control', + '-display', 'none', '-vga', 'none'] ++ if self._machine is not None: ++ args.extend(['-machine', self._machine]) ++ if self._console_device_type is not None: ++ self._console_address = os.path.join(self._temp_dir, ++ self._name + "-console.sock") ++ chardev = ('socket,id=console,path=%s,server,nowait' % ++ self._console_address) ++ device = '%s,chardev=console' % self._console_device_type ++ args.extend(['-chardev', chardev, '-device', device]) ++ return args + + def _pre_launch(self): + self._temp_dir = tempfile.mkdtemp(dir=self._test_dir) +@@ -202,6 +238,10 @@ class QEMUMachine(object): + + self._qemu_log_path = None + ++ if self._console_socket is not None: ++ self._console_socket.close() ++ self._console_socket = None ++ + if self._temp_dir is not None: + shutil.rmtree(self._temp_dir) + self._temp_dir = None +@@ -365,3 +405,58 @@ class QEMUMachine(object): + Adds to the list of extra arguments to be given to the QEMU binary + ''' + self._args.extend(args) ++ ++ def set_machine(self, machine_type): ++ ''' ++ Sets the machine type ++ ++ If set, the machine type will be added to the base arguments ++ of the resulting QEMU command line. ++ ''' ++ self._machine = machine_type ++ ++ def set_console(self, device_type=None): ++ ''' ++ Sets the device type for a console device ++ ++ If set, the console device and a backing character device will ++ be added to the base arguments of the resulting QEMU command ++ line. ++ ++ This is a convenience method that will either use the provided ++ device type, of if not given, it will used the device type set ++ on CONSOLE_DEV_TYPES. ++ ++ The actual setting of command line arguments will be be done at ++ machine launch time, as it depends on the temporary directory ++ to be created. ++ ++ @param device_type: the device type, such as "isa-serial" ++ @raises: QEMUMachineAddDeviceError if the device type is not given ++ and can not be determined. ++ ''' ++ if device_type is None: ++ if self._machine is None: ++ raise QEMUMachineAddDeviceError("Can not add a console device:" ++ " QEMU instance without a " ++ "defined machine type") ++ for regex, device in CONSOLE_DEV_TYPES.items(): ++ if re.match(regex, self._machine): ++ device_type = device ++ break ++ if device_type is None: ++ raise QEMUMachineAddDeviceError("Can not add a console device:" ++ " no matching console device " ++ "type definition") ++ self._console_device_type = device_type ++ ++ @property ++ def console_socket(self): ++ """ ++ Returns a socket connected to the console ++ """ ++ if self._console_socket is None: ++ self._console_socket = socket.socket(socket.AF_UNIX, ++ socket.SOCK_STREAM) ++ self._console_socket.connect(self._console_address) ++ return self._console_socket +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scsi-disk-Block-Device-Characteristics-emulation-fix.patch b/SOURCES/kvm-scsi-disk-Block-Device-Characteristics-emulation-fix.patch new file mode 100644 index 0000000..4524a31 --- /dev/null +++ b/SOURCES/kvm-scsi-disk-Block-Device-Characteristics-emulation-fix.patch @@ -0,0 +1,84 @@ +From 3c469960b02ee2645f582a8f1c072308754aff96 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:30:59 +0000 +Subject: [PATCH 4/8] scsi-disk: Block Device Characteristics emulation fix +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-5-pbonzini@redhat.com> +Patchwork-id: 83717 +O-Subject: [PATCH 4/8] scsi-disk: Block Device Characteristics emulation fix +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +From: Daniel Henrique Barboza + +The current BDC VPD page (page 0xb1) is too short. This can be +seen running sg_utils: + +$ sg_vpd --page=bdc /dev/sda +Block device characteristics VPD page (SBC): +Block device characteristics VPD page length too short=8 + +By the SCSI spec, the expected size of the SBC page is 0x40. +There is no telling how the guest will behave with a shorter +message - it can ignore it, or worse, make (wrong) +assumptions. + +This patch fixes the emulation by setting the size to 0x40. +This is the output of the previous sg_vpd command after +applying it: + +$ sg_vpd --page=bdc /dev/sda -v + inquiry cdb: 12 01 b1 00 fc 00 +Block device characteristics VPD page (SBC): + [PQual=0 Peripheral device type: disk] + Medium rotation rate is not reported + Product type: Not specified + WABEREQ=0 + WACEREQ=0 + Nominal form factor not reported + FUAB=0 + VBULS=0 + +To improve readability, this patch also adds the VBULS value +explictly and add comments on the existing fields we're +setting. + +Signed-off-by: Daniel Henrique Barboza +Acked-by: Paolo Bonzini +Signed-off-by: Kevin Wolf +(cherry picked from commit 740842c9656cd5dbc9ccf2ea0c3a74f0ba35144a) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-disk.c | 7 ++++--- + 1 file changed, 4 insertions(+), 3 deletions(-) + +diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c +index b3d53ec..b8a27d6 100644 +--- a/hw/scsi/scsi-disk.c ++++ b/hw/scsi/scsi-disk.c +@@ -774,11 +774,12 @@ int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf) + } + case 0xb1: /* block device characteristics */ + { +- buflen = 8; ++ buflen = 0x40; + outbuf[4] = (s->rotation_rate >> 8) & 0xff; + outbuf[5] = s->rotation_rate & 0xff; +- outbuf[6] = 0; +- outbuf[7] = 0; ++ outbuf[6] = 0; /* PRODUCT TYPE */ ++ outbuf[7] = 0; /* WABEREQ | WACEREQ | NOMINAL FORM FACTOR */ ++ outbuf[8] = 0; /* VBULS */ + break; + } + case 0xb2: /* thin provisioning */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scsi-generic-avoid-invalid-access-to-struct-when-emu.patch b/SOURCES/kvm-scsi-generic-avoid-invalid-access-to-struct-when-emu.patch new file mode 100644 index 0000000..fe5a067 --- /dev/null +++ b/SOURCES/kvm-scsi-generic-avoid-invalid-access-to-struct-when-emu.patch @@ -0,0 +1,336 @@ +From 00c5ee2a99e03dd349061981d6560975ab9707d0 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:31:02 +0000 +Subject: [PATCH 7/8] scsi-generic: avoid invalid access to struct when + emulating block limits +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-8-pbonzini@redhat.com> +Patchwork-id: 83718 +O-Subject: [PATCH 7/8] scsi-generic: avoid invalid access to struct when emulating block limits +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +Emulation of the block limits VPD page called back into scsi-disk.c, +which however expected the request to be for a SCSIDiskState and +accessed a scsi-generic device outside the bounds of its struct +(namely to retrieve s->max_unmap_size and s->max_io_size). + +To avoid this, move the emulation code to a separate function that +takes a new SCSIBlockLimits struct and marshals it into the VPD +response format. + +Reported-by: Max Reitz +Reviewed-by: Max Reitz +Signed-off-by: Paolo Bonzini +(cherry picked from commit 3d4a8bf0eed68a781e06118e4d1df6e2f106a1f2) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/Makefile.objs | 2 +- + hw/scsi/emulation.c | 42 +++++++++++++++++++++ + hw/scsi/scsi-disk.c | 92 ++++++++++----------------------------------- + hw/scsi/scsi-generic.c | 35 ++++++++++++----- + include/hw/scsi/emulation.h | 16 ++++++++ + include/hw/scsi/scsi.h | 1 - + 6 files changed, 104 insertions(+), 84 deletions(-) + create mode 100644 hw/scsi/emulation.c + create mode 100644 include/hw/scsi/emulation.h + +diff --git a/hw/scsi/Makefile.objs b/hw/scsi/Makefile.objs +index b188f72..c4f249a 100644 +--- a/hw/scsi/Makefile.objs ++++ b/hw/scsi/Makefile.objs +@@ -1,4 +1,4 @@ +-common-obj-y += scsi-disk.o ++common-obj-y += scsi-disk.o emulation.o + common-obj-y += scsi-generic.o scsi-bus.o + common-obj-$(CONFIG_LSI_SCSI_PCI) += lsi53c895a.o + common-obj-$(CONFIG_MPTSAS_SCSI_PCI) += mptsas.o mptconfig.o mptendian.o +diff --git a/hw/scsi/emulation.c b/hw/scsi/emulation.c +new file mode 100644 +index 0000000..06d62f3 +--- /dev/null ++++ b/hw/scsi/emulation.c +@@ -0,0 +1,42 @@ ++#include "qemu/osdep.h" ++#include "qemu/units.h" ++#include "qemu/bswap.h" ++#include "hw/scsi/emulation.h" ++ ++int scsi_emulate_block_limits(uint8_t *outbuf, const SCSIBlockLimits *bl) ++{ ++ /* required VPD size with unmap support */ ++ memset(outbuf, 0, 0x3c); ++ ++ outbuf[0] = bl->wsnz; /* wsnz */ ++ ++ if (bl->max_io_sectors) { ++ /* optimal transfer length granularity. This field and the optimal ++ * transfer length can't be greater than maximum transfer length. ++ */ ++ stw_be_p(outbuf + 2, MIN(bl->min_io_size, bl->max_io_sectors)); ++ ++ /* maximum transfer length */ ++ stl_be_p(outbuf + 4, bl->max_io_sectors); ++ ++ /* optimal transfer length */ ++ stl_be_p(outbuf + 8, MIN(bl->opt_io_size, bl->max_io_sectors)); ++ } else { ++ stw_be_p(outbuf + 2, bl->min_io_size); ++ stl_be_p(outbuf + 8, bl->opt_io_size); ++ } ++ ++ /* max unmap LBA count */ ++ stl_be_p(outbuf + 16, bl->max_unmap_sectors); ++ ++ /* max unmap descriptors */ ++ stl_be_p(outbuf + 20, bl->max_unmap_descr); ++ ++ /* optimal unmap granularity; alignment is zero */ ++ stl_be_p(outbuf + 24, bl->unmap_sectors); ++ ++ /* max write same size, make it the same as maximum transfer length */ ++ stl_be_p(outbuf + 36, bl->max_io_sectors); ++ ++ return 0x3c; ++} +diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c +index b8a27d6..a20ef91 100644 +--- a/hw/scsi/scsi-disk.c ++++ b/hw/scsi/scsi-disk.c +@@ -32,6 +32,7 @@ do { printf("scsi-disk: " fmt , ## __VA_ARGS__); } while (0) + #include "qapi/error.h" + #include "qemu/error-report.h" + #include "hw/scsi/scsi.h" ++#include "hw/scsi/emulation.h" + #include "scsi/constants.h" + #include "sysemu/sysemu.h" + #include "sysemu/block-backend.h" +@@ -585,7 +586,7 @@ static uint8_t *scsi_get_buf(SCSIRequest *req) + return (uint8_t *)r->iov.iov_base; + } + +-int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf) ++static int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf) + { + SCSIDiskState *s = DO_UPCAST(SCSIDiskState, qdev, req->dev); + uint8_t page_code = req->cmd.buf[2]; +@@ -687,89 +688,36 @@ int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf) + } + case 0xb0: /* block limits */ + { +- unsigned int unmap_sectors = +- s->qdev.conf.discard_granularity / s->qdev.blocksize; +- unsigned int min_io_size = +- s->qdev.conf.min_io_size / s->qdev.blocksize; +- unsigned int opt_io_size = +- s->qdev.conf.opt_io_size / s->qdev.blocksize; +- unsigned int max_unmap_sectors = +- s->max_unmap_size / s->qdev.blocksize; +- unsigned int max_io_sectors = +- s->max_io_size / s->qdev.blocksize; ++ SCSIBlockLimits bl = {}; + + if (s->qdev.type == TYPE_ROM) { + DPRINTF("Inquiry (EVPD[%02X] not supported for CDROM\n", + page_code); + return -1; + } ++ bl.wsnz = 1; ++ bl.unmap_sectors = ++ s->qdev.conf.discard_granularity / s->qdev.blocksize; ++ bl.min_io_size = ++ s->qdev.conf.min_io_size / s->qdev.blocksize; ++ bl.opt_io_size = ++ s->qdev.conf.opt_io_size / s->qdev.blocksize; ++ bl.max_unmap_sectors = ++ s->max_unmap_size / s->qdev.blocksize; ++ bl.max_io_sectors = ++ s->max_io_size / s->qdev.blocksize; ++ /* 255 descriptors fit in 4 KiB with an 8-byte header */ ++ bl.max_unmap_descr = 255; ++ + if (s->qdev.type == TYPE_DISK) { + int max_transfer_blk = blk_get_max_transfer(s->qdev.conf.blk); + int max_io_sectors_blk = + max_transfer_blk / s->qdev.blocksize; + +- max_io_sectors = +- MIN_NON_ZERO(max_io_sectors_blk, max_io_sectors); +- +- /* min_io_size and opt_io_size can't be greater than +- * max_io_sectors */ +- if (min_io_size) { +- min_io_size = MIN(min_io_size, max_io_sectors); +- } +- if (opt_io_size) { +- opt_io_size = MIN(opt_io_size, max_io_sectors); +- } ++ bl.max_io_sectors = ++ MIN_NON_ZERO(max_io_sectors_blk, bl.max_io_sectors); + } +- /* required VPD size with unmap support */ +- buflen = 0x40; +- memset(outbuf + 4, 0, buflen - 4); +- +- outbuf[4] = 0x1; /* wsnz */ +- +- /* optimal transfer length granularity */ +- outbuf[6] = (min_io_size >> 8) & 0xff; +- outbuf[7] = min_io_size & 0xff; +- +- /* maximum transfer length */ +- outbuf[8] = (max_io_sectors >> 24) & 0xff; +- outbuf[9] = (max_io_sectors >> 16) & 0xff; +- outbuf[10] = (max_io_sectors >> 8) & 0xff; +- outbuf[11] = max_io_sectors & 0xff; +- +- /* optimal transfer length */ +- outbuf[12] = (opt_io_size >> 24) & 0xff; +- outbuf[13] = (opt_io_size >> 16) & 0xff; +- outbuf[14] = (opt_io_size >> 8) & 0xff; +- outbuf[15] = opt_io_size & 0xff; +- +- /* max unmap LBA count, default is 1GB */ +- outbuf[20] = (max_unmap_sectors >> 24) & 0xff; +- outbuf[21] = (max_unmap_sectors >> 16) & 0xff; +- outbuf[22] = (max_unmap_sectors >> 8) & 0xff; +- outbuf[23] = max_unmap_sectors & 0xff; +- +- /* max unmap descriptors, 255 fit in 4 kb with an 8-byte header */ +- outbuf[24] = 0; +- outbuf[25] = 0; +- outbuf[26] = 0; +- outbuf[27] = 255; +- +- /* optimal unmap granularity */ +- outbuf[28] = (unmap_sectors >> 24) & 0xff; +- outbuf[29] = (unmap_sectors >> 16) & 0xff; +- outbuf[30] = (unmap_sectors >> 8) & 0xff; +- outbuf[31] = unmap_sectors & 0xff; +- +- /* max write same size */ +- outbuf[36] = 0; +- outbuf[37] = 0; +- outbuf[38] = 0; +- outbuf[39] = 0; +- +- outbuf[40] = (max_io_sectors >> 24) & 0xff; +- outbuf[41] = (max_io_sectors >> 16) & 0xff; +- outbuf[42] = (max_io_sectors >> 8) & 0xff; +- outbuf[43] = max_io_sectors & 0xff; ++ buflen += scsi_emulate_block_limits(outbuf + buflen, &bl); + break; + } + case 0xb1: /* block device characteristics */ +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index 6d6db69..f218cef 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -16,6 +16,7 @@ + #include "qemu-common.h" + #include "qemu/error-report.h" + #include "hw/scsi/scsi.h" ++#include "hw/scsi/emulation.h" + #include "sysemu/block-backend.h" + #include "sysemu/blockdev.h" + +@@ -182,7 +183,7 @@ static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + /* Also take care of the opt xfer len. */ + stl_be_p(&r->buf[12], + MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); +- } else if (page == 0x00 && s->needs_vpd_bl_emulation) { ++ } else if (s->needs_vpd_bl_emulation && page == 0x00) { + /* + * Now we're capable of supplying the VPD Block Limits + * response if the hardware can't. Add it in the INQUIRY +@@ -210,9 +211,24 @@ static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + } + } + +-static int scsi_emulate_block_limits(SCSIGenericReq *r) ++static int scsi_generic_emulate_block_limits(SCSIGenericReq *r, SCSIDevice *s) + { +- r->buflen = scsi_disk_emulate_vpd_page(&r->req, r->buf); ++ int len; ++ uint8_t buf[64]; ++ ++ SCSIBlockLimits bl = { ++ .max_io_sectors = blk_get_max_transfer(s->conf.blk) / s->blocksize ++ }; ++ ++ memset(r->buf, 0, r->buflen); ++ stb_p(buf, s->type); ++ stb_p(buf + 1, 0xb0); ++ len = scsi_emulate_block_limits(buf + 4, &bl); ++ assert(len <= sizeof(buf) - 4); ++ stw_be_p(buf + 2, len); ++ ++ memcpy(r->buf, buf, MIN(r->buflen, len + 4)); ++ + r->io_header.sb_len_wr = 0; + + /* +@@ -254,13 +270,12 @@ static void scsi_read_complete(void * opaque, int ret) + * resulted in sense error but would need emulation. + * In this case, emulate a valid VPD response. + */ +- if (s->needs_vpd_bl_emulation) { +- int is_vpd_bl = r->req.cmd.buf[0] == INQUIRY && +- r->req.cmd.buf[1] & 0x01 && +- r->req.cmd.buf[2] == 0xb0; +- +- if (is_vpd_bl && sg_io_sense_from_errno(-ret, &r->io_header, &sense)) { +- len = scsi_emulate_block_limits(r); ++ if (s->needs_vpd_bl_emulation && ++ r->req.cmd.buf[0] == INQUIRY && ++ (r->req.cmd.buf[1] & 0x01) && ++ r->req.cmd.buf[2] == 0xb0) { ++ if (sg_io_sense_from_errno(-ret, &r->io_header, &sense)) { ++ len = scsi_generic_emulate_block_limits(r, s); + /* + * No need to let scsi_read_complete go on and handle an + * INQUIRY VPD BL request we created manually. +diff --git a/include/hw/scsi/emulation.h b/include/hw/scsi/emulation.h +new file mode 100644 +index 0000000..09fba1f +--- /dev/null ++++ b/include/hw/scsi/emulation.h +@@ -0,0 +1,16 @@ ++#ifndef HW_SCSI_EMULATION_H ++#define HW_SCSI_EMULATION_H 1 ++ ++typedef struct SCSIBlockLimits { ++ bool wsnz; ++ uint16_t min_io_size; ++ uint32_t max_unmap_descr; ++ uint32_t opt_io_size; ++ uint32_t max_unmap_sectors; ++ uint32_t unmap_sectors; ++ uint32_t max_io_sectors; ++} SCSIBlockLimits; ++ ++int scsi_emulate_block_limits(uint8_t *outbuf, const SCSIBlockLimits *bl); ++ ++#endif +diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h +index ee3a411..acef25f 100644 +--- a/include/hw/scsi/scsi.h ++++ b/include/hw/scsi/scsi.h +@@ -189,7 +189,6 @@ void scsi_device_report_change(SCSIDevice *dev, SCSISense sense); + void scsi_device_unit_attention_reported(SCSIDevice *dev); + void scsi_generic_read_device_inquiry(SCSIDevice *dev); + int scsi_device_get_sense(SCSIDevice *dev, uint8_t *buf, int len, bool fixed); +-int scsi_disk_emulate_vpd_page(SCSIRequest *req, uint8_t *outbuf); + int scsi_SG_IO_FROM_DEV(BlockBackend *blk, uint8_t *cmd, uint8_t cmd_size, + uint8_t *buf, uint8_t buf_size); + SCSIDevice *scsi_device_find(SCSIBus *bus, int channel, int target, int lun); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scsi-generic-avoid-out-of-bounds-access-to-VPD-page-.patch b/SOURCES/kvm-scsi-generic-avoid-out-of-bounds-access-to-VPD-page-.patch new file mode 100644 index 0000000..1c1cf76 --- /dev/null +++ b/SOURCES/kvm-scsi-generic-avoid-out-of-bounds-access-to-VPD-page-.patch @@ -0,0 +1,49 @@ +From bc4b0e1c80d9f9e2acf44ef99b7771f96be90791 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:31:01 +0000 +Subject: [PATCH 6/8] scsi-generic: avoid out-of-bounds access to VPD page list +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-7-pbonzini@redhat.com> +Patchwork-id: 83715 +O-Subject: [PATCH 6/8] scsi-generic: avoid out-of-bounds access to VPD page list +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +A device can report an excessive number of VPD pages when asked for a +list; this can cause an out-of-bounds access to buf in +scsi_generic_set_vpd_bl_emulation. It should not happen, but +it is technically not incorrect so handle it: do not check any byte +past the allocation length that was sent to the INQUIRY command. + +Reported-by: Max Reitz +Reviewed-by: Max Reitz +Signed-off-by: Paolo Bonzini +(cherry picked from commit 57dbb58d800f62b9e56d946660dba4e8dbd20204) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-generic.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index 98c6a34..6d6db69 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -539,7 +539,7 @@ static void scsi_generic_set_vpd_bl_emulation(SCSIDevice *s) + } + + page_len = buf[3]; +- for (i = 4; i < page_len + 4; i++) { ++ for (i = 4; i < MIN(sizeof(buf), page_len + 4); i++) { + if (buf[i] == 0xb0) { + s->needs_vpd_bl_emulation = false; + return; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scsi-generic-avoid-possible-out-of-bounds-access-to-.patch b/SOURCES/kvm-scsi-generic-avoid-possible-out-of-bounds-access-to-.patch new file mode 100644 index 0000000..a9f4d7e --- /dev/null +++ b/SOURCES/kvm-scsi-generic-avoid-possible-out-of-bounds-access-to-.patch @@ -0,0 +1,85 @@ +From 63cee29a64ac6a02695a91f7a8f29ac0c17ef3f0 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Tue, 26 Feb 2019 14:44:14 +0000 +Subject: [PATCH] scsi-generic: avoid possible out-of-bounds access to r->buf + +RH-Author: Paolo Bonzini +Message-id: <20190226144414.5700-1-pbonzini@redhat.com> +Patchwork-id: 84717 +O-Subject: [RHEL8.0 qemu-kvm PATCH] scsi-generic: avoid possible out-of-bounds access to r->buf +Bugzilla: 1668162 +RH-Acked-by: Thomas Huth +RH-Acked-by: Kevin Wolf +RH-Acked-by: Stefan Hajnoczi + +Bugzilla: 1668162 + +Brew build: 20372796 + +Whenever the allocation length of a SCSI request is shorter than the size of the +VPD page list, page_idx is used blindly to index into r->buf. Even though +the stores in the insertion sort are protected against overflows, the same is not +true of the reads and the final store of 0xb0. + +This basically does the same thing as commit 57dbb58d80 ("scsi-generic: avoid +out-of-bounds access to VPD page list", 2018-11-06), except that here the +allocation length can be chosen by the guest. Note that according to the SCSI +standard, the contents of the PAGE LENGTH field are not altered based +on the allocation length. + +The code was introduced by commit 6c219fc8a1 ("scsi-generic: keep VPD +page list sorted", 2018-11-06) but the overflow was already possible before. + +Reported-by: Kevin Wolf +Fixes: a71c775b24ebc664129eb1d9b4c360590353efd5 +Signed-off-by: Paolo Bonzini +(cherry picked from commit e909ff93698851777faac3c45d03c1b73f311ea6) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-generic.c | 18 ++++++++++-------- + 1 file changed, 10 insertions(+), 8 deletions(-) + +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index 4ac53e4..e21adf9 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -183,7 +183,7 @@ static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + /* Also take care of the opt xfer len. */ + stl_be_p(&r->buf[12], + MIN_NON_ZERO(max_transfer, ldl_be_p(&r->buf[12]))); +- } else if (s->needs_vpd_bl_emulation && page == 0x00) { ++ } else if (s->needs_vpd_bl_emulation && page == 0x00 && r->buflen >= 4) { + /* + * Now we're capable of supplying the VPD Block Limits + * response if the hardware can't. Add it in the INQUIRY +@@ -194,18 +194,20 @@ static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + * and will use it to proper setup the SCSI device. + * + * VPD page numbers must be sorted, so insert 0xb0 at the +- * right place with an in-place insert. After the initialization +- * part of the for loop is executed, the device response is +- * at r[0] to r[page_idx - 1]. ++ * right place with an in-place insert. When the while loop ++ * begins the device response is at r[0] to r[page_idx - 1]. + */ +- for (page_idx = lduw_be_p(r->buf + 2) + 4; +- page_idx > 4 && r->buf[page_idx - 1] >= 0xb0; +- page_idx--) { ++ page_idx = lduw_be_p(r->buf + 2) + 4; ++ page_idx = MIN(page_idx, r->buflen); ++ while (page_idx > 4 && r->buf[page_idx - 1] >= 0xb0) { + if (page_idx < r->buflen) { + r->buf[page_idx] = r->buf[page_idx - 1]; + } ++ page_idx--; ++ } ++ if (page_idx < r->buflen) { ++ r->buf[page_idx] = 0xb0; + } +- r->buf[page_idx] = 0xb0; + stw_be_p(r->buf + 2, lduw_be_p(r->buf + 2) + 1); + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scsi-generic-do-not-do-VPD-emulation-for-sense-other.patch b/SOURCES/kvm-scsi-generic-do-not-do-VPD-emulation-for-sense-other.patch new file mode 100644 index 0000000..c3dcd2d --- /dev/null +++ b/SOURCES/kvm-scsi-generic-do-not-do-VPD-emulation-for-sense-other.patch @@ -0,0 +1,58 @@ +From d186ce91950c1cd0d1ba354daca53a072390fb53 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:31:03 +0000 +Subject: [PATCH 8/8] scsi-generic: do not do VPD emulation for sense other + than ILLEGAL_REQUEST + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-9-pbonzini@redhat.com> +Patchwork-id: 83716 +O-Subject: [PATCH 8/8] scsi-generic: do not do VPD emulation for sense other than ILLEGAL_REQUEST +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +Pass other sense, such as UNIT_ATTENTION or BUSY, directly to the +guest. + +Reported-by: Max Reitz +Signed-off-by: Paolo Bonzini +(cherry picked from commit 763c56872b08b98fde062a1feca003f200e7bd5c) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-generic.c | 8 +++++--- + 1 file changed, 5 insertions(+), 3 deletions(-) + +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index f218cef..4ac53e4 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -247,7 +247,6 @@ static void scsi_read_complete(void * opaque, int ret) + { + SCSIGenericReq *r = (SCSIGenericReq *)opaque; + SCSIDevice *s = r->req.dev; +- SCSISense sense; + int len; + + assert(r->req.aiocb != NULL); +@@ -270,11 +269,14 @@ static void scsi_read_complete(void * opaque, int ret) + * resulted in sense error but would need emulation. + * In this case, emulate a valid VPD response. + */ +- if (s->needs_vpd_bl_emulation && ++ if (s->needs_vpd_bl_emulation && ret == 0 && ++ (r->io_header.driver_status & SG_ERR_DRIVER_SENSE) && + r->req.cmd.buf[0] == INQUIRY && + (r->req.cmd.buf[1] & 0x01) && + r->req.cmd.buf[2] == 0xb0) { +- if (sg_io_sense_from_errno(-ret, &r->io_header, &sense)) { ++ SCSISense sense = ++ scsi_parse_sense_buf(r->req.sense, r->io_header.sb_len_wr); ++ if (sense.key == ILLEGAL_REQUEST) { + len = scsi_generic_emulate_block_limits(r, s); + /* + * No need to let scsi_read_complete go on and handle an +-- +1.8.3.1 + diff --git a/SOURCES/kvm-scsi-generic-keep-VPD-page-list-sorted.patch b/SOURCES/kvm-scsi-generic-keep-VPD-page-list-sorted.patch new file mode 100644 index 0000000..00d241e --- /dev/null +++ b/SOURCES/kvm-scsi-generic-keep-VPD-page-list-sorted.patch @@ -0,0 +1,70 @@ +From 3fcc45e3b9a204454910832f2c9a7fcfc28a0d07 Mon Sep 17 00:00:00 2001 +From: Paolo Bonzini +Date: Thu, 20 Dec 2018 12:31:00 +0000 +Subject: [PATCH 5/8] scsi-generic: keep VPD page list sorted + +RH-Author: Paolo Bonzini +Message-id: <20181220123103.29579-6-pbonzini@redhat.com> +Patchwork-id: 83714 +O-Subject: [PATCH 5/8] scsi-generic: keep VPD page list sorted +Bugzilla: 1639957 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Kevin Wolf +RH-Acked-by: Laurent Vivier + +Block limits emulation is just placing 0xb0 as the final byte of the +VPD pages list. However, VPD page numbers must be sorted, so change +that to an in-place insert. Since I couldn't find any disk that triggered +the loop more than once, this was tested by adding manually 0xb1 +at the end of the list and checking that 0xb0 was added before. + +Reported-by: Max Reitz +Reviewed-by: Max Reitz +Signed-off-by: Paolo Bonzini +(cherry picked from commit 6c219fc8a112fc69b29f59ea2c7865717ff6e3e0) +Signed-off-by: Danilo C. L. de Paula +--- + hw/scsi/scsi-generic.c | 19 +++++++++++++++---- + 1 file changed, 15 insertions(+), 4 deletions(-) + +diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c +index 4266003..98c6a34 100644 +--- a/hw/scsi/scsi-generic.c ++++ b/hw/scsi/scsi-generic.c +@@ -145,7 +145,7 @@ static int execute_command(BlockBackend *blk, + + static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + { +- uint8_t page, page_len; ++ uint8_t page, page_idx; + + /* + * EVPD set to zero returns the standard INQUIRY data. +@@ -191,10 +191,21 @@ static void scsi_handle_inquiry_reply(SCSIGenericReq *r, SCSIDevice *s) + * + * This way, the guest kernel will be aware of the support + * and will use it to proper setup the SCSI device. ++ * ++ * VPD page numbers must be sorted, so insert 0xb0 at the ++ * right place with an in-place insert. After the initialization ++ * part of the for loop is executed, the device response is ++ * at r[0] to r[page_idx - 1]. + */ +- page_len = r->buf[3]; +- r->buf[page_len + 4] = 0xb0; +- r->buf[3] = ++page_len; ++ for (page_idx = lduw_be_p(r->buf + 2) + 4; ++ page_idx > 4 && r->buf[page_idx - 1] >= 0xb0; ++ page_idx--) { ++ if (page_idx < r->buflen) { ++ r->buf[page_idx] = r->buf[page_idx - 1]; ++ } ++ } ++ r->buf[page_idx] = 0xb0; ++ stw_be_p(r->buf + 2, lduw_be_p(r->buf + 2) + 1); + } + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-seccomp-allow-sched_setscheduler-with-SCHED_IDLE-pol.patch b/SOURCES/kvm-seccomp-allow-sched_setscheduler-with-SCHED_IDLE-pol.patch new file mode 100644 index 0000000..3f35bb5 --- /dev/null +++ b/SOURCES/kvm-seccomp-allow-sched_setscheduler-with-SCHED_IDLE-pol.patch @@ -0,0 +1,86 @@ +From a44f33e5cafc02477c182119ee422ea54eb1f3db Mon Sep 17 00:00:00 2001 +From: Eduardo Otubo +Date: Fri, 28 Sep 2018 07:56:35 +0100 +Subject: [PATCH 1/6] seccomp: allow sched_setscheduler() with SCHED_IDLE + policy +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Otubo +Message-id: <20180928075639.16746-2-otubo@redhat.com> +Patchwork-id: 82317 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/5] seccomp: allow sched_setscheduler() with SCHED_IDLE policy +Bugzilla: 1618356 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Thomas Huth + +From: Marc-André Lureau + +commit 056de1e894155fbb99e7b43c1c4382d4920cf437 +Author: Marc-André Lureau +Date: Tue Jul 10 16:55:57 2018 +0200 + + seccomp: allow sched_setscheduler() with SCHED_IDLE policy + + Current and upcoming mesa releases rely on a shader disk cash. It uses + a thread job queue with low priority, set with + sched_setscheduler(SCHED_IDLE). However, that syscall is rejected by + the "resourcecontrol" seccomp qemu filter. + + Since it should be safe to allow lowering thread priority, let's allow + scheduling thread to idle policy. + + Related to: + https://bugzilla.redhat.com/show_bug.cgi?id=1594456 + + Signed-off-by: Marc-André Lureau + Acked-by: Eduardo Otubo + +Signed-by-off: Eduardo Otubo +Signed-off-by: Danilo C. L. de Paula +--- + qemu-seccomp.c | 12 ++++++++++-- + 1 file changed, 10 insertions(+), 2 deletions(-) + +diff --git a/qemu-seccomp.c b/qemu-seccomp.c +index b770a77..845a333 100644 +--- a/qemu-seccomp.c ++++ b/qemu-seccomp.c +@@ -29,6 +29,12 @@ + struct QemuSeccompSyscall { + int32_t num; + uint8_t set; ++ uint8_t narg; ++ const struct scmp_arg_cmp *arg_cmp; ++}; ++ ++const struct scmp_arg_cmp sched_setscheduler_arg[] = { ++ SCMP_A1(SCMP_CMP_NE, SCHED_IDLE) + }; + + static const struct QemuSeccompSyscall blacklist[] = { +@@ -87,7 +93,8 @@ static const struct QemuSeccompSyscall blacklist[] = { + { SCMP_SYS(setpriority), QEMU_SECCOMP_SET_RESOURCECTL }, + { SCMP_SYS(sched_setparam), QEMU_SECCOMP_SET_RESOURCECTL }, + { SCMP_SYS(sched_getparam), QEMU_SECCOMP_SET_RESOURCECTL }, +- { SCMP_SYS(sched_setscheduler), QEMU_SECCOMP_SET_RESOURCECTL }, ++ { SCMP_SYS(sched_setscheduler), QEMU_SECCOMP_SET_RESOURCECTL, ++ ARRAY_SIZE(sched_setscheduler_arg), sched_setscheduler_arg }, + { SCMP_SYS(sched_getscheduler), QEMU_SECCOMP_SET_RESOURCECTL }, + { SCMP_SYS(sched_setaffinity), QEMU_SECCOMP_SET_RESOURCECTL }, + { SCMP_SYS(sched_getaffinity), QEMU_SECCOMP_SET_RESOURCECTL }, +@@ -113,7 +120,8 @@ int seccomp_start(uint32_t seccomp_opts) + continue; + } + +- rc = seccomp_rule_add(ctx, SCMP_ACT_KILL, blacklist[i].num, 0); ++ rc = seccomp_rule_add_array(ctx, SCMP_ACT_KILL, blacklist[i].num, ++ blacklist[i].narg, blacklist[i].arg_cmp); + if (rc < 0) { + goto seccomp_return; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-seccomp-prefer-SCMP_ACT_KILL_PROCESS-if-available.patch b/SOURCES/kvm-seccomp-prefer-SCMP_ACT_KILL_PROCESS-if-available.patch new file mode 100644 index 0000000..d1e0051 --- /dev/null +++ b/SOURCES/kvm-seccomp-prefer-SCMP_ACT_KILL_PROCESS-if-available.patch @@ -0,0 +1,110 @@ +From caa17bcb65ea65a9fa39e7d6117a87cc7cc9c0ce Mon Sep 17 00:00:00 2001 +From: Eduardo Otubo +Date: Fri, 28 Sep 2018 07:56:37 +0100 +Subject: [PATCH 3/6] seccomp: prefer SCMP_ACT_KILL_PROCESS if available +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Otubo +Message-id: <20180928075639.16746-4-otubo@redhat.com> +Patchwork-id: 82315 +O-Subject: [RHEL-8 qemu-kvm PATCH 3/5] seccomp: prefer SCMP_ACT_KILL_PROCESS if available +Bugzilla: 1618356 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Thomas Huth + +From: Marc-André Lureau + +commit bda08a5764d470f101fa38635d30b41179a313e1 +Author: Marc-André Lureau +Date: Wed Aug 22 19:02:48 2018 +0200 + + seccomp: prefer SCMP_ACT_KILL_PROCESS if available + + The upcoming libseccomp release should have SCMP_ACT_KILL_PROCESS + action (https://github.com/seccomp/libseccomp/issues/96). + + SCMP_ACT_KILL_PROCESS is preferable to immediately terminate the + offending process, rather than having the SIGSYS handler running. + + Use SECCOMP_GET_ACTION_AVAIL to check availability of kernel support, + as libseccomp will fallback on SCMP_ACT_KILL otherwise, and we still + prefer SCMP_ACT_TRAP. + + Signed-off-by: Marc-André Lureau + Reviewed-by: Daniel P. Berrangé + Acked-by: Eduardo Otubo + +Signed-off-by: Eduardo Otubo +Signed-off-by: Danilo C. L. de Paula +--- + qemu-seccomp.c | 31 ++++++++++++++++++++++++++++++- + 1 file changed, 30 insertions(+), 1 deletion(-) + +diff --git a/qemu-seccomp.c b/qemu-seccomp.c +index b88fa05..10fcfa3 100644 +--- a/qemu-seccomp.c ++++ b/qemu-seccomp.c +@@ -15,6 +15,7 @@ + #include "qemu/osdep.h" + #include + #include "sysemu/seccomp.h" ++#include + + /* For some architectures (notably ARM) cacheflush is not supported until + * libseccomp 2.2.3, but configure enforces that we are using a more recent +@@ -102,12 +103,40 @@ static const struct QemuSeccompSyscall blacklist[] = { + { SCMP_SYS(sched_get_priority_min), QEMU_SECCOMP_SET_RESOURCECTL }, + }; + ++static inline __attribute__((unused)) int ++qemu_seccomp(unsigned int operation, unsigned int flags, void *args) ++{ ++#ifdef __NR_seccomp ++ return syscall(__NR_seccomp, operation, flags, args); ++#else ++ errno = ENOSYS; ++ return -1; ++#endif ++} ++ ++static uint32_t qemu_seccomp_get_kill_action(void) ++{ ++#if defined(SECCOMP_GET_ACTION_AVAIL) && defined(SCMP_ACT_KILL_PROCESS) && \ ++ defined(SECCOMP_RET_KILL_PROCESS) ++ { ++ uint32_t action = SECCOMP_RET_KILL_PROCESS; ++ ++ if (qemu_seccomp(SECCOMP_GET_ACTION_AVAIL, 0, &action) == 0) { ++ return SCMP_ACT_KILL_PROCESS; ++ } ++ } ++#endif ++ ++ return SCMP_ACT_TRAP; ++} ++ + + int seccomp_start(uint32_t seccomp_opts) + { + int rc = 0; + unsigned int i = 0; + scmp_filter_ctx ctx; ++ uint32_t action = qemu_seccomp_get_kill_action(); + + ctx = seccomp_init(SCMP_ACT_ALLOW); + if (ctx == NULL) { +@@ -120,7 +149,7 @@ int seccomp_start(uint32_t seccomp_opts) + continue; + } + +- rc = seccomp_rule_add_array(ctx, SCMP_ACT_TRAP, blacklist[i].num, ++ rc = seccomp_rule_add_array(ctx, action, blacklist[i].num, + blacklist[i].narg, blacklist[i].arg_cmp); + if (rc < 0) { + goto seccomp_return; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-seccomp-set-the-seccomp-filter-to-all-threads.patch b/SOURCES/kvm-seccomp-set-the-seccomp-filter-to-all-threads.patch new file mode 100644 index 0000000..25cca8a --- /dev/null +++ b/SOURCES/kvm-seccomp-set-the-seccomp-filter-to-all-threads.patch @@ -0,0 +1,77 @@ +From 8bf0cdd586d9868e3432e3d7949c8d628f7e6538 Mon Sep 17 00:00:00 2001 +From: Eduardo Otubo +Date: Fri, 28 Sep 2018 07:56:39 +0100 +Subject: [PATCH 5/6] seccomp: set the seccomp filter to all threads +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Otubo +Message-id: <20180928075639.16746-6-otubo@redhat.com> +Patchwork-id: 82316 +O-Subject: [RHEL-8 qemu-kvm PATCH 5/5] seccomp: set the seccomp filter to all threads +Bugzilla: 1618356 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Thomas Huth + +From: Marc-André Lureau + +commit 70dfabeaa79ba4d7a3b699abe1a047c8012db114 +Author: Marc-André Lureau +Date: Wed Aug 22 19:02:50 2018 +0200 + + seccomp: set the seccomp filter to all threads + + When using "-seccomp on", the seccomp policy is only applied to the + main thread, the vcpu worker thread and other worker threads created + after seccomp policy is applied; the seccomp policy is not applied to + e.g. the RCU thread because it is created before the seccomp policy is + applied and SECCOMP_FILTER_FLAG_TSYNC isn't used. + + This can be verified with + for task in /proc/`pidof qemu`/task/*; do cat $task/status | grep Secc ; done + Seccomp: 2 + Seccomp: 0 + Seccomp: 0 + Seccomp: 2 + Seccomp: 2 + Seccomp: 2 + + Starting with libseccomp 2.2.0 and kernel >= 3.17, we can use + seccomp_attr_set(ctx, > SCMP_FLTATR_CTL_TSYNC, 1) to update the policy + on all threads. + + libseccomp requirement was bumped to 2.2.0 in previous patch. + libseccomp should fail to set the filter if it can't honour + SCMP_FLTATR_CTL_TSYNC (untested), and thus -sandbox will now fail on + kernel < 3.17. + + Signed-off-by: Marc-André Lureau + Acked-by: Eduardo Otubo + +Signed-off-by: Eduardo Otubo +Signed-off-by: Danilo C. L. de Paula +--- + qemu-seccomp.c | 5 +++++ + 1 file changed, 5 insertions(+) + +diff --git a/qemu-seccomp.c b/qemu-seccomp.c +index 10fcfa3..a29e54b 100644 +--- a/qemu-seccomp.c ++++ b/qemu-seccomp.c +@@ -144,6 +144,11 @@ int seccomp_start(uint32_t seccomp_opts) + goto seccomp_return; + } + ++ rc = seccomp_attr_set(ctx, SCMP_FLTATR_CTL_TSYNC, 1); ++ if (rc != 0) { ++ goto seccomp_return; ++ } ++ + for (i = 0; i < ARRAY_SIZE(blacklist); i++) { + if (!(seccomp_opts & blacklist[i].set)) { + continue; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-seccomp-use-SIGSYS-signal-instead-of-killing-the-thr.patch b/SOURCES/kvm-seccomp-use-SIGSYS-signal-instead-of-killing-the-thr.patch new file mode 100644 index 0000000..ce41067 --- /dev/null +++ b/SOURCES/kvm-seccomp-use-SIGSYS-signal-instead-of-killing-the-thr.patch @@ -0,0 +1,67 @@ +From fb1bf29b909f8443aae0171e3b9c8bc369e0a641 Mon Sep 17 00:00:00 2001 +From: Eduardo Otubo +Date: Fri, 28 Sep 2018 07:56:36 +0100 +Subject: [PATCH 2/6] seccomp: use SIGSYS signal instead of killing the thread +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Eduardo Otubo +Message-id: <20180928075639.16746-3-otubo@redhat.com> +Patchwork-id: 82314 +O-Subject: [RHEL-8 qemu-kvm PATCH 2/5] seccomp: use SIGSYS signal instead of killing the thread +Bugzilla: 1618356 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Thomas Huth + +From: Marc-André Lureau + +commit 6f2231e9b0931e1998d9ed0c509adf7aedc02db2 +Author: Marc-André Lureau +Date: Wed Aug 22 19:02:47 2018 +0200 + + seccomp: use SIGSYS signal instead of killing the thread + + The seccomp action SCMP_ACT_KILL results in immediate termination of + the thread that made the bad system call. However, qemu being + multi-threaded, it keeps running. There is no easy way for parent + process / management layer (libvirt) to know about that situation. + + Instead, the default SIGSYS handler when invoked with SCMP_ACT_TRAP + will terminate the program and core dump. + + This may not be the most secure solution, but probably better than + just killing the offending thread. SCMP_ACT_KILL_PROCESS has been + added in Linux 4.14 to improve the situation, which I propose to use + by default if available in the next patch. + + Related to: + https://bugzilla.redhat.com/show_bug.cgi?id=1594456 + + Signed-off-by: Marc-André Lureau + Reviewed-by: Daniel P. Berrangé + Acked-by: Eduardo Otubo + +Signed-off-by: Eduardo Otubo +Signed-off-by: Danilo C. L. de Paula +--- + qemu-seccomp.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/qemu-seccomp.c b/qemu-seccomp.c +index 845a333..b88fa05 100644 +--- a/qemu-seccomp.c ++++ b/qemu-seccomp.c +@@ -120,7 +120,7 @@ int seccomp_start(uint32_t seccomp_opts) + continue; + } + +- rc = seccomp_rule_add_array(ctx, SCMP_ACT_KILL, blacklist[i].num, ++ rc = seccomp_rule_add_array(ctx, SCMP_ACT_TRAP, blacklist[i].num, + blacklist[i].narg, blacklist[i].arg_cmp); + if (rc < 0) { + goto seccomp_return; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-setup b/SOURCES/kvm-setup new file mode 100644 index 0000000..abbd587 --- /dev/null +++ b/SOURCES/kvm-setup @@ -0,0 +1,40 @@ +#! /bin/bash + +kvm_setup_powerpc () { + if grep '^platform[[:space:]]*:[[:space:]]*PowerNV' /proc/cpuinfo > /dev/null; then + # PowerNV platform, which is KVM HV capable + + if [ -z "$SUBCORES" ]; then + SUBCORES=1 + fi + + # Step 1. Load the KVM HVmodule + if ! modprobe -b kvm_hv; then + return + fi + + # On POWER8 a host core can only run threads of a single + # guest, meaning that SMT must be disabled on the host in + # order to run KVM guests. (Also applieds to POWER7, but we + # don't support that). + # + # POWER9 doesn't have this limitation (though it will for hash + # guests on radix host when that's implemented). So, only set + # up subcores and disable SMT for POWER*. + if grep '^cpu[[:space:]]*:[[:space:]]*POWER8' /proc/cpuinfo > /dev/null; then + # Step 2. Configure subcore mode + /usr/sbin/ppc64_cpu --subcores-per-core=$SUBCORES + + # Step 3. Disable SMT (multithreading) + /usr/sbin/ppc64_cpu --smt=off + fi + fi +} + +case $(uname -m) in + ppc64|ppc64le) + kvm_setup_powerpc + ;; +esac + +exit 0 diff --git a/SOURCES/kvm-setup.service b/SOURCES/kvm-setup.service new file mode 100644 index 0000000..9c4bf97 --- /dev/null +++ b/SOURCES/kvm-setup.service @@ -0,0 +1,14 @@ +[Unit] +Description=Perform system configuration to prepare system to run KVM guests +# Offlining CPUs can cause irqbalance to throw warnings if it's running +Before=irqbalance.service +# libvirtd reads CPU topology at startup, so change it before +Before=libvirtd.service + +[Service] +Type=oneshot +EnvironmentFile=-/etc/sysconfig/kvm +ExecStart=/usr/lib/systemd/kvm-setup + +[Install] +WantedBy=multi-user.target diff --git a/SOURCES/kvm-sheepdog-Fix-sd_co_create_opts-memory-leaks.patch b/SOURCES/kvm-sheepdog-Fix-sd_co_create_opts-memory-leaks.patch new file mode 100644 index 0000000..32e76f1 --- /dev/null +++ b/SOURCES/kvm-sheepdog-Fix-sd_co_create_opts-memory-leaks.patch @@ -0,0 +1,63 @@ +From de0616c738537b21828f572c6c679ad4b290b81e Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:47:44 +0200 +Subject: [PATCH 076/268] sheepdog: Fix sd_co_create_opts() memory leaks + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-2-kwolf@redhat.com> +Patchwork-id: 81074 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 01/73] sheepdog: Fix sd_co_create_opts() memory leaks +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +Both the option string for the 'redundancy' option and the +SheepdogRedundancy object that is created accordingly could be leaked in +error paths. This fixes the memory leaks. + +Reported by Coverity (CID 1390614 and 1390641). + +Signed-off-by: Kevin Wolf +Message-id: 20180503153509.22223-1-kwolf@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Jeff Cody +(cherry picked from commit a2cb9239b7610ffb00f9ced5cd7640d40b0e1ccf) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/sheepdog.c | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +diff --git a/block/sheepdog.c b/block/sheepdog.c +index d1c9bf5..933880c 100644 +--- a/block/sheepdog.c ++++ b/block/sheepdog.c +@@ -1977,6 +1977,7 @@ static SheepdogRedundancy *parse_redundancy_str(const char *opt) + } else { + ret = qemu_strtol(n2, NULL, 10, &parity); + if (ret < 0) { ++ g_free(redundancy); + return NULL; + } + +@@ -2172,7 +2173,7 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + BlockdevCreateOptions *create_options = NULL; + QDict *qdict, *location_qdict; + Visitor *v; +- const char *redundancy; ++ char *redundancy; + Error *local_err = NULL; + int ret; + +@@ -2240,6 +2241,7 @@ static int coroutine_fn sd_co_create_opts(const char *filename, QemuOpts *opts, + fail: + qapi_free_BlockdevCreateOptions(create_options); + qobject_unref(qdict); ++ g_free(redundancy); + return ret; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-simpletrace-Convert-name-from-mapping-record-to-str.patch b/SOURCES/kvm-simpletrace-Convert-name-from-mapping-record-to-str.patch new file mode 100644 index 0000000..74d4157 --- /dev/null +++ b/SOURCES/kvm-simpletrace-Convert-name-from-mapping-record-to-str.patch @@ -0,0 +1,54 @@ +From 94150e55c2a2f43e95e5c14c753aee5900df47f1 Mon Sep 17 00:00:00 2001 +From: Stefan Hajnoczi +Date: Wed, 11 Jul 2018 12:31:04 +0100 +Subject: [PATCH 3/4] simpletrace: Convert name from mapping record to str + +RH-Author: Stefan Hajnoczi +Message-id: <20180711123104.31091-2-stefanha@redhat.com> +Patchwork-id: 81299 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 1/1] simpletrace: Convert name from mapping record to str +Bugzilla: 1594969 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Markus Armbruster + +From: Eduardo Habkost + +The rest of the code assumes that idtoname is a (int -> str) +dictionary, so convert the data accordingly. + +This is necessary to make the script work with Python 3 (where +reads from a binary file return 'bytes' objects, not 'str'). + +Fixes the following error: + + $ python3 ./scripts/simpletrace.py trace-events-all trace-27445 + b'object_class_dynamic_cast_assert' event is logged but is not \ + declared in the trace events file, try using trace-events-all instead. + +Signed-off-by: Eduardo Habkost +Message-id: 20180619194549.15584-1-ehabkost@redhat.com +Signed-off-by: Stefan Hajnoczi +(cherry picked from commit 749c1d8e3e12c44247927d8f72f68ec0c0266f8b) +Signed-off-by: Stefan Hajnoczi +Signed-off-by: Danilo C. L. de Paula +--- + scripts/simpletrace.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/scripts/simpletrace.py b/scripts/simpletrace.py +index d4a50a1..4ad34f9 100755 +--- a/scripts/simpletrace.py ++++ b/scripts/simpletrace.py +@@ -70,7 +70,7 @@ def get_record(edict, idtoname, rechdr, fobj): + def get_mapping(fobj): + (event_id, ) = struct.unpack('=Q', fobj.read(8)) + (len, ) = struct.unpack('=L', fobj.read(4)) +- name = fobj.read(len) ++ name = fobj.read(len).decode() + + return (event_id, name) + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-slirp-check-data-length-while-emulating-ident-functi.patch b/SOURCES/kvm-slirp-check-data-length-while-emulating-ident-functi.patch new file mode 100644 index 0000000..376d08e --- /dev/null +++ b/SOURCES/kvm-slirp-check-data-length-while-emulating-ident-functi.patch @@ -0,0 +1,56 @@ +From c894cd80e9b6c7c317da2dbeacf52a34dc5efdbb Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 22 Feb 2019 08:32:02 +0000 +Subject: [PATCH] slirp: check data length while emulating ident function +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190222083202.20283-1-jasowang@redhat.com> +Patchwork-id: 84699 +O-Subject: [RHEL8/rhel qemu-kvm PATCH] slirp: check data length while emulating ident function +Bugzilla: 1669069 +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Philippe Mathieu-Daudé + +From: Prasad J Pandit + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1669069 +Brew Build: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=20325086 +Test status: Tested by myself +Branch: rhel8/master-2.12.0 + +While emulating identification protocol, tcp_emu() does not check +available space in the 'sc_rcv->sb_data' buffer. It could lead to +heap buffer overflow issue. Add check to avoid it. + +Reported-by: Kira <864786842@qq.com> +Signed-off-by: Prasad J Pandit +Signed-off-by: Samuel Thibault +(cherry picked from commit a7104eda7dab99d0cdbd3595c211864cba415905) +Signed-off-by: Danilo C. L. de Paula +--- + slirp/tcp_subr.c | 5 +++++ + 1 file changed, 5 insertions(+) + +diff --git a/slirp/tcp_subr.c b/slirp/tcp_subr.c +index da0d537..1c7eb28 100644 +--- a/slirp/tcp_subr.c ++++ b/slirp/tcp_subr.c +@@ -638,6 +638,11 @@ tcp_emu(struct socket *so, struct mbuf *m) + socklen_t addrlen = sizeof(struct sockaddr_in); + struct sbuf *so_rcv = &so->so_rcv; + ++ if (m->m_len > so_rcv->sb_datalen ++ - (so_rcv->sb_wptr - so_rcv->sb_data)) { ++ return 1; ++ } ++ + memcpy(so_rcv->sb_wptr, m->m_data, m->m_len); + so_rcv->sb_wptr += m->m_len; + so_rcv->sb_rptr += m->m_len; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-slirp-correct-size-computation-while-concatenating-m.patch b/SOURCES/kvm-slirp-correct-size-computation-while-concatenating-m.patch new file mode 100644 index 0000000..8482099 --- /dev/null +++ b/SOURCES/kvm-slirp-correct-size-computation-while-concatenating-m.patch @@ -0,0 +1,110 @@ +From 6de81cfc7762e4177473e188233988e5cebfd25b Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Mon, 30 Jul 2018 08:06:27 +0200 +Subject: [PATCH 266/268] slirp: correct size computation while concatenating + mbuf + +RH-Author: Xiao Wang +Message-id: <1532937987-25050-1-git-send-email-jasowang@redhat.com> +Patchwork-id: 81544 +O-Subject: [RHEL-7.6/7.5z qemu-kvm-rhev] slirp: correct size computation while concatenating mbuf +Bugzilla: 1586255 +RH-Acked-by: wexu@redhat.com +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Miroslav Rezanina + +From: Prasad J Pandit + +While reassembling incoming fragmented datagrams, 'm_cat' routine +extends the 'mbuf' buffer, if it has insufficient room. It computes +a wrong buffer size, which leads to overwriting adjacent heap buffer +area. Correct this size computation in m_cat. + +Reported-by: ZDI Disclosures +Signed-off-by: Prasad J Pandit +Signed-off-by: Samuel Thibault +(cherry picked from commit 864036e251f54c99d31df124aad7f34f01f5344c) +Signed-off-by: Miroslav Rezanina +--- + slirp/mbuf.c | 11 +++++------ + slirp/mbuf.h | 8 +++----- + 2 files changed, 8 insertions(+), 11 deletions(-) + +diff --git a/slirp/mbuf.c b/slirp/mbuf.c +index 5ff2455..18cbf75 100644 +--- a/slirp/mbuf.c ++++ b/slirp/mbuf.c +@@ -138,7 +138,7 @@ m_cat(struct mbuf *m, struct mbuf *n) + * If there's no room, realloc + */ + if (M_FREEROOM(m) < n->m_len) +- m_inc(m,m->m_size+MINCSIZE); ++ m_inc(m, m->m_len + n->m_len); + + memcpy(m->m_data+m->m_len, n->m_data, n->m_len); + m->m_len += n->m_len; +@@ -147,7 +147,7 @@ m_cat(struct mbuf *m, struct mbuf *n) + } + + +-/* make m size bytes large */ ++/* make m 'size' bytes large from m_data */ + void + m_inc(struct mbuf *m, int size) + { +@@ -158,12 +158,12 @@ m_inc(struct mbuf *m, int size) + + if (m->m_flags & M_EXT) { + datasize = m->m_data - m->m_ext; +- m->m_ext = g_realloc(m->m_ext, size); ++ m->m_ext = g_realloc(m->m_ext, size + datasize); + m->m_data = m->m_ext + datasize; + } else { + char *dat; + datasize = m->m_data - m->m_dat; +- dat = g_malloc(size); ++ dat = g_malloc(size + datasize); + memcpy(dat, m->m_dat, m->m_size); + + m->m_ext = dat; +@@ -171,8 +171,7 @@ m_inc(struct mbuf *m, int size) + m->m_flags |= M_EXT; + } + +- m->m_size = size; +- ++ m->m_size = size + datasize; + } + + +diff --git a/slirp/mbuf.h b/slirp/mbuf.h +index 893601f..33b8448 100644 +--- a/slirp/mbuf.h ++++ b/slirp/mbuf.h +@@ -33,8 +33,6 @@ + #ifndef MBUF_H + #define MBUF_H + +-#define MINCSIZE 4096 /* Amount to increase mbuf if too small */ +- + /* + * Macros for type conversion + * mtod(m,t) - convert mbuf pointer to data pointer of correct type +@@ -72,11 +70,11 @@ struct mbuf { + struct mbuf *m_prevpkt; /* Flags aren't used in the output queue */ + int m_flags; /* Misc flags */ + +- int m_size; /* Size of data */ ++ int m_size; /* Size of mbuf, from m_dat or m_ext */ + struct socket *m_so; + +- caddr_t m_data; /* Location of data */ +- int m_len; /* Amount of data in this mbuf */ ++ caddr_t m_data; /* Current location of data */ ++ int m_len; /* Amount of data in this mbuf, from m_data */ + + Slirp *slirp; + bool resolution_requested; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-slow-train-kvm-clear-out-KVM_ASYNC_PF_DELIVERY_AS_PF.patch b/SOURCES/kvm-slow-train-kvm-clear-out-KVM_ASYNC_PF_DELIVERY_AS_PF.patch new file mode 100644 index 0000000..e1035f1 --- /dev/null +++ b/SOURCES/kvm-slow-train-kvm-clear-out-KVM_ASYNC_PF_DELIVERY_AS_PF.patch @@ -0,0 +1,97 @@ +From 68d5cd786a054bc1e84b1ee6a0deeefbfe2c33b9 Mon Sep 17 00:00:00 2001 +From: Bandan Das +Date: Fri, 14 Dec 2018 19:35:46 +0000 +Subject: [PATCH 5/5] slow train kvm: clear out + KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT for older machine types + +RH-Author: Bandan Das +Message-id: +Patchwork-id: 83524 +O-Subject: [RHEL8 qemu-kvm PATCH] slow train kvm: clear out KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT for older machine types +Bugzilla: 1656829 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Paolo Bonzini + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1656829 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=19521977 +Upstream: Not applicable +Branch: rhel8/master-2.12.0 + +After the addition of support for async pf injection to L1, newer +hypervisors advertise the feature using bit 2 of the +MSR_KVM_ASYNC_PF_EN msr. However, this was reserved in older +hypervisors which results in an error during migration like so: + +qemu-kvm: error: failed to set MSR 0x4b564d02 to 0x27fc13285 +qemu-kvm: /builddir/build/BUILD/qemu-2.12.0/target/i386/kvm.c:1940: kvm_put_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed. +Aborted (core dumped) + +This patch introduces a new bool that is set for older machine types. +When set, Qemu's stored value clears out bit 2. This should be safe +because the guest can still enable it by writing to the MSR after +checking for support. A reset/migration for <7.6 machine type would +reset the bit though. + +Signed-off-by: Bandan Das +Signed-off-by: Danilo C. L. de Paula +--- + hw/i386/pc.c | 5 +++++ + include/hw/boards.h | 2 ++ + target/i386/kvm.c | 4 ++++ + 3 files changed, 11 insertions(+) + +diff --git a/hw/i386/pc.c b/hw/i386/pc.c +index 9034f02..9e1e6ae 100644 +--- a/hw/i386/pc.c ++++ b/hw/i386/pc.c +@@ -2363,6 +2363,11 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) + pcmc->save_tsc_khz = true; + pcmc->linuxboot_dma_enabled = true; + pcmc->pc_rom_ro = true; ++ /* ++ * This might have to be set to false ++ * when and if there's a new machine type ++ */ ++ mc->async_pf_vmexit_disable = true; + mc->get_hotplug_handler = pc_get_hotpug_handler; + mc->cpu_index_to_instance_props = pc_cpu_index_to_props; + mc->get_default_cpu_node_id = pc_get_default_cpu_node_id; +diff --git a/include/hw/boards.h b/include/hw/boards.h +index a609239..9b4a69b 100644 +--- a/include/hw/boards.h ++++ b/include/hw/boards.h +@@ -203,6 +203,8 @@ struct MachineClass { + const char **valid_cpu_types; + strList *allowed_dynamic_sysbus_devices; + bool auto_enable_numa_with_memhp; ++ /* RHEL only */ ++ bool async_pf_vmexit_disable; + void (*numa_auto_assign_ram)(MachineClass *mc, NodeInfo *nodes, + int nb_nodes, ram_addr_t size); + +diff --git a/target/i386/kvm.c b/target/i386/kvm.c +index 00f2141..87e4771 100644 +--- a/target/i386/kvm.c ++++ b/target/i386/kvm.c +@@ -2072,6 +2072,7 @@ static int kvm_get_msrs(X86CPU *cpu) + struct kvm_msr_entry *msrs = cpu->kvm_msr_buf->entries; + int ret, i; + uint64_t mtrr_top_bits; ++ MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine()); + + kvm_msr_buf_reset(cpu); + +@@ -2364,6 +2365,9 @@ static int kvm_get_msrs(X86CPU *cpu) + break; + case MSR_KVM_ASYNC_PF_EN: + env->async_pf_en_msr = msrs[i].data; ++ if (mc->async_pf_vmexit_disable) { ++ env->async_pf_en_msr &= ~(1ULL << 2); ++ } + break; + case MSR_KVM_PV_EOI_EN: + env->pv_eoi_en_msr = msrs[i].data; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-spapr-Add-ibm-max-associativity-domains-property.patch b/SOURCES/kvm-spapr-Add-ibm-max-associativity-domains-property.patch new file mode 100644 index 0000000..0cc4164 --- /dev/null +++ b/SOURCES/kvm-spapr-Add-ibm-max-associativity-domains-property.patch @@ -0,0 +1,63 @@ +From 7dbee79b31930ae0eab366f807ea2b6ff506fc1a Mon Sep 17 00:00:00 2001 +From: Serhii Popovych +Date: Wed, 11 Jul 2018 17:11:44 +0100 +Subject: [PATCH 1/4] spapr: Add ibm, max-associativity-domains property + +RH-Author: Serhii Popovych +Message-id: <1531329105-80927-2-git-send-email-spopovyc@redhat.com> +Patchwork-id: 81311 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 1/2] spapr: Add ibm, max-associativity-domains property +Bugzilla: 1599593 +RH-Acked-by: Laurent Vivier +RH-Acked-by: David Gibson +RH-Acked-by: Thomas Huth + +Now recent kernels (i.e. since linux-stable commit a346137e9142 +("powerpc/numa: Use ibm,max-associativity-domains to discover possible nodes") +support this property to mark initially memory-less NUMA nodes as "possible" +to allow further memory hot-add to them. + +Advertise this property for pSeries machines to let guest kernels detect +maximum supported node configuration and benefit from kernel side change +when hot-add memory to specific, possibly empty before, NUMA node. + +Signed-off-by: Serhii Popovych +Signed-off-by: David Gibson +(cherry picked from commit da9f80fbad21319194e73355dea8a1cff6a574e4) +Signed-off-by: Serhii Popovych +Signed-off-by: Danilo C. L. de Paula +--- + hw/ppc/spapr.c | 10 ++++++++++ + 1 file changed, 10 insertions(+) + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index c3f08b3..f3da93f 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -910,6 +910,13 @@ static void spapr_dt_rtas(sPAPRMachineState *spapr, void *fdt) + 0, cpu_to_be32(SPAPR_MEMORY_BLOCK_SIZE), + cpu_to_be32(max_cpus / smp_threads), + }; ++ uint32_t maxdomains[] = { ++ cpu_to_be32(4), ++ cpu_to_be32(0), ++ cpu_to_be32(0), ++ cpu_to_be32(0), ++ cpu_to_be32(nb_numa_nodes ? nb_numa_nodes - 1 : 0), ++ }; + + _FDT(rtas = fdt_add_subnode(fdt, 0, "rtas")); + +@@ -946,6 +953,9 @@ static void spapr_dt_rtas(sPAPRMachineState *spapr, void *fdt) + _FDT(fdt_setprop(fdt, rtas, "ibm,associativity-reference-points", + refpoints, sizeof(refpoints))); + ++ _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains", ++ maxdomains, sizeof(maxdomains))); ++ + _FDT(fdt_setprop_cell(fdt, rtas, "rtas-error-log-max", + RTAS_ERROR_LOG_MAX)); + _FDT(fdt_setprop_cell(fdt, rtas, "rtas-event-scan-rate", +-- +1.8.3.1 + diff --git a/SOURCES/kvm-spapr-Correct-inverted-test-in-spapr_pc_dimm_node.patch b/SOURCES/kvm-spapr-Correct-inverted-test-in-spapr_pc_dimm_node.patch new file mode 100644 index 0000000..1384670 --- /dev/null +++ b/SOURCES/kvm-spapr-Correct-inverted-test-in-spapr_pc_dimm_node.patch @@ -0,0 +1,55 @@ +From c279e652b149621847ef38f08254569632f7ee3b Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Wed, 25 Jul 2018 07:23:41 +0100 +Subject: [PATCH 01/14] spapr: Correct inverted test in spapr_pc_dimm_node() + +RH-Author: David Gibson +Message-id: <20180725072341.8452-1-dgibson@redhat.com> +Patchwork-id: 81498 +O-Subject: [RHEL-8.0 qemu-kvm PATCH] spapr: Correct inverted test in spapr_pc_dimm_node() +Bugzilla: 1601671 +RH-Acked-by: Thomas Huth +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych + +From: David Gibson + +This function was introduced between v2.11 and v2.12 to replace obsolete +ways of specifying the NUMA nodes for DIMMs. It's used to find the correct +node for an LMB, by locating which DIMM object it lies within. + +Unfortunately, one of the checks is inverted, so we check whether the +address is less than two different things, rather than actually checking +a range. This introduced a regression, meaning that after a reboot qemu +will advertise incorrect node information for memory to the guest. + +Signed-off-by: David Gibson +Reviewed-by: Greg Kurz +Reviewed-by: Igor Mammedov +(cherry picked from commit ccc2cef8b3f1dedd059924eb8ec1a87eff8ef607) + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1601671 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=17345816 + +Signed-off-by: David Gibson +Signed-off-by: Danilo C. L. de Paula +--- + hw/ppc/spapr.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c +index ef00937..7de3f07 100644 +--- a/hw/ppc/spapr.c ++++ b/hw/ppc/spapr.c +@@ -659,7 +659,7 @@ static uint32_t spapr_pc_dimm_node(MemoryDeviceInfoList *list, ram_addr_t addr) + if (value && value->type == MEMORY_DEVICE_INFO_KIND_DIMM) { + PCDIMMDeviceInfo *pcdimm_info = value->u.dimm.data; + +- if (pcdimm_info->addr >= addr && ++ if (addr >= pcdimm_info->addr && + addr < (pcdimm_info->addr + pcdimm_info->size)) { + return pcdimm_info->node; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-spapr_pci-Remove-unhelpful-pagesize-warning.patch b/SOURCES/kvm-spapr_pci-Remove-unhelpful-pagesize-warning.patch new file mode 100644 index 0000000..b0d465f --- /dev/null +++ b/SOURCES/kvm-spapr_pci-Remove-unhelpful-pagesize-warning.patch @@ -0,0 +1,61 @@ +From 7cce70900ecb644004144c999ee082ff79c0c25e Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Thu, 14 Jun 2018 01:31:36 +0200 +Subject: [PATCH 008/268] spapr_pci: Remove unhelpful pagesize warning + +RH-Author: David Gibson +Message-id: <20180614013136.3504-1-dgibson@redhat.com> +Patchwork-id: 80681 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH] spapr_pci: Remove unhelpful pagesize warning +Bugzilla: 1505664 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Thomas Huth +RH-Acked-by: Serhii Popovych + +From: David Gibson + +By default, the IOMMU model built into the spapr virtual PCI host bridge +supports 4kiB and 64kiB IOMMU page sizes. However this can be overridden +which may be desirable to allow larger IOMMU page sizes when running a +guest with hugepage backing and passthrough devices. For that reason a +warning was printed when the device wasn't configured to allow the pagesize +with which guest RAM is backed. + +Experience has proven, however, that this message is more confusing than +useful. Worse it sometimes makes little sense when the host-available page +sizes don't match those available on the guest, which can happen with +a POWER8 guest running on a POWER9 KVM host. + +Long term we do want better handling to allow large IOMMU page sizes to be +used, but for now this parameter and warning don't really accomplish it. +So, remove the message, pending a better solution. + +Signed-off-by: David Gibson + +Signed-off-by: David Gibson +Signed-off-by: Miroslav Rezanina +--- + hw/ppc/spapr_pci.c | 7 ------- + 1 file changed, 7 deletions(-) + +diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c +index 39a1498..f936ce6 100644 +--- a/hw/ppc/spapr_pci.c ++++ b/hw/ppc/spapr_pci.c +@@ -1717,13 +1717,6 @@ static void spapr_phb_realize(DeviceState *dev, Error **errp) + } + + /* DMA setup */ +- if (((sphb->page_size_mask & qemu_getrampagesize()) == 0) +- && kvm_enabled()) { +- warn_report("System page size 0x%lx is not enabled in page_size_mask " +- "(0x%"PRIx64"). Performance may be slow", +- qemu_getrampagesize(), sphb->page_size_mask); +- } +- + for (i = 0; i < windows_supported; ++i) { + tcet = spapr_tce_new_table(DEVICE(sphb), sphb->dma_liobn[i]); + if (!tcet) { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-target-i386-sev-fix-memory-leaks.patch b/SOURCES/kvm-target-i386-sev-fix-memory-leaks.patch new file mode 100644 index 0000000..344ff90 --- /dev/null +++ b/SOURCES/kvm-target-i386-sev-fix-memory-leaks.patch @@ -0,0 +1,148 @@ +From 4bb6d68815ce5ab1ebd5030b94031e0621806822 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Fri, 31 Aug 2018 13:59:21 +0100 +Subject: [PATCH 1/3] target/i386: sev: fix memory leaks + +RH-Author: Markus Armbruster +Message-id: <20180831135922.6073-2-armbru@redhat.com> +Patchwork-id: 81981 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 1/2] target/i386: sev: fix memory leaks +Bugzilla: 1615717 +RH-Acked-by: Miroslav Rezanina +RH-Acked-by: Eduardo Habkost +RH-Acked-by: Laszlo Ersek + +From: Paolo Bonzini + +Reported by Coverity. + +Signed-off-by: Paolo Bonzini +(cherry picked from commit bf3175b49952628f96d72d1247d8bb3aa5c2466c) +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/sev.c | 32 +++++++++++++++++--------------- + 1 file changed, 17 insertions(+), 15 deletions(-) + +diff --git a/target/i386/sev.c b/target/i386/sev.c +index c011671..2395171 100644 +--- a/target/i386/sev.c ++++ b/target/i386/sev.c +@@ -430,7 +430,8 @@ static int + sev_get_pdh_info(int fd, guchar **pdh, size_t *pdh_len, guchar **cert_chain, + size_t *cert_chain_len) + { +- guchar *pdh_data, *cert_chain_data; ++ guchar *pdh_data = NULL; ++ guchar *cert_chain_data = NULL; + struct sev_user_data_pdh_cert_export export = {}; + int err, r; + +@@ -471,8 +472,9 @@ e_free: + SevCapability * + sev_get_capabilities(void) + { +- SevCapability *cap; +- guchar *pdh_data, *cert_chain_data; ++ SevCapability *cap = NULL; ++ guchar *pdh_data = NULL; ++ guchar *cert_chain_data = NULL; + size_t pdh_len = 0, cert_chain_len = 0; + uint32_t ebx; + int fd; +@@ -486,7 +488,7 @@ sev_get_capabilities(void) + + if (sev_get_pdh_info(fd, &pdh_data, &pdh_len, + &cert_chain_data, &cert_chain_len)) { +- return NULL; ++ goto out; + } + + cap = g_new0(SevCapability, 1); +@@ -502,9 +504,9 @@ sev_get_capabilities(void) + */ + cap->reduced_phys_bits = 1; + ++out: + g_free(pdh_data); + g_free(cert_chain_data); +- + close(fd); + return cap; + } +@@ -530,7 +532,7 @@ sev_launch_start(SEVState *s) + { + gsize sz; + int ret = 1; +- int fw_error; ++ int fw_error, rc; + QSevGuestInfo *sev = s->sev_info; + struct kvm_sev_launch_start *start; + guchar *session = NULL, *dh_cert = NULL; +@@ -543,7 +545,7 @@ sev_launch_start(SEVState *s) + &error_abort); + if (sev->session_file) { + if (sev_read_file_base64(sev->session_file, &session, &sz) < 0) { +- return 1; ++ goto out; + } + start->session_uaddr = (unsigned long)session; + start->session_len = sz; +@@ -551,18 +553,18 @@ sev_launch_start(SEVState *s) + + if (sev->dh_cert_file) { + if (sev_read_file_base64(sev->dh_cert_file, &dh_cert, &sz) < 0) { +- return 1; ++ goto out; + } + start->dh_uaddr = (unsigned long)dh_cert; + start->dh_len = sz; + } + + trace_kvm_sev_launch_start(start->policy, session, dh_cert); +- ret = sev_ioctl(s->sev_fd, KVM_SEV_LAUNCH_START, start, &fw_error); +- if (ret < 0) { ++ rc = sev_ioctl(s->sev_fd, KVM_SEV_LAUNCH_START, start, &fw_error); ++ if (rc < 0) { + error_report("%s: LAUNCH_START ret=%d fw_error=%d '%s'", + __func__, ret, fw_error, fw_error_to_str(fw_error)); +- return 1; ++ goto out; + } + + object_property_set_int(OBJECT(sev), start->handle, "handle", +@@ -570,12 +572,13 @@ sev_launch_start(SEVState *s) + sev_set_guest_state(SEV_STATE_LAUNCH_UPDATE); + s->handle = start->handle; + s->policy = start->policy; ++ ret = 0; + ++out: + g_free(start); + g_free(session); + g_free(dh_cert); +- +- return 0; ++ return ret; + } + + static int +@@ -712,7 +715,7 @@ sev_guest_init(const char *id) + uint32_t host_cbitpos; + struct sev_user_data_status status = {}; + +- s = g_new0(SEVState, 1); ++ sev_state = s = g_new0(SEVState, 1); + s->sev_info = lookup_sev_guest_info(id); + if (!s->sev_info) { + error_report("%s: '%s' is not a valid '%s' object", +@@ -720,7 +723,6 @@ sev_guest_init(const char *id) + goto err; + } + +- sev_state = s; + s->state = SEV_STATE_UNINIT; + + host_cpuid(0x8000001F, 0, NULL, &ebx, NULL, NULL); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-target-ppc-Add-one-reg-id-for-ptcr.patch b/SOURCES/kvm-target-ppc-Add-one-reg-id-for-ptcr.patch new file mode 100644 index 0000000..916cd26 --- /dev/null +++ b/SOURCES/kvm-target-ppc-Add-one-reg-id-for-ptcr.patch @@ -0,0 +1,66 @@ +From 246d822cd39a0c9ee0cac082e64a20ea6d49b1c9 Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Mon, 12 Nov 2018 01:28:34 +0000 +Subject: [PATCH 03/16] target/ppc: Add one reg id for ptcr + +RH-Author: David Gibson +Message-id: <20181112012835.21863-4-dgibson@redhat.com> +Patchwork-id: 82979 +O-Subject: [RHEL-8 qemu-kvm PATCH 3/4] target/ppc: Add one reg id for ptcr +Bugzilla: 1639069 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych +RH-Acked-by: Thomas Huth + +From: Suraj Jitindar Singh + +The ptcr (partition table control register) is used to store the address +and size of the partition table. For nested kvm-hv we have a level 1 +guest register the location of it's partition table with the hypervisor. +Thus to support migration we need to be able to read this out of kvm +and restore it post migration. + +Add the one reg id for the ptcr. + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: David Gibson +(cherry picked from commit 56de52cad954a94530953bf979007db84c5f4dbb) +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + target/ppc/translate_init.inc.c + +Adjusting for an upstream rename / code rearrangement that we don't +have downstream. + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1639069 + +Signed-off-by: David Gibson +--- + target/ppc/translate_init.c | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +diff --git a/target/ppc/translate_init.c b/target/ppc/translate_init.c +index 926efbc..db9df8a 100644 +--- a/target/ppc/translate_init.c ++++ b/target/ppc/translate_init.c +@@ -8176,11 +8176,11 @@ static void gen_spr_power9_mmu(CPUPPCState *env) + { + #if !defined(CONFIG_USER_ONLY) + /* Partition Table Control */ +- spr_register_hv(env, SPR_PTCR, "PTCR", +- SPR_NOACCESS, SPR_NOACCESS, +- SPR_NOACCESS, SPR_NOACCESS, +- &spr_read_generic, &spr_write_ptcr, +- 0x00000000); ++ spr_register_kvm_hv(env, SPR_PTCR, "PTCR", ++ SPR_NOACCESS, SPR_NOACCESS, ++ SPR_NOACCESS, SPR_NOACCESS, ++ &spr_read_generic, &spr_write_ptcr, ++ KVM_REG_PPC_PTCR, 0x00000000); + #endif + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-target-ppc-Don-t-require-private-l1d-cache-on-POWER8.patch b/SOURCES/kvm-target-ppc-Don-t-require-private-l1d-cache-on-POWER8.patch new file mode 100644 index 0000000..8874577 --- /dev/null +++ b/SOURCES/kvm-target-ppc-Don-t-require-private-l1d-cache-on-POWER8.patch @@ -0,0 +1,72 @@ +From ad1adce48f771ecc03b75575edbe09bca569167a Mon Sep 17 00:00:00 2001 +From: Suraj Jitindar Singh +Date: Thu, 21 Jun 2018 06:56:48 +0200 +Subject: [PATCH 061/268] target/ppc: Don't require private l1d cache on POWER8 + for cap_ppc_safe_cache + +RH-Author: Suraj Jitindar Singh +Message-id: <1529564209-30369-3-git-send-email-sursingh@redhat.com> +Patchwork-id: 80930 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/3] target/ppc: Don't require private l1d cache on POWER8 for cap_ppc_safe_cache +Bugzilla: 1560847 +RH-Acked-by: Laurent Vivier +RH-Acked-by: David Gibson +RH-Acked-by: Miroslav Rezanina + +From: Suraj Jitindar Singh + +For cap_ppc_safe_cache to be set to workaround, we require both a l1d +cache flush instruction and private l1d cache. + +On POWER8 don't require private l1d cache. This means a guest on a +POWER8 machine can make use of the cache flush workarounds. + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: David Gibson +(cherry picked from commit 072f416a53ead5211c987cb2068ee9dbd7ba06cc) + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1560847 + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: Miroslav Rezanina +--- + target/ppc/kvm.c | 19 ++++++++++++++++++- + 1 file changed, 18 insertions(+), 1 deletion(-) + +diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c +index d787032..192c40d 100644 +--- a/target/ppc/kvm.c ++++ b/target/ppc/kvm.c +@@ -2461,11 +2461,28 @@ bool kvmppc_has_cap_mmu_hash_v3(void) + return cap_mmu_hash_v3; + } + ++static bool kvmppc_power8_host(void) ++{ ++ bool ret = false; ++#ifdef TARGET_PPC64 ++ { ++ uint32_t base_pvr = CPU_POWERPC_POWER_SERVER_MASK & mfpvr(); ++ ret = (base_pvr == CPU_POWERPC_POWER8E_BASE) || ++ (base_pvr == CPU_POWERPC_POWER8NVL_BASE) || ++ (base_pvr == CPU_POWERPC_POWER8_BASE); ++ } ++#endif /* TARGET_PPC64 */ ++ return ret; ++} ++ + static int parse_cap_ppc_safe_cache(struct kvm_ppc_cpu_char c) + { ++ bool l1d_thread_priv_req = !kvmppc_power8_host(); ++ + if (~c.behaviour & c.behaviour_mask & H_CPU_BEHAV_L1D_FLUSH_PR) { + return 2; +- } else if ((c.character & c.character_mask & H_CPU_CHAR_L1D_THREAD_PRIV) && ++ } else if ((!l1d_thread_priv_req || ++ c.character & c.character_mask & H_CPU_CHAR_L1D_THREAD_PRIV) && + (c.character & c.character_mask + & (H_CPU_CHAR_L1D_FLUSH_ORI30 | H_CPU_CHAR_L1D_FLUSH_TRIG2))) { + return 1; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-target-ppc-Factor-out-the-parsing-in-kvmppc_get_cpu_.patch b/SOURCES/kvm-target-ppc-Factor-out-the-parsing-in-kvmppc_get_cpu_.patch new file mode 100644 index 0000000..a8f6b92 --- /dev/null +++ b/SOURCES/kvm-target-ppc-Factor-out-the-parsing-in-kvmppc_get_cpu_.patch @@ -0,0 +1,113 @@ +From e4af250e57428a1a29b8f18c4ceaeb6be3d1eff6 Mon Sep 17 00:00:00 2001 +From: Suraj Jitindar Singh +Date: Thu, 21 Jun 2018 06:56:47 +0200 +Subject: [PATCH 060/268] target/ppc: Factor out the parsing in + kvmppc_get_cpu_characteristics() + +RH-Author: Suraj Jitindar Singh +Message-id: <1529564209-30369-2-git-send-email-sursingh@redhat.com> +Patchwork-id: 80927 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/3] target/ppc: Factor out the parsing in kvmppc_get_cpu_characteristics() +Bugzilla: 1560847 +RH-Acked-by: Laurent Vivier +RH-Acked-by: David Gibson +RH-Acked-by: Miroslav Rezanina + +From: Suraj Jitindar Singh + +Factor out the parsing of struct kvm_ppc_cpu_char in +kvmppc_get_cpu_characteristics() into a separate function for each cap +for simplicity. + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: David Gibson +(cherry picked from commit 8fea70440eb0d095442de7e80d586a285cf96be5) + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1560847 + +Signed-off-by: Suraj Jitindar Singh +Signed-off-by: Miroslav Rezanina +--- + target/ppc/kvm.c | 59 +++++++++++++++++++++++++++++++++++++------------------- + 1 file changed, 39 insertions(+), 20 deletions(-) + +diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c +index 79a436a..d787032 100644 +--- a/target/ppc/kvm.c ++++ b/target/ppc/kvm.c +@@ -2461,6 +2461,41 @@ bool kvmppc_has_cap_mmu_hash_v3(void) + return cap_mmu_hash_v3; + } + ++static int parse_cap_ppc_safe_cache(struct kvm_ppc_cpu_char c) ++{ ++ if (~c.behaviour & c.behaviour_mask & H_CPU_BEHAV_L1D_FLUSH_PR) { ++ return 2; ++ } else if ((c.character & c.character_mask & H_CPU_CHAR_L1D_THREAD_PRIV) && ++ (c.character & c.character_mask ++ & (H_CPU_CHAR_L1D_FLUSH_ORI30 | H_CPU_CHAR_L1D_FLUSH_TRIG2))) { ++ return 1; ++ } ++ ++ return 0; ++} ++ ++static int parse_cap_ppc_safe_bounds_check(struct kvm_ppc_cpu_char c) ++{ ++ if (~c.behaviour & c.behaviour_mask & H_CPU_BEHAV_BNDS_CHK_SPEC_BAR) { ++ return 2; ++ } else if (c.character & c.character_mask & H_CPU_CHAR_SPEC_BAR_ORI31) { ++ return 1; ++ } ++ ++ return 0; ++} ++ ++static int parse_cap_ppc_safe_indirect_branch(struct kvm_ppc_cpu_char c) ++{ ++ if (c.character & c.character_mask & H_CPU_CHAR_CACHE_COUNT_DIS) { ++ return SPAPR_CAP_FIXED_CCD; ++ } else if (c.character & c.character_mask & H_CPU_CHAR_BCCTRL_SERIALISED) { ++ return SPAPR_CAP_FIXED_IBS; ++ } ++ ++ return 0; ++} ++ + static void kvmppc_get_cpu_characteristics(KVMState *s) + { + struct kvm_ppc_cpu_char c; +@@ -2479,26 +2514,10 @@ static void kvmppc_get_cpu_characteristics(KVMState *s) + if (ret < 0) { + return; + } +- /* Parse and set cap_ppc_safe_cache */ +- if (~c.behaviour & c.behaviour_mask & H_CPU_BEHAV_L1D_FLUSH_PR) { +- cap_ppc_safe_cache = 2; +- } else if ((c.character & c.character_mask & H_CPU_CHAR_L1D_THREAD_PRIV) && +- (c.character & c.character_mask +- & (H_CPU_CHAR_L1D_FLUSH_ORI30 | H_CPU_CHAR_L1D_FLUSH_TRIG2))) { +- cap_ppc_safe_cache = 1; +- } +- /* Parse and set cap_ppc_safe_bounds_check */ +- if (~c.behaviour & c.behaviour_mask & H_CPU_BEHAV_BNDS_CHK_SPEC_BAR) { +- cap_ppc_safe_bounds_check = 2; +- } else if (c.character & c.character_mask & H_CPU_CHAR_SPEC_BAR_ORI31) { +- cap_ppc_safe_bounds_check = 1; +- } +- /* Parse and set cap_ppc_safe_indirect_branch */ +- if (c.character & c.character_mask & H_CPU_CHAR_CACHE_COUNT_DIS) { +- cap_ppc_safe_indirect_branch = SPAPR_CAP_FIXED_CCD; +- } else if (c.character & c.character_mask & H_CPU_CHAR_BCCTRL_SERIALISED) { +- cap_ppc_safe_indirect_branch = SPAPR_CAP_FIXED_IBS; +- } ++ ++ cap_ppc_safe_cache = parse_cap_ppc_safe_cache(c); ++ cap_ppc_safe_bounds_check = parse_cap_ppc_safe_bounds_check(c); ++ cap_ppc_safe_indirect_branch = parse_cap_ppc_safe_indirect_branch(c); + } + + int kvmppc_get_cap_safe_cache(void) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-target-ppc-add-basic-support-for-PTCR-on-POWER9.patch b/SOURCES/kvm-target-ppc-add-basic-support-for-PTCR-on-POWER9.patch new file mode 100644 index 0000000..69544c4 --- /dev/null +++ b/SOURCES/kvm-target-ppc-add-basic-support-for-PTCR-on-POWER9.patch @@ -0,0 +1,214 @@ +From 3b97963ddb435e25f758032691cb2315570a2093 Mon Sep 17 00:00:00 2001 +From: David Gibson +Date: Mon, 12 Nov 2018 01:28:32 +0000 +Subject: [PATCH 01/16] target/ppc: add basic support for PTCR on POWER9 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: David Gibson +Message-id: <20181112012835.21863-2-dgibson@redhat.com> +Patchwork-id: 82978 +O-Subject: [RHEL-8 qemu-kvm PATCH 1/4] target/ppc: add basic support for PTCR on POWER9 +Bugzilla: 1639069 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Serhii Popovych +RH-Acked-by: Thomas Huth + +From: Cédric Le Goater + +The Partition Table Control Register (PTCR) is a hypervisor privileged +SPR. It contains the host real address of the Partition Table and its +size. + +Signed-off-by: Cédric Le Goater +Reviewed-by: David Gibson +Signed-off-by: David Gibson +(cherry picked from commit 4a7518e0fdaa20525730ae0709a4afa0960a6c67) + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1639069 + +Signed-off-by: David Gibson +Signed-off-by: Danilo C. L. de Paula +--- + target/ppc/cpu.h | 2 ++ + target/ppc/helper.h | 1 + + target/ppc/misc_helper.c | 12 ++++++++++++ + target/ppc/mmu-book3s-v3.h | 6 ++++++ + target/ppc/mmu_helper.c | 29 +++++++++++++++++++++++++++++ + target/ppc/translate.c | 3 +++ + target/ppc/translate_init.c | 18 ++++++++++++++++++ + 7 files changed, 71 insertions(+) + +diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h +index 1932c2e..8f3cf44 100644 +--- a/target/ppc/cpu.h ++++ b/target/ppc/cpu.h +@@ -1313,6 +1313,7 @@ int ppc_cpu_handle_mmu_fault(CPUState *cpu, vaddr address, int size, int rw, + + #if !defined(CONFIG_USER_ONLY) + void ppc_store_sdr1 (CPUPPCState *env, target_ulong value); ++void ppc_store_ptcr(CPUPPCState *env, target_ulong value); + #endif /* !defined(CONFIG_USER_ONLY) */ + void ppc_store_msr (CPUPPCState *env, target_ulong value); + +@@ -1604,6 +1605,7 @@ void ppc_compat_add_property(Object *obj, const char *name, + #define SPR_BOOKE_GIVOR13 (0x1BC) + #define SPR_BOOKE_GIVOR14 (0x1BD) + #define SPR_TIR (0x1BE) ++#define SPR_PTCR (0x1D0) + #define SPR_BOOKE_SPEFSCR (0x200) + #define SPR_Exxx_BBEAR (0x201) + #define SPR_Exxx_BBTAR (0x202) +diff --git a/target/ppc/helper.h b/target/ppc/helper.h +index 5b73917..19453c6 100644 +--- a/target/ppc/helper.h ++++ b/target/ppc/helper.h +@@ -709,6 +709,7 @@ DEF_HELPER_FLAGS_1(load_601_rtcu, TCG_CALL_NO_RWG, tl, env) + #if !defined(CONFIG_USER_ONLY) + #if defined(TARGET_PPC64) + DEF_HELPER_FLAGS_1(load_purr, TCG_CALL_NO_RWG, tl, env) ++DEF_HELPER_2(store_ptcr, void, env, tl) + #endif + DEF_HELPER_2(store_sdr1, void, env, tl) + DEF_HELPER_2(store_pidr, void, env, tl) +diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c +index 0e42178..8c8cba5 100644 +--- a/target/ppc/misc_helper.c ++++ b/target/ppc/misc_helper.c +@@ -88,6 +88,18 @@ void helper_store_sdr1(CPUPPCState *env, target_ulong val) + } + } + ++#if defined(TARGET_PPC64) ++void helper_store_ptcr(CPUPPCState *env, target_ulong val) ++{ ++ PowerPCCPU *cpu = ppc_env_get_cpu(env); ++ ++ if (env->spr[SPR_PTCR] != val) { ++ ppc_store_ptcr(env, val); ++ tlb_flush(CPU(cpu)); ++ } ++} ++#endif /* defined(TARGET_PPC64) */ ++ + void helper_store_pidr(CPUPPCState *env, target_ulong val) + { + PowerPCCPU *cpu = ppc_env_get_cpu(env); +diff --git a/target/ppc/mmu-book3s-v3.h b/target/ppc/mmu-book3s-v3.h +index 56095da..fdf8098 100644 +--- a/target/ppc/mmu-book3s-v3.h ++++ b/target/ppc/mmu-book3s-v3.h +@@ -22,6 +22,12 @@ + + #ifndef CONFIG_USER_ONLY + ++/* ++ * Partition table definitions ++ */ ++#define PTCR_PATB 0x0FFFFFFFFFFFF000ULL /* Partition Table Base */ ++#define PTCR_PATS 0x000000000000001FULL /* Partition Table Size */ ++ + /* Partition Table Entry Fields */ + #define PATBE1_GR 0x8000000000000000 + +diff --git a/target/ppc/mmu_helper.c b/target/ppc/mmu_helper.c +index 5568d16..e2197a5 100644 +--- a/target/ppc/mmu_helper.c ++++ b/target/ppc/mmu_helper.c +@@ -2028,6 +2028,35 @@ void ppc_store_sdr1(CPUPPCState *env, target_ulong value) + env->spr[SPR_SDR1] = value; + } + ++#if defined(TARGET_PPC64) ++void ppc_store_ptcr(CPUPPCState *env, target_ulong value) ++{ ++ PowerPCCPU *cpu = ppc_env_get_cpu(env); ++ target_ulong ptcr_mask = PTCR_PATB | PTCR_PATS; ++ target_ulong patbsize = value & PTCR_PATS; ++ ++ qemu_log_mask(CPU_LOG_MMU, "%s: " TARGET_FMT_lx "\n", __func__, value); ++ ++ assert(!cpu->vhyp); ++ assert(env->mmu_model & POWERPC_MMU_3_00); ++ ++ if (value & ~ptcr_mask) { ++ error_report("Invalid bits 0x"TARGET_FMT_lx" set in PTCR", ++ value & ~ptcr_mask); ++ value &= ptcr_mask; ++ } ++ ++ if (patbsize > 24) { ++ error_report("Invalid Partition Table size 0x" TARGET_FMT_lx ++ " stored in PTCR", patbsize); ++ return; ++ } ++ ++ env->spr[SPR_PTCR] = value; ++} ++ ++#endif /* defined(TARGET_PPC64) */ ++ + /* Segment registers load and store */ + target_ulong helper_load_sr(CPUPPCState *env, target_ulong sr_num) + { +diff --git a/target/ppc/translate.c b/target/ppc/translate.c +index 3457d29..7da9b67 100644 +--- a/target/ppc/translate.c ++++ b/target/ppc/translate.c +@@ -7136,6 +7136,9 @@ void ppc_cpu_dump_state(CPUState *cs, FILE *f, fprintf_function cpu_fprintf, + if (env->spr_cb[SPR_SDR1].name) { /* SDR1 Exists */ + cpu_fprintf(f, " SDR1 " TARGET_FMT_lx " ", env->spr[SPR_SDR1]); + } ++ if (env->spr_cb[SPR_PTCR].name) { /* PTCR Exists */ ++ cpu_fprintf(f, " PTCR " TARGET_FMT_lx " ", env->spr[SPR_PTCR]); ++ } + cpu_fprintf(f, " DAR " TARGET_FMT_lx " DSISR " TARGET_FMT_lx "\n", + env->spr[SPR_DAR], env->spr[SPR_DSISR]); + break; +diff --git a/target/ppc/translate_init.c b/target/ppc/translate_init.c +index 17b06c7..926efbc 100644 +--- a/target/ppc/translate_init.c ++++ b/target/ppc/translate_init.c +@@ -420,6 +420,11 @@ static void spr_write_hior(DisasContext *ctx, int sprn, int gprn) + tcg_gen_st_tl(t0, cpu_env, offsetof(CPUPPCState, excp_prefix)); + tcg_temp_free(t0); + } ++static void spr_write_ptcr(DisasContext *ctx, int sprn, int gprn) ++{ ++ gen_helper_store_ptcr(cpu_env, cpu_gpr[gprn]); ++} ++ + #endif + #endif + +@@ -8167,6 +8172,18 @@ static void gen_spr_power8_rpr(CPUPPCState *env) + #endif + } + ++static void gen_spr_power9_mmu(CPUPPCState *env) ++{ ++#if !defined(CONFIG_USER_ONLY) ++ /* Partition Table Control */ ++ spr_register_hv(env, SPR_PTCR, "PTCR", ++ SPR_NOACCESS, SPR_NOACCESS, ++ SPR_NOACCESS, SPR_NOACCESS, ++ &spr_read_generic, &spr_write_ptcr, ++ 0x00000000); ++#endif ++} ++ + static void init_proc_book3s_common(CPUPPCState *env) + { + gen_spr_ne_601(env); +@@ -8761,6 +8778,7 @@ static void init_proc_POWER9(CPUPPCState *env) + gen_spr_power8_ic(env); + gen_spr_power8_book4(env); + gen_spr_power8_rpr(env); ++ gen_spr_power9_mmu(env); + + /* POWER9 Specific registers */ + spr_register_kvm(env, SPR_TIDR, "TIDR", NULL, NULL, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-AIO_WAIT_WHILE-in-job-.commit-.abort.patch b/SOURCES/kvm-test-bdrv-drain-AIO_WAIT_WHILE-in-job-.commit-.abort.patch new file mode 100644 index 0000000..06c2eff --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-AIO_WAIT_WHILE-in-job-.commit-.abort.patch @@ -0,0 +1,241 @@ +From be943af669f0514cd0a83b1a9f1c984530459ba5 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:10 +0100 +Subject: [PATCH 44/49] test-bdrv-drain: AIO_WAIT_WHILE() in job .commit/.abort + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-32-kwolf@redhat.com> +Patchwork-id: 82620 +O-Subject: [RHEL-8 qemu-kvm PATCH 41/44] test-bdrv-drain: AIO_WAIT_WHILE() in job .commit/.abort +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +This adds tests for calling AIO_WAIT_WHILE() in the .commit and .abort +callbacks. Both reasons why .abort could be called for a single job are +tested: Either .run or .prepare could return an error. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +(cherry picked from commit d49725af46a7710cde02cc120b7f1e485154b483) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 116 +++++++++++++++++++++++++++++++++++++++++++----- + 1 file changed, 104 insertions(+), 12 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index f4b57f7..d6202b2 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -784,6 +784,8 @@ static void test_iothread_drain_subtree(void) + + typedef struct TestBlockJob { + BlockJob common; ++ int run_ret; ++ int prepare_ret; + bool should_complete; + } TestBlockJob; + +@@ -793,7 +795,23 @@ static int test_job_prepare(Job *job) + + /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ + blk_flush(s->common.blk); +- return 0; ++ return s->prepare_ret; ++} ++ ++static void test_job_commit(Job *job) ++{ ++ TestBlockJob *s = container_of(job, TestBlockJob, common.job); ++ ++ /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ ++ blk_flush(s->common.blk); ++} ++ ++static void test_job_abort(Job *job) ++{ ++ TestBlockJob *s = container_of(job, TestBlockJob, common.job); ++ ++ /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ ++ blk_flush(s->common.blk); + } + + static int coroutine_fn test_job_run(Job *job, Error **errp) +@@ -809,7 +827,7 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) + job_pause_point(&s->common.job); + } + +- return 0; ++ return s->run_ret; + } + + static void test_job_complete(Job *job, Error **errp) +@@ -827,14 +845,24 @@ BlockJobDriver test_job_driver = { + .run = test_job_run, + .complete = test_job_complete, + .prepare = test_job_prepare, ++ .commit = test_job_commit, ++ .abort = test_job_abort, + }, + }; + +-static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) ++enum test_job_result { ++ TEST_JOB_SUCCESS, ++ TEST_JOB_FAIL_RUN, ++ TEST_JOB_FAIL_PREPARE, ++}; ++ ++static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, ++ enum test_job_result result) + { + BlockBackend *blk_src, *blk_target; + BlockDriverState *src, *target; + BlockJob *job; ++ TestBlockJob *tjob; + IOThread *iothread = NULL; + AioContext *ctx; + int ret; +@@ -858,9 +886,23 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) + blk_insert_bs(blk_target, target, &error_abort); + + aio_context_acquire(ctx); +- job = block_job_create("job0", &test_job_driver, NULL, src, 0, BLK_PERM_ALL, +- 0, 0, NULL, NULL, &error_abort); ++ tjob = block_job_create("job0", &test_job_driver, NULL, src, ++ 0, BLK_PERM_ALL, ++ 0, 0, NULL, NULL, &error_abort); ++ job = &tjob->common; + block_job_add_bdrv(job, "target", target, 0, BLK_PERM_ALL, &error_abort); ++ ++ switch (result) { ++ case TEST_JOB_SUCCESS: ++ break; ++ case TEST_JOB_FAIL_RUN: ++ tjob->run_ret = -EIO; ++ break; ++ case TEST_JOB_FAIL_PREPARE: ++ tjob->prepare_ret = -EIO; ++ break; ++ } ++ + job_start(&job->job); + aio_context_release(ctx); + +@@ -918,7 +960,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) + + aio_context_acquire(ctx); + ret = job_complete_sync(&job->job, &error_abort); +- g_assert_cmpint(ret, ==, 0); ++ g_assert_cmpint(ret, ==, (result == TEST_JOB_SUCCESS ? 0 : -EIO)); + + if (use_iothread) { + blk_set_aio_context(blk_src, qemu_get_aio_context()); +@@ -937,32 +979,68 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) + + static void test_blockjob_drain_all(void) + { +- test_blockjob_common(BDRV_DRAIN_ALL, false); ++ test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_SUCCESS); + } + + static void test_blockjob_drain(void) + { +- test_blockjob_common(BDRV_DRAIN, false); ++ test_blockjob_common(BDRV_DRAIN, false, TEST_JOB_SUCCESS); + } + + static void test_blockjob_drain_subtree(void) + { +- test_blockjob_common(BDRV_SUBTREE_DRAIN, false); ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, false, TEST_JOB_SUCCESS); ++} ++ ++static void test_blockjob_error_drain_all(void) ++{ ++ test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_FAIL_RUN); ++ test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_FAIL_PREPARE); ++} ++ ++static void test_blockjob_error_drain(void) ++{ ++ test_blockjob_common(BDRV_DRAIN, false, TEST_JOB_FAIL_RUN); ++ test_blockjob_common(BDRV_DRAIN, false, TEST_JOB_FAIL_PREPARE); ++} ++ ++static void test_blockjob_error_drain_subtree(void) ++{ ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, false, TEST_JOB_FAIL_RUN); ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, false, TEST_JOB_FAIL_PREPARE); + } + + static void test_blockjob_iothread_drain_all(void) + { +- test_blockjob_common(BDRV_DRAIN_ALL, true); ++ test_blockjob_common(BDRV_DRAIN_ALL, true, TEST_JOB_SUCCESS); + } + + static void test_blockjob_iothread_drain(void) + { +- test_blockjob_common(BDRV_DRAIN, true); ++ test_blockjob_common(BDRV_DRAIN, true, TEST_JOB_SUCCESS); + } + + static void test_blockjob_iothread_drain_subtree(void) + { +- test_blockjob_common(BDRV_SUBTREE_DRAIN, true); ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, true, TEST_JOB_SUCCESS); ++} ++ ++static void test_blockjob_iothread_error_drain_all(void) ++{ ++ test_blockjob_common(BDRV_DRAIN_ALL, true, TEST_JOB_FAIL_RUN); ++ test_blockjob_common(BDRV_DRAIN_ALL, true, TEST_JOB_FAIL_PREPARE); ++} ++ ++static void test_blockjob_iothread_error_drain(void) ++{ ++ test_blockjob_common(BDRV_DRAIN, true, TEST_JOB_FAIL_RUN); ++ test_blockjob_common(BDRV_DRAIN, true, TEST_JOB_FAIL_PREPARE); ++} ++ ++static void test_blockjob_iothread_error_drain_subtree(void) ++{ ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, true, TEST_JOB_FAIL_RUN); ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, true, TEST_JOB_FAIL_PREPARE); + } + + +@@ -1433,6 +1511,13 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/blockjob/drain_subtree", + test_blockjob_drain_subtree); + ++ g_test_add_func("/bdrv-drain/blockjob/error/drain_all", ++ test_blockjob_error_drain_all); ++ g_test_add_func("/bdrv-drain/blockjob/error/drain", ++ test_blockjob_error_drain); ++ g_test_add_func("/bdrv-drain/blockjob/error/drain_subtree", ++ test_blockjob_error_drain_subtree); ++ + g_test_add_func("/bdrv-drain/blockjob/iothread/drain_all", + test_blockjob_iothread_drain_all); + g_test_add_func("/bdrv-drain/blockjob/iothread/drain", +@@ -1440,6 +1525,13 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/blockjob/iothread/drain_subtree", + test_blockjob_iothread_drain_subtree); + ++ g_test_add_func("/bdrv-drain/blockjob/iothread/error/drain_all", ++ test_blockjob_iothread_error_drain_all); ++ g_test_add_func("/bdrv-drain/blockjob/iothread/error/drain", ++ test_blockjob_iothread_error_drain); ++ g_test_add_func("/bdrv-drain/blockjob/iothread/error/drain_subtree", ++ test_blockjob_iothread_error_drain_subtree); ++ + g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); + g_test_add_func("/bdrv-drain/detach/drain_all", test_detach_by_drain_all); + g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Add-test-for-node-deletion.patch b/SOURCES/kvm-test-bdrv-drain-Add-test-for-node-deletion.patch new file mode 100644 index 0000000..b2a261a --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Add-test-for-node-deletion.patch @@ -0,0 +1,226 @@ +From 789fe0e994045478e61395ba88c61a7f07527b25 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:43 +0100 +Subject: [PATCH 12/49] test-bdrv-drain: Add test for node deletion + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-10-kwolf@redhat.com> +Patchwork-id: 82589 +O-Subject: [RHEL-8 qemu-kvm PATCH 09/44] test-bdrv-drain: Add test for node deletion +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +From: Max Reitz + +This patch adds two bdrv-drain tests for what happens if some BDS goes +away during the drainage. + +The basic idea is that you have a parent BDS with some child nodes. +Then, you drain one of the children. Because of that, the party who +actually owns the parent decides to (A) delete it, or (B) detach all its +children from it -- both while the child is still being drained. + +A real-world case where this can happen is the mirror block job, which +may exit if you drain one of its children. + +Signed-off-by: Max Reitz +Signed-off-by: Kevin Wolf +(cherry picked from commit 4c8158e359d194394c64acd21caf5e3f3f3141c2) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 169 ++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 169 insertions(+) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 49786ea..8918a94 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -792,6 +792,172 @@ static void test_blockjob_drain_subtree(void) + test_blockjob_common(BDRV_SUBTREE_DRAIN); + } + ++ ++typedef struct BDRVTestTopState { ++ BdrvChild *wait_child; ++} BDRVTestTopState; ++ ++static void bdrv_test_top_close(BlockDriverState *bs) ++{ ++ BdrvChild *c, *next_c; ++ QLIST_FOREACH_SAFE(c, &bs->children, next, next_c) { ++ bdrv_unref_child(bs, c); ++ } ++} ++ ++static int coroutine_fn bdrv_test_top_co_preadv(BlockDriverState *bs, ++ uint64_t offset, uint64_t bytes, ++ QEMUIOVector *qiov, int flags) ++{ ++ BDRVTestTopState *tts = bs->opaque; ++ return bdrv_co_preadv(tts->wait_child, offset, bytes, qiov, flags); ++} ++ ++static BlockDriver bdrv_test_top_driver = { ++ .format_name = "test_top_driver", ++ .instance_size = sizeof(BDRVTestTopState), ++ ++ .bdrv_close = bdrv_test_top_close, ++ .bdrv_co_preadv = bdrv_test_top_co_preadv, ++ ++ .bdrv_child_perm = bdrv_format_default_perms, ++}; ++ ++typedef struct TestCoDeleteByDrainData { ++ BlockBackend *blk; ++ bool detach_instead_of_delete; ++ bool done; ++} TestCoDeleteByDrainData; ++ ++static void coroutine_fn test_co_delete_by_drain(void *opaque) ++{ ++ TestCoDeleteByDrainData *dbdd = opaque; ++ BlockBackend *blk = dbdd->blk; ++ BlockDriverState *bs = blk_bs(blk); ++ BDRVTestTopState *tts = bs->opaque; ++ void *buffer = g_malloc(65536); ++ QEMUIOVector qiov; ++ struct iovec iov = { ++ .iov_base = buffer, ++ .iov_len = 65536, ++ }; ++ ++ qemu_iovec_init_external(&qiov, &iov, 1); ++ ++ /* Pretend some internal write operation from parent to child. ++ * Important: We have to read from the child, not from the parent! ++ * Draining works by first propagating it all up the tree to the ++ * root and then waiting for drainage from root to the leaves ++ * (protocol nodes). If we have a request waiting on the root, ++ * everything will be drained before we go back down the tree, but ++ * we do not want that. We want to be in the middle of draining ++ * when this following requests returns. */ ++ bdrv_co_preadv(tts->wait_child, 0, 65536, &qiov, 0); ++ ++ g_assert_cmpint(bs->refcnt, ==, 1); ++ ++ if (!dbdd->detach_instead_of_delete) { ++ blk_unref(blk); ++ } else { ++ BdrvChild *c, *next_c; ++ QLIST_FOREACH_SAFE(c, &bs->children, next, next_c) { ++ bdrv_unref_child(bs, c); ++ } ++ } ++ ++ dbdd->done = true; ++} ++ ++/** ++ * Test what happens when some BDS has some children, you drain one of ++ * them and this results in the BDS being deleted. ++ * ++ * If @detach_instead_of_delete is set, the BDS is not going to be ++ * deleted but will only detach all of its children. ++ */ ++static void do_test_delete_by_drain(bool detach_instead_of_delete) ++{ ++ BlockBackend *blk; ++ BlockDriverState *bs, *child_bs, *null_bs; ++ BDRVTestTopState *tts; ++ TestCoDeleteByDrainData dbdd; ++ Coroutine *co; ++ ++ bs = bdrv_new_open_driver(&bdrv_test_top_driver, "top", BDRV_O_RDWR, ++ &error_abort); ++ bs->total_sectors = 65536 >> BDRV_SECTOR_BITS; ++ tts = bs->opaque; ++ ++ null_bs = bdrv_open("null-co://", NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL, ++ &error_abort); ++ bdrv_attach_child(bs, null_bs, "null-child", &child_file, &error_abort); ++ ++ /* This child will be the one to pass to requests through to, and ++ * it will stall until a drain occurs */ ++ child_bs = bdrv_new_open_driver(&bdrv_test, "child", BDRV_O_RDWR, ++ &error_abort); ++ child_bs->total_sectors = 65536 >> BDRV_SECTOR_BITS; ++ /* Takes our reference to child_bs */ ++ tts->wait_child = bdrv_attach_child(bs, child_bs, "wait-child", &child_file, ++ &error_abort); ++ ++ /* This child is just there to be deleted ++ * (for detach_instead_of_delete == true) */ ++ null_bs = bdrv_open("null-co://", NULL, NULL, BDRV_O_RDWR | BDRV_O_PROTOCOL, ++ &error_abort); ++ bdrv_attach_child(bs, null_bs, "null-child", &child_file, &error_abort); ++ ++ blk = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); ++ blk_insert_bs(blk, bs, &error_abort); ++ ++ /* Referenced by blk now */ ++ bdrv_unref(bs); ++ ++ g_assert_cmpint(bs->refcnt, ==, 1); ++ g_assert_cmpint(child_bs->refcnt, ==, 1); ++ g_assert_cmpint(null_bs->refcnt, ==, 1); ++ ++ ++ dbdd = (TestCoDeleteByDrainData){ ++ .blk = blk, ++ .detach_instead_of_delete = detach_instead_of_delete, ++ .done = false, ++ }; ++ co = qemu_coroutine_create(test_co_delete_by_drain, &dbdd); ++ qemu_coroutine_enter(co); ++ ++ /* Drain the child while the read operation is still pending. ++ * This should result in the operation finishing and ++ * test_co_delete_by_drain() resuming. Thus, @bs will be deleted ++ * and the coroutine will exit while this drain operation is still ++ * in progress. */ ++ bdrv_ref(child_bs); ++ bdrv_drain(child_bs); ++ bdrv_unref(child_bs); ++ ++ while (!dbdd.done) { ++ aio_poll(qemu_get_aio_context(), true); ++ } ++ ++ if (detach_instead_of_delete) { ++ /* Here, the reference has not passed over to the coroutine, ++ * so we have to delete the BB ourselves */ ++ blk_unref(blk); ++ } ++} ++ ++ ++static void test_delete_by_drain(void) ++{ ++ do_test_delete_by_drain(false); ++} ++ ++static void test_detach_by_drain(void) ++{ ++ do_test_delete_by_drain(true); ++} ++ ++ + int main(int argc, char **argv) + { + int ret; +@@ -839,6 +1005,9 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/blockjob/drain_subtree", + test_blockjob_drain_subtree); + ++ g_test_add_func("/bdrv-drain/deletion", test_delete_by_drain); ++ g_test_add_func("/bdrv-drain/detach", test_detach_by_drain); ++ + ret = g_test_run(); + qemu_event_destroy(&done_event); + return ret; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Drain-with-block-jobs-in-an-I-O-thre.patch b/SOURCES/kvm-test-bdrv-drain-Drain-with-block-jobs-in-an-I-O-thre.patch new file mode 100644 index 0000000..4d51cce --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Drain-with-block-jobs-in-an-I-O-thre.patch @@ -0,0 +1,208 @@ +From f894047d028d1b80ab1aeb954464542f89d66f8f Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:58 +0100 +Subject: [PATCH 32/49] test-bdrv-drain: Drain with block jobs in an I/O thread + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-20-kwolf@redhat.com> +Patchwork-id: 82608 +O-Subject: [RHEL-8 qemu-kvm PATCH 29/44] test-bdrv-drain: Drain with block jobs in an I/O thread +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +This extends the existing drain test with a block job to include +variants where the block job runs in a different AioContext. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +(cherry picked from commit f62c172959cd2b6de4dd8ba782e855d64d94764b) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 92 +++++++++++++++++++++++++++++++++++++++++++++---- + 1 file changed, 86 insertions(+), 6 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 9bcb3c7..3cf3ba3 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -174,6 +174,28 @@ static void do_drain_end(enum drain_type drain_type, BlockDriverState *bs) + } + } + ++static void do_drain_begin_unlocked(enum drain_type drain_type, BlockDriverState *bs) ++{ ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_acquire(bdrv_get_aio_context(bs)); ++ } ++ do_drain_begin(drain_type, bs); ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_release(bdrv_get_aio_context(bs)); ++ } ++} ++ ++static void do_drain_end_unlocked(enum drain_type drain_type, BlockDriverState *bs) ++{ ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_acquire(bdrv_get_aio_context(bs)); ++ } ++ do_drain_end(drain_type, bs); ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_release(bdrv_get_aio_context(bs)); ++ } ++} ++ + static void test_drv_cb_common(enum drain_type drain_type, bool recursive) + { + BlockBackend *blk; +@@ -785,11 +807,13 @@ BlockJobDriver test_job_driver = { + }, + }; + +-static void test_blockjob_common(enum drain_type drain_type) ++static void test_blockjob_common(enum drain_type drain_type, bool use_iothread) + { + BlockBackend *blk_src, *blk_target; + BlockDriverState *src, *target; + BlockJob *job; ++ IOThread *iothread = NULL; ++ AioContext *ctx; + int ret; + + src = bdrv_new_open_driver(&bdrv_test, "source", BDRV_O_RDWR, +@@ -797,21 +821,31 @@ static void test_blockjob_common(enum drain_type drain_type) + blk_src = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); + blk_insert_bs(blk_src, src, &error_abort); + ++ if (use_iothread) { ++ iothread = iothread_new(); ++ ctx = iothread_get_aio_context(iothread); ++ blk_set_aio_context(blk_src, ctx); ++ } else { ++ ctx = qemu_get_aio_context(); ++ } ++ + target = bdrv_new_open_driver(&bdrv_test, "target", BDRV_O_RDWR, + &error_abort); + blk_target = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); + blk_insert_bs(blk_target, target, &error_abort); + ++ aio_context_acquire(ctx); + job = block_job_create("job0", &test_job_driver, NULL, src, 0, BLK_PERM_ALL, + 0, 0, NULL, NULL, &error_abort); + block_job_add_bdrv(job, "target", target, 0, BLK_PERM_ALL, &error_abort); + job_start(&job->job); ++ aio_context_release(ctx); + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); + g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + +- do_drain_begin(drain_type, src); ++ do_drain_begin_unlocked(drain_type, src); + + if (drain_type == BDRV_DRAIN_ALL) { + /* bdrv_drain_all() drains both src and target */ +@@ -822,7 +856,14 @@ static void test_blockjob_common(enum drain_type drain_type) + g_assert_true(job->job.paused); + g_assert_false(job->job.busy); /* The job is paused */ + +- do_drain_end(drain_type, src); ++ do_drain_end_unlocked(drain_type, src); ++ ++ if (use_iothread) { ++ /* paused is reset in the I/O thread, wait for it */ ++ while (job->job.paused) { ++ aio_poll(qemu_get_aio_context(), false); ++ } ++ } + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +@@ -841,32 +882,64 @@ static void test_blockjob_common(enum drain_type drain_type) + + do_drain_end(drain_type, target); + ++ if (use_iothread) { ++ /* paused is reset in the I/O thread, wait for it */ ++ while (job->job.paused) { ++ aio_poll(qemu_get_aio_context(), false); ++ } ++ } ++ + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); + g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ + ++ aio_context_acquire(ctx); + ret = job_complete_sync(&job->job, &error_abort); + g_assert_cmpint(ret, ==, 0); + ++ if (use_iothread) { ++ blk_set_aio_context(blk_src, qemu_get_aio_context()); ++ } ++ aio_context_release(ctx); ++ + blk_unref(blk_src); + blk_unref(blk_target); + bdrv_unref(src); + bdrv_unref(target); ++ ++ if (iothread) { ++ iothread_join(iothread); ++ } + } + + static void test_blockjob_drain_all(void) + { +- test_blockjob_common(BDRV_DRAIN_ALL); ++ test_blockjob_common(BDRV_DRAIN_ALL, false); + } + + static void test_blockjob_drain(void) + { +- test_blockjob_common(BDRV_DRAIN); ++ test_blockjob_common(BDRV_DRAIN, false); + } + + static void test_blockjob_drain_subtree(void) + { +- test_blockjob_common(BDRV_SUBTREE_DRAIN); ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, false); ++} ++ ++static void test_blockjob_iothread_drain_all(void) ++{ ++ test_blockjob_common(BDRV_DRAIN_ALL, true); ++} ++ ++static void test_blockjob_iothread_drain(void) ++{ ++ test_blockjob_common(BDRV_DRAIN, true); ++} ++ ++static void test_blockjob_iothread_drain_subtree(void) ++{ ++ test_blockjob_common(BDRV_SUBTREE_DRAIN, true); + } + + +@@ -1337,6 +1410,13 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/blockjob/drain_subtree", + test_blockjob_drain_subtree); + ++ g_test_add_func("/bdrv-drain/blockjob/iothread/drain_all", ++ test_blockjob_iothread_drain_all); ++ g_test_add_func("/bdrv-drain/blockjob/iothread/drain", ++ test_blockjob_iothread_drain); ++ g_test_add_func("/bdrv-drain/blockjob/iothread/drain_subtree", ++ test_blockjob_iothread_drain_subtree); ++ + g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); + g_test_add_func("/bdrv-drain/detach/drain_all", test_detach_by_drain_all); + g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Fix-outdated-comments.patch b/SOURCES/kvm-test-bdrv-drain-Fix-outdated-comments.patch new file mode 100644 index 0000000..95e875c --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Fix-outdated-comments.patch @@ -0,0 +1,69 @@ +From f495ce4d9953a5e7634c0891edede7fd75b403de Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:11 +0100 +Subject: [PATCH 45/49] test-bdrv-drain: Fix outdated comments + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-33-kwolf@redhat.com> +Patchwork-id: 82621 +O-Subject: [RHEL-8 qemu-kvm PATCH 42/44] test-bdrv-drain: Fix outdated comments +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Commit 89bd030533e changed the test case from using job_sleep_ns() to +using qemu_co_sleep_ns() instead. Also, block_job_sleep_ns() became +job_sleep_ns() in commit 5d43e86e11f. + +In both cases, some comments in the test case were not updated. Do that +now. + +Reported-by: Max Reitz +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +(cherry picked from commit 5599c162c3bec2bc8f0123e4d5802a70d9984b3b) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index d6202b2..7e7ba9b 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -820,9 +820,9 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) + + job_transition_to_ready(&s->common.job); + while (!s->should_complete) { +- /* Avoid block_job_sleep_ns() because it marks the job as !busy. We +- * want to emulate some actual activity (probably some I/O) here so +- * that drain has to wait for this acitivity to stop. */ ++ /* Avoid job_sleep_ns() because it marks the job as !busy. We want to ++ * emulate some actual activity (probably some I/O) here so that drain ++ * has to wait for this activity to stop. */ + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); + job_pause_point(&s->common.job); + } +@@ -908,7 +908,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ ++ g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + + do_drain_begin_unlocked(drain_type, src); + +@@ -956,7 +956,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); +- g_assert_true(job->job.busy); /* We're in job_sleep_ns() */ ++ g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + + aio_context_acquire(ctx); + ret = job_complete_sync(&job->job, &error_abort); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Graph-change-through-parent-callback.patch b/SOURCES/kvm-test-bdrv-drain-Graph-change-through-parent-callback.patch new file mode 100644 index 0000000..2f9c15b --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Graph-change-through-parent-callback.patch @@ -0,0 +1,173 @@ +From 58476c0b2e67824bd2108055a315f8b9ebc54e31 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:42 +0100 +Subject: [PATCH 16/49] test-bdrv-drain: Graph change through parent callback + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-4-kwolf@redhat.com> +Patchwork-id: 82595 +O-Subject: [RHEL-8 qemu-kvm PATCH 13/44] test-bdrv-drain: Graph change through parent callback +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Signed-off-by: Kevin Wolf +(cherry picked from commit 231281ab42dad2b407b941e36ad11cbc6586e937) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 130 ++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 130 insertions(+) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 38706b0..f786326 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -977,6 +977,135 @@ static void test_detach_by_drain_subtree(void) + } + + ++struct detach_by_parent_data { ++ BlockDriverState *parent_b; ++ BdrvChild *child_b; ++ BlockDriverState *c; ++ BdrvChild *child_c; ++}; ++ ++static void detach_by_parent_aio_cb(void *opaque, int ret) ++{ ++ struct detach_by_parent_data *data = opaque; ++ ++ g_assert_cmpint(ret, ==, 0); ++ bdrv_unref_child(data->parent_b, data->child_b); ++ ++ bdrv_ref(data->c); ++ data->child_c = bdrv_attach_child(data->parent_b, data->c, "PB-C", ++ &child_file, &error_abort); ++} ++ ++/* ++ * Initial graph: ++ * ++ * PA PB ++ * \ / \ ++ * A B C ++ * ++ * PA has a pending write request whose callback changes the child nodes of PB: ++ * It removes B and adds C instead. The subtree of PB is drained, which will ++ * indirectly drain the write request, too. ++ */ ++static void test_detach_by_parent_cb(void) ++{ ++ BlockBackend *blk; ++ BlockDriverState *parent_a, *parent_b, *a, *b, *c; ++ BdrvChild *child_a, *child_b; ++ BlockAIOCB *acb; ++ struct detach_by_parent_data data; ++ ++ QEMUIOVector qiov; ++ struct iovec iov = { ++ .iov_base = NULL, ++ .iov_len = 0, ++ }; ++ qemu_iovec_init_external(&qiov, &iov, 1); ++ ++ /* Create all involved nodes */ ++ parent_a = bdrv_new_open_driver(&bdrv_test, "parent-a", BDRV_O_RDWR, ++ &error_abort); ++ parent_b = bdrv_new_open_driver(&bdrv_test, "parent-b", 0, ++ &error_abort); ++ ++ a = bdrv_new_open_driver(&bdrv_test, "a", BDRV_O_RDWR, &error_abort); ++ b = bdrv_new_open_driver(&bdrv_test, "b", BDRV_O_RDWR, &error_abort); ++ c = bdrv_new_open_driver(&bdrv_test, "c", BDRV_O_RDWR, &error_abort); ++ ++ /* blk is a BB for parent-a */ ++ blk = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); ++ blk_insert_bs(blk, parent_a, &error_abort); ++ bdrv_unref(parent_a); ++ ++ /* Set child relationships */ ++ bdrv_ref(b); ++ bdrv_ref(a); ++ child_b = bdrv_attach_child(parent_b, b, "PB-B", &child_file, &error_abort); ++ child_a = bdrv_attach_child(parent_b, a, "PB-A", &child_backing, &error_abort); ++ ++ bdrv_ref(a); ++ bdrv_attach_child(parent_a, a, "PA-A", &child_file, &error_abort); ++ ++ g_assert_cmpint(parent_a->refcnt, ==, 1); ++ g_assert_cmpint(parent_b->refcnt, ==, 1); ++ g_assert_cmpint(a->refcnt, ==, 3); ++ g_assert_cmpint(b->refcnt, ==, 2); ++ g_assert_cmpint(c->refcnt, ==, 1); ++ ++ g_assert(QLIST_FIRST(&parent_b->children) == child_a); ++ g_assert(QLIST_NEXT(child_a, next) == child_b); ++ g_assert(QLIST_NEXT(child_b, next) == NULL); ++ ++ /* Start the evil write request */ ++ data = (struct detach_by_parent_data) { ++ .parent_b = parent_b, ++ .child_b = child_b, ++ .c = c, ++ }; ++ acb = blk_aio_preadv(blk, 0, &qiov, 0, detach_by_parent_aio_cb, &data); ++ g_assert(acb != NULL); ++ ++ /* Drain and check the expected result */ ++ bdrv_subtree_drained_begin(parent_b); ++ ++ g_assert(data.child_c != NULL); ++ ++ g_assert_cmpint(parent_a->refcnt, ==, 1); ++ g_assert_cmpint(parent_b->refcnt, ==, 1); ++ g_assert_cmpint(a->refcnt, ==, 3); ++ g_assert_cmpint(b->refcnt, ==, 1); ++ g_assert_cmpint(c->refcnt, ==, 2); ++ ++ g_assert(QLIST_FIRST(&parent_b->children) == data.child_c); ++ g_assert(QLIST_NEXT(data.child_c, next) == child_a); ++ g_assert(QLIST_NEXT(child_a, next) == NULL); ++ ++ g_assert_cmpint(parent_a->quiesce_counter, ==, 1); ++ g_assert_cmpint(parent_b->quiesce_counter, ==, 1); ++ g_assert_cmpint(a->quiesce_counter, ==, 1); ++ g_assert_cmpint(b->quiesce_counter, ==, 0); ++ g_assert_cmpint(c->quiesce_counter, ==, 1); ++ ++ bdrv_subtree_drained_end(parent_b); ++ ++ bdrv_unref(parent_b); ++ blk_unref(blk); ++ ++ /* XXX Once bdrv_close() unref's children instead of just detaching them, ++ * this won't be necessary any more. */ ++ bdrv_unref(a); ++ bdrv_unref(a); ++ bdrv_unref(c); ++ ++ g_assert_cmpint(a->refcnt, ==, 1); ++ g_assert_cmpint(b->refcnt, ==, 1); ++ g_assert_cmpint(c->refcnt, ==, 1); ++ bdrv_unref(a); ++ bdrv_unref(b); ++ bdrv_unref(c); ++} ++ ++ + int main(int argc, char **argv) + { + int ret; +@@ -1027,6 +1156,7 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); + g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); + g_test_add_func("/bdrv-drain/detach/drain_subtree", test_detach_by_drain_subtree); ++ g_test_add_func("/bdrv-drain/detach/parent_cb", test_detach_by_parent_cb); + + ret = g_test_run(); + qemu_event_destroy(&done_event); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-AIO_WAIT_WHILE-in-completion-ca.patch b/SOURCES/kvm-test-bdrv-drain-Test-AIO_WAIT_WHILE-in-completion-ca.patch new file mode 100644 index 0000000..def8ddc --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-AIO_WAIT_WHILE-in-completion-ca.patch @@ -0,0 +1,60 @@ +From 1169efb3bbd8600721e981c5992962b4563a2d6a Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:01 +0100 +Subject: [PATCH 35/49] test-bdrv-drain: Test AIO_WAIT_WHILE() in completion + callback + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-23-kwolf@redhat.com> +Patchwork-id: 82611 +O-Subject: [RHEL-8 qemu-kvm PATCH 32/44] test-bdrv-drain: Test AIO_WAIT_WHILE() in completion callback +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +This is a regression test for a deadlock that occurred in block job +completion callbacks (via job_defer_to_main_loop) because the AioContext +lock was taken twice: once in job_finish_sync() and then again in +job_defer_to_main_loop_bh(). This would cause AIO_WAIT_WHILE() to hang. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +(cherry picked from commit ae23dde9dd486e57e152a0ebc9802caddedc45fc) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 10 ++++++++++ + 1 file changed, 10 insertions(+) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 3cf3ba3..05f3b55 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -774,6 +774,15 @@ typedef struct TestBlockJob { + bool should_complete; + } TestBlockJob; + ++static int test_job_prepare(Job *job) ++{ ++ TestBlockJob *s = container_of(job, TestBlockJob, common.job); ++ ++ /* Provoke an AIO_WAIT_WHILE() call to verify there is no deadlock */ ++ blk_flush(s->common.blk); ++ return 0; ++} ++ + static int coroutine_fn test_job_run(Job *job, Error **errp) + { + TestBlockJob *s = container_of(job, TestBlockJob, common.job); +@@ -804,6 +813,7 @@ BlockJobDriver test_job_driver = { + .drain = block_job_drain, + .run = test_job_run, + .complete = test_job_complete, ++ .prepare = test_job_prepare, + }, + }; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-bdrv_append-to-drained-node.patch b/SOURCES/kvm-test-bdrv-drain-Test-bdrv_append-to-drained-node.patch new file mode 100644 index 0000000..b914711 --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-bdrv_append-to-drained-node.patch @@ -0,0 +1,86 @@ +From 38f4c8c2161484794015df1d44a8e84d7e4cddb3 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:51 +0100 +Subject: [PATCH 25/49] test-bdrv-drain: Test bdrv_append() to drained node + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-13-kwolf@redhat.com> +Patchwork-id: 82602 +O-Subject: [RHEL-8 qemu-kvm PATCH 22/44] test-bdrv-drain: Test bdrv_append() to drained node +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Signed-off-by: Kevin Wolf +(cherry picked from commit b994c5bc515fe611885113e7cfa7e87817bfd4e2) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 43 +++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 43 insertions(+) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 1c8162f..9bcb3c7 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -1245,6 +1245,47 @@ static void test_detach_by_driver_cb(void) + test_detach_indirect(false); + } + ++static void test_append_to_drained(void) ++{ ++ BlockBackend *blk; ++ BlockDriverState *base, *overlay; ++ BDRVTestState *base_s, *overlay_s; ++ ++ blk = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); ++ base = bdrv_new_open_driver(&bdrv_test, "base", BDRV_O_RDWR, &error_abort); ++ base_s = base->opaque; ++ blk_insert_bs(blk, base, &error_abort); ++ ++ overlay = bdrv_new_open_driver(&bdrv_test, "overlay", BDRV_O_RDWR, ++ &error_abort); ++ overlay_s = overlay->opaque; ++ ++ do_drain_begin(BDRV_DRAIN, base); ++ g_assert_cmpint(base->quiesce_counter, ==, 1); ++ g_assert_cmpint(base_s->drain_count, ==, 1); ++ g_assert_cmpint(base->in_flight, ==, 0); ++ ++ /* Takes ownership of overlay, so we don't have to unref it later */ ++ bdrv_append(overlay, base, &error_abort); ++ g_assert_cmpint(base->in_flight, ==, 0); ++ g_assert_cmpint(overlay->in_flight, ==, 0); ++ ++ g_assert_cmpint(base->quiesce_counter, ==, 1); ++ g_assert_cmpint(base_s->drain_count, ==, 1); ++ g_assert_cmpint(overlay->quiesce_counter, ==, 1); ++ g_assert_cmpint(overlay_s->drain_count, ==, 1); ++ ++ do_drain_end(BDRV_DRAIN, base); ++ ++ g_assert_cmpint(base->quiesce_counter, ==, 0); ++ g_assert_cmpint(base_s->drain_count, ==, 0); ++ g_assert_cmpint(overlay->quiesce_counter, ==, 0); ++ g_assert_cmpint(overlay_s->drain_count, ==, 0); ++ ++ bdrv_unref(base); ++ blk_unref(blk); ++} ++ + int main(int argc, char **argv) + { + int ret; +@@ -1303,6 +1344,8 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/detach/parent_cb", test_detach_by_parent_cb); + g_test_add_func("/bdrv-drain/detach/driver_cb", test_detach_by_driver_cb); + ++ g_test_add_func("/bdrv-drain/attach/drain", test_append_to_drained); ++ + ret = g_test_run(); + qemu_event_destroy(&done_event); + return ret; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-draining-job-source-child-and-p.patch b/SOURCES/kvm-test-bdrv-drain-Test-draining-job-source-child-and-p.patch new file mode 100644 index 0000000..f389a47 --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-draining-job-source-child-and-p.patch @@ -0,0 +1,199 @@ +From f90f3f9a94cb09c3d568fbc9dc338b5f8c5ea17c Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:13 +0100 +Subject: [PATCH 47/49] test-bdrv-drain: Test draining job source child and + parent + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-35-kwolf@redhat.com> +Patchwork-id: 82624 +O-Subject: [RHEL-8 qemu-kvm PATCH 44/44] test-bdrv-drain: Test draining job source child and parent +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +For the block job drain test, don't only test draining the source and +the target node, but create a backing chain for the source +(source_backing <- source <- source_overlay) and test draining each of +the nodes in it. + +When using iothreads, the source node (and therefore the job) is in a +different AioContext than the drain, which happens from the main +thread. This way, the main thread waits in AIO_WAIT_WHILE() for the +iothread to make process and aio_wait_kick() is required to notify it. +The test validates that calling bdrv_wakeup() for a child or a parent +node will actually notify AIO_WAIT_WHILE() instead of letting it hang. + +Increase the sleep time a bit (to 1 ms) because the test case is racy +and with the shorter sleep, it didn't reproduce the bug it is supposed +to test for me under 'rr record -n'. + +This was because bdrv_drain_invoke_entry() (in the main thread) was only +called after the job had already reached the pause point, so we got a +bdrv_dec_in_flight() from the main thread and the additional +aio_wait_kick() when the job becomes idle (that we really wanted to test +here) wasn't even necessary any more to make progress. + +Signed-off-by: Kevin Wolf +Reviewed-by: Eric Blake +Reviewed-by: Max Reitz +(cherry picked from commit d8b3afd597d54e496809b05ac39ac29a5799664f) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 77 ++++++++++++++++++++++++++++++++++++++++++++----- + 1 file changed, 69 insertions(+), 8 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 7e7ba9b..8641b54 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -786,6 +786,7 @@ typedef struct TestBlockJob { + BlockJob common; + int run_ret; + int prepare_ret; ++ bool running; + bool should_complete; + } TestBlockJob; + +@@ -818,12 +819,17 @@ static int coroutine_fn test_job_run(Job *job, Error **errp) + { + TestBlockJob *s = container_of(job, TestBlockJob, common.job); + ++ /* We are running the actual job code past the pause point in ++ * job_co_entry(). */ ++ s->running = true; ++ + job_transition_to_ready(&s->common.job); + while (!s->should_complete) { + /* Avoid job_sleep_ns() because it marks the job as !busy. We want to + * emulate some actual activity (probably some I/O) here so that drain + * has to wait for this activity to stop. */ +- qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); ++ qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 1000000); ++ + job_pause_point(&s->common.job); + } + +@@ -856,11 +862,19 @@ enum test_job_result { + TEST_JOB_FAIL_PREPARE, + }; + +-static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, +- enum test_job_result result) ++enum test_job_drain_node { ++ TEST_JOB_DRAIN_SRC, ++ TEST_JOB_DRAIN_SRC_CHILD, ++ TEST_JOB_DRAIN_SRC_PARENT, ++}; ++ ++static void test_blockjob_common_drain_node(enum drain_type drain_type, ++ bool use_iothread, ++ enum test_job_result result, ++ enum test_job_drain_node drain_node) + { + BlockBackend *blk_src, *blk_target; +- BlockDriverState *src, *target; ++ BlockDriverState *src, *src_backing, *src_overlay, *target, *drain_bs; + BlockJob *job; + TestBlockJob *tjob; + IOThread *iothread = NULL; +@@ -869,8 +883,32 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + + src = bdrv_new_open_driver(&bdrv_test, "source", BDRV_O_RDWR, + &error_abort); ++ src_backing = bdrv_new_open_driver(&bdrv_test, "source-backing", ++ BDRV_O_RDWR, &error_abort); ++ src_overlay = bdrv_new_open_driver(&bdrv_test, "source-overlay", ++ BDRV_O_RDWR, &error_abort); ++ ++ bdrv_set_backing_hd(src_overlay, src, &error_abort); ++ bdrv_unref(src); ++ bdrv_set_backing_hd(src, src_backing, &error_abort); ++ bdrv_unref(src_backing); ++ + blk_src = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); +- blk_insert_bs(blk_src, src, &error_abort); ++ blk_insert_bs(blk_src, src_overlay, &error_abort); ++ ++ switch (drain_node) { ++ case TEST_JOB_DRAIN_SRC: ++ drain_bs = src; ++ break; ++ case TEST_JOB_DRAIN_SRC_CHILD: ++ drain_bs = src_backing; ++ break; ++ case TEST_JOB_DRAIN_SRC_PARENT: ++ drain_bs = src_overlay; ++ break; ++ default: ++ g_assert_not_reached(); ++ } + + if (use_iothread) { + iothread = iothread_new(); +@@ -906,11 +944,21 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + job_start(&job->job); + aio_context_release(ctx); + ++ if (use_iothread) { ++ /* job_co_entry() is run in the I/O thread, wait for the actual job ++ * code to start (we don't want to catch the job in the pause point in ++ * job_co_entry(). */ ++ while (!tjob->running) { ++ aio_poll(qemu_get_aio_context(), false); ++ } ++ } ++ + g_assert_cmpint(job->job.pause_count, ==, 0); + g_assert_false(job->job.paused); ++ g_assert_true(tjob->running); + g_assert_true(job->job.busy); /* We're in qemu_co_sleep_ns() */ + +- do_drain_begin_unlocked(drain_type, src); ++ do_drain_begin_unlocked(drain_type, drain_bs); + + if (drain_type == BDRV_DRAIN_ALL) { + /* bdrv_drain_all() drains both src and target */ +@@ -921,7 +969,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + g_assert_true(job->job.paused); + g_assert_false(job->job.busy); /* The job is paused */ + +- do_drain_end_unlocked(drain_type, src); ++ do_drain_end_unlocked(drain_type, drain_bs); + + if (use_iothread) { + /* paused is reset in the I/O thread, wait for it */ +@@ -969,7 +1017,7 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + + blk_unref(blk_src); + blk_unref(blk_target); +- bdrv_unref(src); ++ bdrv_unref(src_overlay); + bdrv_unref(target); + + if (iothread) { +@@ -977,6 +1025,19 @@ static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, + } + } + ++static void test_blockjob_common(enum drain_type drain_type, bool use_iothread, ++ enum test_job_result result) ++{ ++ test_blockjob_common_drain_node(drain_type, use_iothread, result, ++ TEST_JOB_DRAIN_SRC); ++ test_blockjob_common_drain_node(drain_type, use_iothread, result, ++ TEST_JOB_DRAIN_SRC_CHILD); ++ if (drain_type == BDRV_SUBTREE_DRAIN) { ++ test_blockjob_common_drain_node(drain_type, use_iothread, result, ++ TEST_JOB_DRAIN_SRC_PARENT); ++ } ++} ++ + static void test_blockjob_drain_all(void) + { + test_blockjob_common(BDRV_DRAIN_ALL, false, TEST_JOB_SUCCESS); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-graph-changes-in-drain_all-sect.patch b/SOURCES/kvm-test-bdrv-drain-Test-graph-changes-in-drain_all-sect.patch new file mode 100644 index 0000000..1ee447e --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-graph-changes-in-drain_all-sect.patch @@ -0,0 +1,151 @@ +From 784ce8c1c23db1b654e85804e6d19508429ce5b5 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:49 +0100 +Subject: [PATCH 23/49] test-bdrv-drain: Test graph changes in drain_all + section + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-11-kwolf@redhat.com> +Patchwork-id: 82600 +O-Subject: [RHEL-8 qemu-kvm PATCH 20/44] test-bdrv-drain: Test graph changes in drain_all section +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +This tests both adding and remove a node between bdrv_drain_all_begin() +and bdrv_drain_all_end(), and enabled the existing detach test for +drain_all. + +Signed-off-by: Kevin Wolf +(cherry picked from commit 19f7a7e574a099dca13120441fbe723cea9c1dc2) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++-- + 1 file changed, 73 insertions(+), 2 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index c4aa913..1c8162f 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -452,7 +452,7 @@ static void test_multiparent(void) + blk_unref(blk_b); + } + +-static void test_graph_change(void) ++static void test_graph_change_drain_subtree(void) + { + BlockBackend *blk_a, *blk_b; + BlockDriverState *bs_a, *bs_b, *backing; +@@ -531,6 +531,63 @@ static void test_graph_change(void) + blk_unref(blk_b); + } + ++static void test_graph_change_drain_all(void) ++{ ++ BlockBackend *blk_a, *blk_b; ++ BlockDriverState *bs_a, *bs_b; ++ BDRVTestState *a_s, *b_s; ++ ++ /* Create node A with a BlockBackend */ ++ blk_a = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); ++ bs_a = bdrv_new_open_driver(&bdrv_test, "test-node-a", BDRV_O_RDWR, ++ &error_abort); ++ a_s = bs_a->opaque; ++ blk_insert_bs(blk_a, bs_a, &error_abort); ++ ++ g_assert_cmpint(bs_a->quiesce_counter, ==, 0); ++ g_assert_cmpint(a_s->drain_count, ==, 0); ++ ++ /* Call bdrv_drain_all_begin() */ ++ bdrv_drain_all_begin(); ++ ++ g_assert_cmpint(bs_a->quiesce_counter, ==, 1); ++ g_assert_cmpint(a_s->drain_count, ==, 1); ++ ++ /* Create node B with a BlockBackend */ ++ blk_b = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); ++ bs_b = bdrv_new_open_driver(&bdrv_test, "test-node-b", BDRV_O_RDWR, ++ &error_abort); ++ b_s = bs_b->opaque; ++ blk_insert_bs(blk_b, bs_b, &error_abort); ++ ++ g_assert_cmpint(bs_a->quiesce_counter, ==, 1); ++ g_assert_cmpint(bs_b->quiesce_counter, ==, 1); ++ g_assert_cmpint(a_s->drain_count, ==, 1); ++ g_assert_cmpint(b_s->drain_count, ==, 1); ++ ++ /* Unref and finally delete node A */ ++ blk_unref(blk_a); ++ ++ g_assert_cmpint(bs_a->quiesce_counter, ==, 1); ++ g_assert_cmpint(bs_b->quiesce_counter, ==, 1); ++ g_assert_cmpint(a_s->drain_count, ==, 1); ++ g_assert_cmpint(b_s->drain_count, ==, 1); ++ ++ bdrv_unref(bs_a); ++ ++ g_assert_cmpint(bs_b->quiesce_counter, ==, 1); ++ g_assert_cmpint(b_s->drain_count, ==, 1); ++ ++ /* End the drained section */ ++ bdrv_drain_all_end(); ++ ++ g_assert_cmpint(bs_b->quiesce_counter, ==, 0); ++ g_assert_cmpint(b_s->drain_count, ==, 0); ++ ++ bdrv_unref(bs_b); ++ blk_unref(blk_b); ++} ++ + struct test_iothread_data { + BlockDriverState *bs; + enum drain_type drain_type; +@@ -966,6 +1023,10 @@ static void do_test_delete_by_drain(bool detach_instead_of_delete, + bdrv_subtree_drained_begin(bs); + bdrv_subtree_drained_end(bs); + break; ++ case BDRV_DRAIN_ALL: ++ bdrv_drain_all_begin(); ++ bdrv_drain_all_end(); ++ break; + default: + g_assert_not_reached(); + } +@@ -986,6 +1047,11 @@ static void test_delete_by_drain(void) + do_test_delete_by_drain(false, BDRV_DRAIN); + } + ++static void test_detach_by_drain_all(void) ++{ ++ do_test_delete_by_drain(true, BDRV_DRAIN_ALL); ++} ++ + static void test_detach_by_drain(void) + { + do_test_delete_by_drain(true, BDRV_DRAIN); +@@ -1214,7 +1280,11 @@ int main(int argc, char **argv) + + g_test_add_func("/bdrv-drain/nested", test_nested); + g_test_add_func("/bdrv-drain/multiparent", test_multiparent); +- g_test_add_func("/bdrv-drain/graph-change", test_graph_change); ++ ++ g_test_add_func("/bdrv-drain/graph-change/drain_subtree", ++ test_graph_change_drain_subtree); ++ g_test_add_func("/bdrv-drain/graph-change/drain_all", ++ test_graph_change_drain_all); + + g_test_add_func("/bdrv-drain/iothread/drain_all", test_iothread_drain_all); + g_test_add_func("/bdrv-drain/iothread/drain", test_iothread_drain); +@@ -1227,6 +1297,7 @@ int main(int argc, char **argv) + test_blockjob_drain_subtree); + + g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); ++ g_test_add_func("/bdrv-drain/detach/drain_all", test_detach_by_drain_all); + g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); + g_test_add_func("/bdrv-drain/detach/drain_subtree", test_detach_by_drain_subtree); + g_test_add_func("/bdrv-drain/detach/parent_cb", test_detach_by_parent_cb); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-nested-poll-in-bdrv_drain_poll_.patch b/SOURCES/kvm-test-bdrv-drain-Test-nested-poll-in-bdrv_drain_poll_.patch new file mode 100644 index 0000000..230741a --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-nested-poll-in-bdrv_drain_poll_.patch @@ -0,0 +1,64 @@ +From ad9c7d63179e24afa5357ada56fccaeab10545c7 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:22:08 +0100 +Subject: [PATCH 42/49] test-bdrv-drain: Test nested poll in + bdrv_drain_poll_top_level() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-30-kwolf@redhat.com> +Patchwork-id: 82618 +O-Subject: [RHEL-8 qemu-kvm PATCH 39/44] test-bdrv-drain: Test nested poll in bdrv_drain_poll_top_level() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +This is a regression test for a deadlock that could occur in callbacks +called from the aio_poll() in bdrv_drain_poll_top_level(). The +AioContext lock wasn't released and therefore would be taken a second +time in the callback. This would cause a possible AIO_WAIT_WHILE() in +the callback to hang. + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +(cherry picked from commit ecc1a5c790cf2c7732cb9755ca388c2fe108d1a1) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 13 +++++++++++++ + 1 file changed, 13 insertions(+) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 05f3b55..f4b57f7 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -636,6 +636,17 @@ static void test_iothread_aio_cb(void *opaque, int ret) + qemu_event_set(&done_event); + } + ++static void test_iothread_main_thread_bh(void *opaque) ++{ ++ struct test_iothread_data *data = opaque; ++ ++ /* Test that the AioContext is not yet locked in a random BH that is ++ * executed during drain, otherwise this would deadlock. */ ++ aio_context_acquire(bdrv_get_aio_context(data->bs)); ++ bdrv_flush(data->bs); ++ aio_context_release(bdrv_get_aio_context(data->bs)); ++} ++ + /* + * Starts an AIO request on a BDS that runs in the AioContext of iothread 1. + * The request involves a BH on iothread 2 before it can complete. +@@ -705,6 +716,8 @@ static void test_iothread_common(enum drain_type drain_type, int drain_thread) + aio_context_acquire(ctx_a); + } + ++ aio_bh_schedule_oneshot(ctx_a, test_iothread_main_thread_bh, &data); ++ + /* The request is running on the IOThread a. Draining its block device + * will make sure that it has completed as far as the BDS is concerned, + * but the drain in this thread can continue immediately after +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-node-deletion-in-subtree-recurs.patch b/SOURCES/kvm-test-bdrv-drain-Test-node-deletion-in-subtree-recurs.patch new file mode 100644 index 0000000..d96b4f2 --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-node-deletion-in-subtree-recurs.patch @@ -0,0 +1,106 @@ +From 419e706e717dd4fa9c63c118590788aea53b4001 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:40 +0100 +Subject: [PATCH 14/49] test-bdrv-drain: Test node deletion in subtree + recursion + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-2-kwolf@redhat.com> +Patchwork-id: 82591 +O-Subject: [RHEL-8 qemu-kvm PATCH 11/44] test-bdrv-drain: Test node deletion in subtree recursion +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +If bdrv_do_drained_begin() polls during its subtree recursion, the graph +can change and mess up the bs->children iteration. Test that this +doesn't happen. + +Signed-off-by: Kevin Wolf +(cherry picked from commit ebd31837618cdc7bda83090773dcdd87475d55b7) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 38 +++++++++++++++++++++++++++++--------- + 1 file changed, 29 insertions(+), 9 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 8918a94..38706b0 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -875,7 +875,8 @@ static void coroutine_fn test_co_delete_by_drain(void *opaque) + * If @detach_instead_of_delete is set, the BDS is not going to be + * deleted but will only detach all of its children. + */ +-static void do_test_delete_by_drain(bool detach_instead_of_delete) ++static void do_test_delete_by_drain(bool detach_instead_of_delete, ++ enum drain_type drain_type) + { + BlockBackend *blk; + BlockDriverState *bs, *child_bs, *null_bs; +@@ -931,9 +932,23 @@ static void do_test_delete_by_drain(bool detach_instead_of_delete) + * test_co_delete_by_drain() resuming. Thus, @bs will be deleted + * and the coroutine will exit while this drain operation is still + * in progress. */ +- bdrv_ref(child_bs); +- bdrv_drain(child_bs); +- bdrv_unref(child_bs); ++ switch (drain_type) { ++ case BDRV_DRAIN: ++ bdrv_ref(child_bs); ++ bdrv_drain(child_bs); ++ bdrv_unref(child_bs); ++ break; ++ case BDRV_SUBTREE_DRAIN: ++ /* Would have to ref/unref bs here for !detach_instead_of_delete, but ++ * then the whole test becomes pointless because the graph changes ++ * don't occur during the drain any more. */ ++ assert(detach_instead_of_delete); ++ bdrv_subtree_drained_begin(bs); ++ bdrv_subtree_drained_end(bs); ++ break; ++ default: ++ g_assert_not_reached(); ++ } + + while (!dbdd.done) { + aio_poll(qemu_get_aio_context(), true); +@@ -946,15 +961,19 @@ static void do_test_delete_by_drain(bool detach_instead_of_delete) + } + } + +- + static void test_delete_by_drain(void) + { +- do_test_delete_by_drain(false); ++ do_test_delete_by_drain(false, BDRV_DRAIN); + } + + static void test_detach_by_drain(void) + { +- do_test_delete_by_drain(true); ++ do_test_delete_by_drain(true, BDRV_DRAIN); ++} ++ ++static void test_detach_by_drain_subtree(void) ++{ ++ do_test_delete_by_drain(true, BDRV_SUBTREE_DRAIN); + } + + +@@ -1005,8 +1024,9 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/blockjob/drain_subtree", + test_blockjob_drain_subtree); + +- g_test_add_func("/bdrv-drain/deletion", test_delete_by_drain); +- g_test_add_func("/bdrv-drain/detach", test_detach_by_drain); ++ g_test_add_func("/bdrv-drain/deletion/drain", test_delete_by_drain); ++ g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); ++ g_test_add_func("/bdrv-drain/detach/drain_subtree", test_detach_by_drain_subtree); + + ret = g_test_run(); + qemu_event_destroy(&done_event); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-Test-that-bdrv_drain_invoke-doesn-t-.patch b/SOURCES/kvm-test-bdrv-drain-Test-that-bdrv_drain_invoke-doesn-t-.patch new file mode 100644 index 0000000..4481bbc --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-Test-that-bdrv_drain_invoke-doesn-t-.patch @@ -0,0 +1,252 @@ +From 74105e9b2d948c03786e0f86f116d5e410e8d1c6 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:44 +0100 +Subject: [PATCH 18/49] test-bdrv-drain: Test that bdrv_drain_invoke() doesn't + poll + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-6-kwolf@redhat.com> +Patchwork-id: 82593 +O-Subject: [RHEL-8 qemu-kvm PATCH 15/44] test-bdrv-drain: Test that bdrv_drain_invoke() doesn't poll +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +This adds a test case that goes wrong if bdrv_drain_invoke() calls +aio_poll(). + +Signed-off-by: Kevin Wolf +(cherry picked from commit 57320ca961c2e8488e1884b4ebbcb929b6901dc6) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 102 +++++++++++++++++++++++++++++++++++++++++------- + 1 file changed, 88 insertions(+), 14 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index f786326..c4aa913 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -34,12 +34,16 @@ static QemuEvent done_event; + typedef struct BDRVTestState { + int drain_count; + AioContext *bh_indirection_ctx; ++ bool sleep_in_drain_begin; + } BDRVTestState; + + static void coroutine_fn bdrv_test_co_drain_begin(BlockDriverState *bs) + { + BDRVTestState *s = bs->opaque; + s->drain_count++; ++ if (s->sleep_in_drain_begin) { ++ qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); ++ } + } + + static void coroutine_fn bdrv_test_co_drain_end(BlockDriverState *bs) +@@ -80,6 +84,22 @@ static int coroutine_fn bdrv_test_co_preadv(BlockDriverState *bs, + return 0; + } + ++static void bdrv_test_child_perm(BlockDriverState *bs, BdrvChild *c, ++ const BdrvChildRole *role, ++ BlockReopenQueue *reopen_queue, ++ uint64_t perm, uint64_t shared, ++ uint64_t *nperm, uint64_t *nshared) ++{ ++ /* bdrv_format_default_perms() accepts only these two, so disguise ++ * detach_by_driver_cb_role as one of them. */ ++ if (role != &child_file && role != &child_backing) { ++ role = &child_file; ++ } ++ ++ bdrv_format_default_perms(bs, c, role, reopen_queue, perm, shared, ++ nperm, nshared); ++} ++ + static BlockDriver bdrv_test = { + .format_name = "test", + .instance_size = sizeof(BDRVTestState), +@@ -90,7 +110,7 @@ static BlockDriver bdrv_test = { + .bdrv_co_drain_begin = bdrv_test_co_drain_begin, + .bdrv_co_drain_end = bdrv_test_co_drain_end, + +- .bdrv_child_perm = bdrv_format_default_perms, ++ .bdrv_child_perm = bdrv_test_child_perm, + }; + + static void aio_ret_cb(void *opaque, int ret) +@@ -982,13 +1002,14 @@ struct detach_by_parent_data { + BdrvChild *child_b; + BlockDriverState *c; + BdrvChild *child_c; ++ bool by_parent_cb; + }; ++static struct detach_by_parent_data detach_by_parent_data; + +-static void detach_by_parent_aio_cb(void *opaque, int ret) ++static void detach_indirect_bh(void *opaque) + { + struct detach_by_parent_data *data = opaque; + +- g_assert_cmpint(ret, ==, 0); + bdrv_unref_child(data->parent_b, data->child_b); + + bdrv_ref(data->c); +@@ -996,6 +1017,25 @@ static void detach_by_parent_aio_cb(void *opaque, int ret) + &child_file, &error_abort); + } + ++static void detach_by_parent_aio_cb(void *opaque, int ret) ++{ ++ struct detach_by_parent_data *data = &detach_by_parent_data; ++ ++ g_assert_cmpint(ret, ==, 0); ++ if (data->by_parent_cb) { ++ detach_indirect_bh(data); ++ } ++} ++ ++static void detach_by_driver_cb_drained_begin(BdrvChild *child) ++{ ++ aio_bh_schedule_oneshot(qemu_get_current_aio_context(), ++ detach_indirect_bh, &detach_by_parent_data); ++ child_file.drained_begin(child); ++} ++ ++static BdrvChildRole detach_by_driver_cb_role; ++ + /* + * Initial graph: + * +@@ -1003,17 +1043,25 @@ static void detach_by_parent_aio_cb(void *opaque, int ret) + * \ / \ + * A B C + * +- * PA has a pending write request whose callback changes the child nodes of PB: +- * It removes B and adds C instead. The subtree of PB is drained, which will +- * indirectly drain the write request, too. ++ * by_parent_cb == true: Test that parent callbacks don't poll ++ * ++ * PA has a pending write request whose callback changes the child nodes of ++ * PB: It removes B and adds C instead. The subtree of PB is drained, which ++ * will indirectly drain the write request, too. ++ * ++ * by_parent_cb == false: Test that bdrv_drain_invoke() doesn't poll ++ * ++ * PA's BdrvChildRole has a .drained_begin callback that schedules a BH ++ * that does the same graph change. If bdrv_drain_invoke() calls it, the ++ * state is messed up, but if it is only polled in the single ++ * BDRV_POLL_WHILE() at the end of the drain, this should work fine. + */ +-static void test_detach_by_parent_cb(void) ++static void test_detach_indirect(bool by_parent_cb) + { + BlockBackend *blk; + BlockDriverState *parent_a, *parent_b, *a, *b, *c; + BdrvChild *child_a, *child_b; + BlockAIOCB *acb; +- struct detach_by_parent_data data; + + QEMUIOVector qiov; + struct iovec iov = { +@@ -1022,6 +1070,12 @@ static void test_detach_by_parent_cb(void) + }; + qemu_iovec_init_external(&qiov, &iov, 1); + ++ if (!by_parent_cb) { ++ detach_by_driver_cb_role = child_file; ++ detach_by_driver_cb_role.drained_begin = ++ detach_by_driver_cb_drained_begin; ++ } ++ + /* Create all involved nodes */ + parent_a = bdrv_new_open_driver(&bdrv_test, "parent-a", BDRV_O_RDWR, + &error_abort); +@@ -1037,6 +1091,13 @@ static void test_detach_by_parent_cb(void) + blk_insert_bs(blk, parent_a, &error_abort); + bdrv_unref(parent_a); + ++ /* If we want to get bdrv_drain_invoke() to call aio_poll(), the driver ++ * callback must not return immediately. */ ++ if (!by_parent_cb) { ++ BDRVTestState *s = parent_a->opaque; ++ s->sleep_in_drain_begin = true; ++ } ++ + /* Set child relationships */ + bdrv_ref(b); + bdrv_ref(a); +@@ -1044,7 +1105,9 @@ static void test_detach_by_parent_cb(void) + child_a = bdrv_attach_child(parent_b, a, "PB-A", &child_backing, &error_abort); + + bdrv_ref(a); +- bdrv_attach_child(parent_a, a, "PA-A", &child_file, &error_abort); ++ bdrv_attach_child(parent_a, a, "PA-A", ++ by_parent_cb ? &child_file : &detach_by_driver_cb_role, ++ &error_abort); + + g_assert_cmpint(parent_a->refcnt, ==, 1); + g_assert_cmpint(parent_b->refcnt, ==, 1); +@@ -1057,18 +1120,19 @@ static void test_detach_by_parent_cb(void) + g_assert(QLIST_NEXT(child_b, next) == NULL); + + /* Start the evil write request */ +- data = (struct detach_by_parent_data) { ++ detach_by_parent_data = (struct detach_by_parent_data) { + .parent_b = parent_b, + .child_b = child_b, + .c = c, ++ .by_parent_cb = by_parent_cb, + }; +- acb = blk_aio_preadv(blk, 0, &qiov, 0, detach_by_parent_aio_cb, &data); ++ acb = blk_aio_preadv(blk, 0, &qiov, 0, detach_by_parent_aio_cb, NULL); + g_assert(acb != NULL); + + /* Drain and check the expected result */ + bdrv_subtree_drained_begin(parent_b); + +- g_assert(data.child_c != NULL); ++ g_assert(detach_by_parent_data.child_c != NULL); + + g_assert_cmpint(parent_a->refcnt, ==, 1); + g_assert_cmpint(parent_b->refcnt, ==, 1); +@@ -1076,8 +1140,8 @@ static void test_detach_by_parent_cb(void) + g_assert_cmpint(b->refcnt, ==, 1); + g_assert_cmpint(c->refcnt, ==, 2); + +- g_assert(QLIST_FIRST(&parent_b->children) == data.child_c); +- g_assert(QLIST_NEXT(data.child_c, next) == child_a); ++ g_assert(QLIST_FIRST(&parent_b->children) == detach_by_parent_data.child_c); ++ g_assert(QLIST_NEXT(detach_by_parent_data.child_c, next) == child_a); + g_assert(QLIST_NEXT(child_a, next) == NULL); + + g_assert_cmpint(parent_a->quiesce_counter, ==, 1); +@@ -1105,6 +1169,15 @@ static void test_detach_by_parent_cb(void) + bdrv_unref(c); + } + ++static void test_detach_by_parent_cb(void) ++{ ++ test_detach_indirect(true); ++} ++ ++static void test_detach_by_driver_cb(void) ++{ ++ test_detach_indirect(false); ++} + + int main(int argc, char **argv) + { +@@ -1157,6 +1230,7 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/detach/drain", test_detach_by_drain); + g_test_add_func("/bdrv-drain/detach/drain_subtree", test_detach_by_drain_subtree); + g_test_add_func("/bdrv-drain/detach/parent_cb", test_detach_by_parent_cb); ++ g_test_add_func("/bdrv-drain/detach/driver_cb", test_detach_by_driver_cb); + + ret = g_test_run(); + qemu_event_destroy(&done_event); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-bdrv-drain-bdrv_drain-works-with-cross-AioConte.patch b/SOURCES/kvm-test-bdrv-drain-bdrv_drain-works-with-cross-AioConte.patch new file mode 100644 index 0000000..c0343b2 --- /dev/null +++ b/SOURCES/kvm-test-bdrv-drain-bdrv_drain-works-with-cross-AioConte.patch @@ -0,0 +1,297 @@ +From ae1deee29ac316ea755c96f15b739db7a59e5c61 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:35 +0100 +Subject: [PATCH 04/49] test-bdrv-drain: bdrv_drain() works with + cross-AioContext events + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-2-kwolf@redhat.com> +Patchwork-id: 82581 +O-Subject: [RHEL-8 qemu-kvm PATCH 01/44] test-bdrv-drain: bdrv_drain() works with cross-AioContext events +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +As long as nobody keeps the other I/O thread from working, there is no +reason why bdrv_drain() wouldn't work with cross-AioContext events. The +key is that the root request we're waiting for is in the AioContext +we're polling (which it always is for bdrv_drain()) so that aio_poll() +is woken up in the end. + +Add a test case that shows that it works. Remove the comment in +bdrv_drain() that claims otherwise. + +Signed-off-by: Kevin Wolf +(cherry picked from commit bb6756895459f181e2f25e877d3d7a10c297b5c8) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + block/io.c | 4 -- + tests/test-bdrv-drain.c | 187 +++++++++++++++++++++++++++++++++++++++++++++++- + 2 files changed, 186 insertions(+), 5 deletions(-) + +diff --git a/block/io.c b/block/io.c +index bb617de..7e0a169 100644 +--- a/block/io.c ++++ b/block/io.c +@@ -369,10 +369,6 @@ void bdrv_unapply_subtree_drain(BdrvChild *child, BlockDriverState *old_parent) + * + * Note that unlike bdrv_drain_all(), the caller must hold the BlockDriverState + * AioContext. +- * +- * Only this BlockDriverState's AioContext is run, so in-flight requests must +- * not depend on events in other AioContexts. In that case, use +- * bdrv_drain_all() instead. + */ + void coroutine_fn bdrv_co_drain(BlockDriverState *bs) + { +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index 403446e..dee0a10 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -27,9 +27,13 @@ + #include "block/blockjob_int.h" + #include "sysemu/block-backend.h" + #include "qapi/error.h" ++#include "iothread.h" ++ ++static QemuEvent done_event; + + typedef struct BDRVTestState { + int drain_count; ++ AioContext *bh_indirection_ctx; + } BDRVTestState; + + static void coroutine_fn bdrv_test_co_drain_begin(BlockDriverState *bs) +@@ -50,16 +54,29 @@ static void bdrv_test_close(BlockDriverState *bs) + g_assert_cmpint(s->drain_count, >, 0); + } + ++static void co_reenter_bh(void *opaque) ++{ ++ aio_co_wake(opaque); ++} ++ + static int coroutine_fn bdrv_test_co_preadv(BlockDriverState *bs, + uint64_t offset, uint64_t bytes, + QEMUIOVector *qiov, int flags) + { ++ BDRVTestState *s = bs->opaque; ++ + /* We want this request to stay until the polling loop in drain waits for + * it to complete. We need to sleep a while as bdrv_drain_invoke() comes + * first and polls its result, too, but it shouldn't accidentally complete + * this request yet. */ + qemu_co_sleep_ns(QEMU_CLOCK_REALTIME, 100000); + ++ if (s->bh_indirection_ctx) { ++ aio_bh_schedule_oneshot(s->bh_indirection_ctx, co_reenter_bh, ++ qemu_coroutine_self()); ++ qemu_coroutine_yield(); ++ } ++ + return 0; + } + +@@ -490,6 +507,164 @@ static void test_graph_change(void) + blk_unref(blk_b); + } + ++struct test_iothread_data { ++ BlockDriverState *bs; ++ enum drain_type drain_type; ++ int *aio_ret; ++}; ++ ++static void test_iothread_drain_entry(void *opaque) ++{ ++ struct test_iothread_data *data = opaque; ++ ++ aio_context_acquire(bdrv_get_aio_context(data->bs)); ++ do_drain_begin(data->drain_type, data->bs); ++ g_assert_cmpint(*data->aio_ret, ==, 0); ++ do_drain_end(data->drain_type, data->bs); ++ aio_context_release(bdrv_get_aio_context(data->bs)); ++ ++ qemu_event_set(&done_event); ++} ++ ++static void test_iothread_aio_cb(void *opaque, int ret) ++{ ++ int *aio_ret = opaque; ++ *aio_ret = ret; ++ qemu_event_set(&done_event); ++} ++ ++/* ++ * Starts an AIO request on a BDS that runs in the AioContext of iothread 1. ++ * The request involves a BH on iothread 2 before it can complete. ++ * ++ * @drain_thread = 0 means that do_drain_begin/end are called from the main ++ * thread, @drain_thread = 1 means that they are called from iothread 1. Drain ++ * for this BDS cannot be called from iothread 2 because only the main thread ++ * may do cross-AioContext polling. ++ */ ++static void test_iothread_common(enum drain_type drain_type, int drain_thread) ++{ ++ BlockBackend *blk; ++ BlockDriverState *bs; ++ BDRVTestState *s; ++ BlockAIOCB *acb; ++ int aio_ret; ++ struct test_iothread_data data; ++ ++ IOThread *a = iothread_new(); ++ IOThread *b = iothread_new(); ++ AioContext *ctx_a = iothread_get_aio_context(a); ++ AioContext *ctx_b = iothread_get_aio_context(b); ++ ++ QEMUIOVector qiov; ++ struct iovec iov = { ++ .iov_base = NULL, ++ .iov_len = 0, ++ }; ++ qemu_iovec_init_external(&qiov, &iov, 1); ++ ++ /* bdrv_drain_all() may only be called from the main loop thread */ ++ if (drain_type == BDRV_DRAIN_ALL && drain_thread != 0) { ++ goto out; ++ } ++ ++ blk = blk_new(BLK_PERM_ALL, BLK_PERM_ALL); ++ bs = bdrv_new_open_driver(&bdrv_test, "test-node", BDRV_O_RDWR, ++ &error_abort); ++ s = bs->opaque; ++ blk_insert_bs(blk, bs, &error_abort); ++ ++ blk_set_aio_context(blk, ctx_a); ++ aio_context_acquire(ctx_a); ++ ++ s->bh_indirection_ctx = ctx_b; ++ ++ aio_ret = -EINPROGRESS; ++ if (drain_thread == 0) { ++ acb = blk_aio_preadv(blk, 0, &qiov, 0, test_iothread_aio_cb, &aio_ret); ++ } else { ++ acb = blk_aio_preadv(blk, 0, &qiov, 0, aio_ret_cb, &aio_ret); ++ } ++ g_assert(acb != NULL); ++ g_assert_cmpint(aio_ret, ==, -EINPROGRESS); ++ ++ aio_context_release(ctx_a); ++ ++ data = (struct test_iothread_data) { ++ .bs = bs, ++ .drain_type = drain_type, ++ .aio_ret = &aio_ret, ++ }; ++ ++ switch (drain_thread) { ++ case 0: ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_acquire(ctx_a); ++ } ++ ++ /* The request is running on the IOThread a. Draining its block device ++ * will make sure that it has completed as far as the BDS is concerned, ++ * but the drain in this thread can continue immediately after ++ * bdrv_dec_in_flight() and aio_ret might be assigned only slightly ++ * later. */ ++ qemu_event_reset(&done_event); ++ do_drain_begin(drain_type, bs); ++ g_assert_cmpint(bs->in_flight, ==, 0); ++ ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_release(ctx_a); ++ } ++ qemu_event_wait(&done_event); ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_acquire(ctx_a); ++ } ++ ++ g_assert_cmpint(aio_ret, ==, 0); ++ do_drain_end(drain_type, bs); ++ ++ if (drain_type != BDRV_DRAIN_ALL) { ++ aio_context_release(ctx_a); ++ } ++ break; ++ case 1: ++ qemu_event_reset(&done_event); ++ aio_bh_schedule_oneshot(ctx_a, test_iothread_drain_entry, &data); ++ qemu_event_wait(&done_event); ++ break; ++ default: ++ g_assert_not_reached(); ++ } ++ ++ aio_context_acquire(ctx_a); ++ blk_set_aio_context(blk, qemu_get_aio_context()); ++ aio_context_release(ctx_a); ++ ++ bdrv_unref(bs); ++ blk_unref(blk); ++ ++out: ++ iothread_join(a); ++ iothread_join(b); ++} ++ ++static void test_iothread_drain_all(void) ++{ ++ test_iothread_common(BDRV_DRAIN_ALL, 0); ++ test_iothread_common(BDRV_DRAIN_ALL, 1); ++} ++ ++static void test_iothread_drain(void) ++{ ++ test_iothread_common(BDRV_DRAIN, 0); ++ test_iothread_common(BDRV_DRAIN, 1); ++} ++ ++static void test_iothread_drain_subtree(void) ++{ ++ test_iothread_common(BDRV_SUBTREE_DRAIN, 0); ++ test_iothread_common(BDRV_SUBTREE_DRAIN, 1); ++} ++ + + typedef struct TestBlockJob { + BlockJob common; +@@ -613,10 +788,13 @@ static void test_blockjob_drain_subtree(void) + + int main(int argc, char **argv) + { ++ int ret; ++ + bdrv_init(); + qemu_init_main_loop(&error_abort); + + g_test_init(&argc, &argv, NULL); ++ qemu_event_init(&done_event, false); + + g_test_add_func("/bdrv-drain/driver-cb/drain_all", test_drv_cb_drain_all); + g_test_add_func("/bdrv-drain/driver-cb/drain", test_drv_cb_drain); +@@ -643,10 +821,17 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/multiparent", test_multiparent); + g_test_add_func("/bdrv-drain/graph-change", test_graph_change); + ++ g_test_add_func("/bdrv-drain/iothread/drain_all", test_iothread_drain_all); ++ g_test_add_func("/bdrv-drain/iothread/drain", test_iothread_drain); ++ g_test_add_func("/bdrv-drain/iothread/drain_subtree", ++ test_iothread_drain_subtree); ++ + g_test_add_func("/bdrv-drain/blockjob/drain_all", test_blockjob_drain_all); + g_test_add_func("/bdrv-drain/blockjob/drain", test_blockjob_drain); + g_test_add_func("/bdrv-drain/blockjob/drain_subtree", + test_blockjob_drain_subtree); + +- return g_test_run(); ++ ret = g_test_run(); ++ qemu_event_destroy(&done_event); ++ return ret; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-blockjob-Acquire-AioContext-around-job_cancel_s.patch b/SOURCES/kvm-test-blockjob-Acquire-AioContext-around-job_cancel_s.patch new file mode 100644 index 0000000..8cbd53c --- /dev/null +++ b/SOURCES/kvm-test-blockjob-Acquire-AioContext-around-job_cancel_s.patch @@ -0,0 +1,87 @@ +From 7d96749f6e1f97c41f37ad6ec2ff23eb3ae27c28 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:59 +0100 +Subject: [PATCH 33/49] test-blockjob: Acquire AioContext around + job_cancel_sync() + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-21-kwolf@redhat.com> +Patchwork-id: 82606 +O-Subject: [RHEL-8 qemu-kvm PATCH 30/44] test-blockjob: Acquire AioContext around job_cancel_sync() +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +All callers in QEMU proper hold the AioContext lock when calling +job_finish_sync(). test-blockjob should do the same when it calls the +function indirectly through job_cancel_sync(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Fam Zheng +(cherry picked from commit 30c070a547322a5e41ce129d540bca3653b1a9c8) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + include/qemu/job.h | 6 ++++++ + tests/test-blockjob.c | 6 ++++++ + 2 files changed, 12 insertions(+) + +diff --git a/include/qemu/job.h b/include/qemu/job.h +index 407d549..35ac7a9 100644 +--- a/include/qemu/job.h ++++ b/include/qemu/job.h +@@ -509,6 +509,8 @@ void job_user_cancel(Job *job, bool force, Error **errp); + * + * Returns the return value from the job if the job actually completed + * during the call, or -ECANCELED if it was canceled. ++ * ++ * Callers must hold the AioContext lock of job->aio_context. + */ + int job_cancel_sync(Job *job); + +@@ -526,6 +528,8 @@ void job_cancel_sync_all(void); + * function). + * + * Returns the return value from the job. ++ * ++ * Callers must hold the AioContext lock of job->aio_context. + */ + int job_complete_sync(Job *job, Error **errp); + +@@ -551,6 +555,8 @@ void job_dismiss(Job **job, Error **errp); + * + * Returns 0 if the job is successfully completed, -ECANCELED if the job was + * cancelled before completing, and -errno in other error cases. ++ * ++ * Callers must hold the AioContext lock of job->aio_context. + */ + int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp); + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index de4c1c2..652d1e8 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -223,6 +223,10 @@ static void cancel_common(CancelJob *s) + BlockJob *job = &s->common; + BlockBackend *blk = s->blk; + JobStatus sts = job->job.status; ++ AioContext *ctx; ++ ++ ctx = job->job.aio_context; ++ aio_context_acquire(ctx); + + job_cancel_sync(&job->job); + if (sts != JOB_STATUS_CREATED && sts != JOB_STATUS_CONCLUDED) { +@@ -232,6 +236,8 @@ static void cancel_common(CancelJob *s) + assert(job->job.status == JOB_STATUS_NULL); + job_unref(&job->job); + destroy_blk(blk); ++ ++ aio_context_release(ctx); + } + + static void test_cancel_created(void) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-test-hbitmap-Add-non-advancing-iter_next-tests.patch b/SOURCES/kvm-test-hbitmap-Add-non-advancing-iter_next-tests.patch new file mode 100644 index 0000000..3a5061f --- /dev/null +++ b/SOURCES/kvm-test-hbitmap-Add-non-advancing-iter_next-tests.patch @@ -0,0 +1,126 @@ +From 4dc36767c7dacd88e0ce52aea59dad6bccb167c4 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 20 Nov 2018 18:18:08 +0000 +Subject: [PATCH 14/35] test-hbitmap: Add non-advancing iter_next tests + +RH-Author: John Snow +Message-id: <20181120181828.15132-5-jsnow@redhat.com> +Patchwork-id: 83064 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 04/24] test-hbitmap: Add non-advancing iter_next tests +Bugzilla: 1518989 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Max Reitz +RH-Acked-by: Stefan Hajnoczi + +From: Max Reitz + +Add a function that wraps hbitmap_iter_next() and always calls it in +non-advancing mode first, and in advancing mode next. The result should +always be the same. + +By using this function everywhere we called hbitmap_iter_next() before, +we should get good test coverage for non-advancing hbitmap_iter_next(). + +Signed-off-by: Max Reitz +Reviewed-by: Fam Zheng +Reviewed-by: John Snow +Message-id: 20180613181823.13618-9-mreitz@redhat.com +Signed-off-by: Max Reitz +(cherry picked from commit 269576848ec3d57d2d958cf5ac69b08c44adf816) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-hbitmap.c | 36 ++++++++++++++++++++++++------------ + 1 file changed, 24 insertions(+), 12 deletions(-) + +diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c +index f2158f7..5e67ac1 100644 +--- a/tests/test-hbitmap.c ++++ b/tests/test-hbitmap.c +@@ -30,6 +30,18 @@ typedef struct TestHBitmapData { + } TestHBitmapData; + + ++static int64_t check_hbitmap_iter_next(HBitmapIter *hbi) ++{ ++ int next0, next1; ++ ++ next0 = hbitmap_iter_next(hbi, false); ++ next1 = hbitmap_iter_next(hbi, true); ++ ++ g_assert_cmpint(next0, ==, next1); ++ ++ return next0; ++} ++ + /* Check that the HBitmap and the shadow bitmap contain the same data, + * ignoring the same "first" bits. + */ +@@ -46,7 +58,7 @@ static void hbitmap_test_check(TestHBitmapData *data, + + i = first; + for (;;) { +- next = hbitmap_iter_next(&hbi, true); ++ next = check_hbitmap_iter_next(&hbi); + if (next < 0) { + next = data->size; + } +@@ -435,25 +447,25 @@ static void test_hbitmap_iter_granularity(TestHBitmapData *data, + /* Note that hbitmap_test_check has to be invoked manually in this test. */ + hbitmap_test_init(data, 131072 << 7, 7); + hbitmap_iter_init(&hbi, data->hb, 0); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), <, 0); + + hbitmap_test_set(data, ((L2 + L1 + 1) << 7) + 8, 8); + hbitmap_iter_init(&hbi, data->hb, 0); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, (L2 + L1 + 1) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), ==, (L2 + L1 + 1) << 7); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), <, 0); + + hbitmap_iter_init(&hbi, data->hb, (L2 + L1 + 2) << 7); + g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); + + hbitmap_test_set(data, (131072 << 7) - 8, 8); + hbitmap_iter_init(&hbi, data->hb, 0); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, (L2 + L1 + 1) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, 131071 << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), ==, (L2 + L1 + 1) << 7); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), ==, 131071 << 7); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), <, 0); + + hbitmap_iter_init(&hbi, data->hb, (L2 + L1 + 2) << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), ==, 131071 << 7); +- g_assert_cmpint(hbitmap_iter_next(&hbi, true), <, 0); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), ==, 131071 << 7); ++ g_assert_cmpint(check_hbitmap_iter_next(&hbi), <, 0); + } + + static void hbitmap_test_set_boundary_bits(TestHBitmapData *data, ssize_t diff) +@@ -893,7 +905,7 @@ static void test_hbitmap_serialize_zeroes(TestHBitmapData *data, + for (i = 0; i < num_positions; i++) { + hbitmap_deserialize_zeroes(data->hb, positions[i], min_l1, true); + hbitmap_iter_init(&iter, data->hb, 0); +- next = hbitmap_iter_next(&iter, true); ++ next = check_hbitmap_iter_next(&iter); + if (i == num_positions - 1) { + g_assert_cmpint(next, ==, -1); + } else { +@@ -919,10 +931,10 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData *data, + + hbitmap_iter_init(&hbi, data->hb, BITS_PER_LONG - 1); + +- hbitmap_iter_next(&hbi, true); ++ check_hbitmap_iter_next(&hbi); + + hbitmap_reset_all(data->hb); +- hbitmap_iter_next(&hbi, true); ++ check_hbitmap_iter_next(&hbi); + } + + static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-tests-blockjob-replace-Blockjob-with-Job.patch b/SOURCES/kvm-tests-blockjob-replace-Blockjob-with-Job.patch new file mode 100644 index 0000000..3bbe22a --- /dev/null +++ b/SOURCES/kvm-tests-blockjob-replace-Blockjob-with-Job.patch @@ -0,0 +1,233 @@ +From 3c31c95203eefe5c58eb53bed04c804c8a2984ce Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:23 +0100 +Subject: [PATCH 20/28] tests/blockjob: replace Blockjob with Job + +RH-Author: John Snow +Message-id: <20180925223431.24791-18-jsnow@redhat.com> +Patchwork-id: 82281 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 17/25] tests/blockjob: replace Blockjob with Job +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +These tests don't actually test blockjobs anymore, they test +generic Job lifetimes. Change the types accordingly. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-9-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 0cc4643b01a0138543e886db8e3bf8a3f74ff8f9) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-blockjob.c | 98 ++++++++++++++++++++++++++------------------------- + 1 file changed, 50 insertions(+), 48 deletions(-) + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index ad4a65b..8e8b680 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -206,18 +206,20 @@ static const BlockJobDriver test_cancel_driver = { + }, + }; + +-static CancelJob *create_common(BlockJob **pjob) ++static CancelJob *create_common(Job **pjob) + { + BlockBackend *blk; +- BlockJob *job; ++ Job *job; ++ BlockJob *bjob; + CancelJob *s; + + blk = create_blk(NULL); +- job = mk_job(blk, "Steve", &test_cancel_driver, true, +- JOB_MANUAL_FINALIZE | JOB_MANUAL_DISMISS); +- job_ref(&job->job); +- assert(job->job.status == JOB_STATUS_CREATED); +- s = container_of(job, CancelJob, common); ++ bjob = mk_job(blk, "Steve", &test_cancel_driver, true, ++ JOB_MANUAL_FINALIZE | JOB_MANUAL_DISMISS); ++ job = &bjob->job; ++ job_ref(job); ++ assert(job->status == JOB_STATUS_CREATED); ++ s = container_of(bjob, CancelJob, common); + s->blk = blk; + + *pjob = job; +@@ -242,7 +244,7 @@ static void cancel_common(CancelJob *s) + + static void test_cancel_created(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); +@@ -251,119 +253,119 @@ static void test_cancel_created(void) + + static void test_cancel_running(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); + +- job_start(&job->job); +- assert(job->job.status == JOB_STATUS_RUNNING); ++ job_start(job); ++ assert(job->status == JOB_STATUS_RUNNING); + + cancel_common(s); + } + + static void test_cancel_paused(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); + +- job_start(&job->job); +- assert(job->job.status == JOB_STATUS_RUNNING); ++ job_start(job); ++ assert(job->status == JOB_STATUS_RUNNING); + +- job_user_pause(&job->job, &error_abort); +- job_enter(&job->job); +- assert(job->job.status == JOB_STATUS_PAUSED); ++ job_user_pause(job, &error_abort); ++ job_enter(job); ++ assert(job->status == JOB_STATUS_PAUSED); + + cancel_common(s); + } + + static void test_cancel_ready(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); + +- job_start(&job->job); +- assert(job->job.status == JOB_STATUS_RUNNING); ++ job_start(job); ++ assert(job->status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- job_enter(&job->job); +- assert(job->job.status == JOB_STATUS_READY); ++ job_enter(job); ++ assert(job->status == JOB_STATUS_READY); + + cancel_common(s); + } + + static void test_cancel_standby(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); + +- job_start(&job->job); +- assert(job->job.status == JOB_STATUS_RUNNING); ++ job_start(job); ++ assert(job->status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- job_enter(&job->job); +- assert(job->job.status == JOB_STATUS_READY); ++ job_enter(job); ++ assert(job->status == JOB_STATUS_READY); + +- job_user_pause(&job->job, &error_abort); +- job_enter(&job->job); +- assert(job->job.status == JOB_STATUS_STANDBY); ++ job_user_pause(job, &error_abort); ++ job_enter(job); ++ assert(job->status == JOB_STATUS_STANDBY); + + cancel_common(s); + } + + static void test_cancel_pending(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); + +- job_start(&job->job); +- assert(job->job.status == JOB_STATUS_RUNNING); ++ job_start(job); ++ assert(job->status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- job_enter(&job->job); +- assert(job->job.status == JOB_STATUS_READY); ++ job_enter(job); ++ assert(job->status == JOB_STATUS_READY); + +- job_complete(&job->job, &error_abort); +- job_enter(&job->job); ++ job_complete(job, &error_abort); ++ job_enter(job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); + } +- assert(job->job.status == JOB_STATUS_PENDING); ++ assert(job->status == JOB_STATUS_PENDING); + + cancel_common(s); + } + + static void test_cancel_concluded(void) + { +- BlockJob *job; ++ Job *job; + CancelJob *s; + + s = create_common(&job); + +- job_start(&job->job); +- assert(job->job.status == JOB_STATUS_RUNNING); ++ job_start(job); ++ assert(job->status == JOB_STATUS_RUNNING); + + s->should_converge = true; +- job_enter(&job->job); +- assert(job->job.status == JOB_STATUS_READY); ++ job_enter(job); ++ assert(job->status == JOB_STATUS_READY); + +- job_complete(&job->job, &error_abort); +- job_enter(&job->job); ++ job_complete(job, &error_abort); ++ job_enter(job); + while (!s->completed) { + aio_poll(qemu_get_aio_context(), true); + } +- assert(job->job.status == JOB_STATUS_PENDING); ++ assert(job->status == JOB_STATUS_PENDING); + +- job_finalize(&job->job, &error_abort); +- assert(job->job.status == JOB_STATUS_CONCLUDED); ++ job_finalize(job, &error_abort); ++ assert(job->status == JOB_STATUS_CONCLUDED); + + cancel_common(s); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-tests-fix-TLS-handshake-failure-with-TLS-1.3.patch b/SOURCES/kvm-tests-fix-TLS-handshake-failure-with-TLS-1.3.patch new file mode 100644 index 0000000..69055a1 --- /dev/null +++ b/SOURCES/kvm-tests-fix-TLS-handshake-failure-with-TLS-1.3.patch @@ -0,0 +1,55 @@ +From dc340428ac10233432dc6048c972197163eb13e7 Mon Sep 17 00:00:00 2001 +From: "Daniel P. Berrange" +Date: Tue, 24 Jul 2018 17:17:43 +0100 +Subject: [PATCH 4/4] tests: fix TLS handshake failure with TLS 1.3 +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Daniel P. Berrange +Message-id: <20180724171743.10146-2-berrange@redhat.com> +Patchwork-id: 81490 +O-Subject: [qemu-kvm RHEL8/virt212 PATCH 1/1] tests: fix TLS handshake failure with TLS 1.3 +Bugzilla: 1602403 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Danilo de Paula + +When gnutls negotiates TLS 1.3 instead of 1.2, the order of messages +sent by the handshake changes. This exposed a logic bug in the test +suite which caused us to wait for the server to see handshake +completion, but not wait for the client to see completion. The result +was the client didn't receive the certificate for verification and the +test failed. + +This is exposed in Fedora 29 rawhide which has just enabled TLS 1.3 in +its GNUTLS builds. + +Reviewed-by: Eric Blake +Signed-off-by: Daniel P. Berrangé +(cherry picked from commit db0a8c70f25fe497c4b786d8edac063daa744c0d) + + Conflicts: + tests/test-crypto-tlssession.c - no PSK tests in 2.12 + +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-crypto-tlssession.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/tests/test-crypto-tlssession.c b/tests/test-crypto-tlssession.c +index 82f21c2..4416a85 100644 +--- a/tests/test-crypto-tlssession.c ++++ b/tests/test-crypto-tlssession.c +@@ -227,7 +227,7 @@ static void test_crypto_tls_session(const void *opaque) + clientShake = true; + } + } +- } while (!clientShake && !serverShake); ++ } while (!clientShake || !serverShake); + + + /* Finally make sure the server validation does what +-- +1.8.3.1 + diff --git a/SOURCES/kvm-tests-test-bdrv-drain-bdrv_drain_all-works-in-corout.patch b/SOURCES/kvm-tests-test-bdrv-drain-bdrv_drain_all-works-in-corout.patch new file mode 100644 index 0000000..fb97b3e --- /dev/null +++ b/SOURCES/kvm-tests-test-bdrv-drain-bdrv_drain_all-works-in-corout.patch @@ -0,0 +1,83 @@ +From 6d23d16153f4fe9c01c9f2e02620ce324219b027 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:08:39 +0100 +Subject: [PATCH 08/49] tests/test-bdrv-drain: bdrv_drain_all() works in + coroutines now + +RH-Author: Kevin Wolf +Message-id: <20181010200843.6710-6-kwolf@redhat.com> +Patchwork-id: 82582 +O-Subject: [RHEL-8 qemu-kvm PATCH 05/44] tests/test-bdrv-drain: bdrv_drain_all() works in coroutines now +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +Since we use bdrv_do_drained_begin/end() for bdrv_drain_all_begin/end(), +coroutine context is automatically left with a BH, preventing the +deadlocks that made bdrv_drain_all*() unsafe in coroutine context. Now +that we even removed the old polling code as dead code, it's obvious +that it's compatible now. + +Enable the coroutine test cases for bdrv_drain_all(). + +Signed-off-by: Kevin Wolf +Reviewed-by: Stefan Hajnoczi +(cherry picked from commit 6d0252f2f9cb49925deb1c41101462c9481dfc90) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-bdrv-drain.c | 16 ++++++++++++++-- + 1 file changed, 14 insertions(+), 2 deletions(-) + +diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c +index f1276a1..f5d85c9 100644 +--- a/tests/test-bdrv-drain.c ++++ b/tests/test-bdrv-drain.c +@@ -233,6 +233,11 @@ static void test_drv_cb_drain_subtree(void) + test_drv_cb_common(BDRV_SUBTREE_DRAIN, true); + } + ++static void test_drv_cb_co_drain_all(void) ++{ ++ call_in_coroutine(test_drv_cb_drain_all); ++} ++ + static void test_drv_cb_co_drain(void) + { + call_in_coroutine(test_drv_cb_drain); +@@ -289,6 +294,11 @@ static void test_quiesce_drain_subtree(void) + test_quiesce_common(BDRV_SUBTREE_DRAIN, true); + } + ++static void test_quiesce_co_drain_all(void) ++{ ++ call_in_coroutine(test_quiesce_drain_all); ++} ++ + static void test_quiesce_co_drain(void) + { + call_in_coroutine(test_quiesce_drain); +@@ -795,7 +805,8 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/driver-cb/drain_subtree", + test_drv_cb_drain_subtree); + +- // XXX bdrv_drain_all() doesn't work in coroutine context ++ g_test_add_func("/bdrv-drain/driver-cb/co/drain_all", ++ test_drv_cb_co_drain_all); + g_test_add_func("/bdrv-drain/driver-cb/co/drain", test_drv_cb_co_drain); + g_test_add_func("/bdrv-drain/driver-cb/co/drain_subtree", + test_drv_cb_co_drain_subtree); +@@ -806,7 +817,8 @@ int main(int argc, char **argv) + g_test_add_func("/bdrv-drain/quiesce/drain_subtree", + test_quiesce_drain_subtree); + +- // XXX bdrv_drain_all() doesn't work in coroutine context ++ g_test_add_func("/bdrv-drain/quiesce/co/drain_all", ++ test_quiesce_co_drain_all); + g_test_add_func("/bdrv-drain/quiesce/co/drain", test_quiesce_co_drain); + g_test_add_func("/bdrv-drain/quiesce/co/drain_subtree", + test_quiesce_co_drain_subtree); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-tests-test-blockjob-remove-exit-callback.patch b/SOURCES/kvm-tests-test-blockjob-remove-exit-callback.patch new file mode 100644 index 0000000..33ff8ce --- /dev/null +++ b/SOURCES/kvm-tests-test-blockjob-remove-exit-callback.patch @@ -0,0 +1,88 @@ +From 811f1e5213c0d9cc05ecfe033f6409f261355d56 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:24 +0100 +Subject: [PATCH 21/28] tests/test-blockjob: remove exit callback + +RH-Author: John Snow +Message-id: <20180925223431.24791-19-jsnow@redhat.com> +Patchwork-id: 82276 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 18/25] tests/test-blockjob: remove exit callback +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +We remove the exit callback and the completed boolean along with it. +We can simulate it just fine by waiting for the job to defer to the +main loop, and then giving it one final kick to get the main loop +portion to run. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-10-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit 977d26fdbeb35d8d2d0f203f9556d44a353e0dfd) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-blockjob.c | 16 ++++++---------- + 1 file changed, 6 insertions(+), 10 deletions(-) + +diff --git a/tests/test-blockjob.c b/tests/test-blockjob.c +index 8e8b680..de4c1c2 100644 +--- a/tests/test-blockjob.c ++++ b/tests/test-blockjob.c +@@ -160,15 +160,8 @@ typedef struct CancelJob { + BlockBackend *blk; + bool should_converge; + bool should_complete; +- bool completed; + } CancelJob; + +-static void cancel_job_exit(Job *job) +-{ +- CancelJob *s = container_of(job, CancelJob, common.job); +- s->completed = true; +-} +- + static void cancel_job_complete(Job *job, Error **errp) + { + CancelJob *s = container_of(job, CancelJob, common.job); +@@ -201,7 +194,6 @@ static const BlockJobDriver test_cancel_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = cancel_job_run, +- .exit = cancel_job_exit, + .complete = cancel_job_complete, + }, + }; +@@ -335,9 +327,11 @@ static void test_cancel_pending(void) + + job_complete(job, &error_abort); + job_enter(job); +- while (!s->completed) { ++ while (!job->deferred_to_main_loop) { + aio_poll(qemu_get_aio_context(), true); + } ++ assert(job->status == JOB_STATUS_READY); ++ aio_poll(qemu_get_aio_context(), true); + assert(job->status == JOB_STATUS_PENDING); + + cancel_common(s); +@@ -359,9 +353,11 @@ static void test_cancel_concluded(void) + + job_complete(job, &error_abort); + job_enter(job); +- while (!s->completed) { ++ while (!job->deferred_to_main_loop) { + aio_poll(qemu_get_aio_context(), true); + } ++ assert(job->status == JOB_STATUS_READY); ++ aio_poll(qemu_get_aio_context(), true); + assert(job->status == JOB_STATUS_PENDING); + + job_finalize(job, &error_abort); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-tests-test-blockjob-txn-move-.exit-to-.clean.patch b/SOURCES/kvm-tests-test-blockjob-txn-move-.exit-to-.clean.patch new file mode 100644 index 0000000..c40a080 --- /dev/null +++ b/SOURCES/kvm-tests-test-blockjob-txn-move-.exit-to-.clean.patch @@ -0,0 +1,53 @@ +From 1b01d1f1273a9307c46fb7c72a407b8ab0a16f77 Mon Sep 17 00:00:00 2001 +From: John Snow +Date: Tue, 25 Sep 2018 22:34:25 +0100 +Subject: [PATCH 22/28] tests/test-blockjob-txn: move .exit to .clean + +RH-Author: John Snow +Message-id: <20180925223431.24791-20-jsnow@redhat.com> +Patchwork-id: 82282 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 19/25] tests/test-blockjob-txn: move .exit to .clean +Bugzilla: 1632939 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Kevin Wolf + +The exit callback in this test actually only performs cleanup. + +Signed-off-by: John Snow +Reviewed-by: Max Reitz +Message-id: 20180906130225.5118-11-jsnow@redhat.com +Reviewed-by: Jeff Cody +Signed-off-by: Max Reitz +(cherry picked from commit e4dad4275d51b594c8abbe726a4927f6f388e427) +Signed-off-by: John Snow +Signed-off-by: Danilo C. L. de Paula +--- + tests/test-blockjob-txn.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +diff --git a/tests/test-blockjob-txn.c b/tests/test-blockjob-txn.c +index ef29f35..86606f9 100644 +--- a/tests/test-blockjob-txn.c ++++ b/tests/test-blockjob-txn.c +@@ -24,7 +24,7 @@ typedef struct { + int *result; + } TestBlockJob; + +-static void test_block_job_exit(Job *job) ++static void test_block_job_clean(Job *job) + { + BlockJob *bjob = container_of(job, BlockJob, job); + BlockDriverState *bs = blk_bs(bjob->blk); +@@ -73,7 +73,7 @@ static const BlockJobDriver test_block_job_driver = { + .user_resume = block_job_user_resume, + .drain = block_job_drain, + .run = test_block_job_run, +- .exit = test_block_job_exit, ++ .clean = test_block_job_clean, + }, + }; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-throttle-groups-fix-hang-when-group-member-leaves.patch b/SOURCES/kvm-throttle-groups-fix-hang-when-group-member-leaves.patch new file mode 100644 index 0000000..ba0e2cd --- /dev/null +++ b/SOURCES/kvm-throttle-groups-fix-hang-when-group-member-leaves.patch @@ -0,0 +1,68 @@ +From 1b487f2a77c49129267e4f779956dd549684d8d9 Mon Sep 17 00:00:00 2001 +From: Stefan Hajnoczi +Date: Mon, 23 Jul 2018 16:36:29 +0200 +Subject: [PATCH 256/268] throttle-groups: fix hang when group member leaves + +RH-Author: Stefan Hajnoczi +Message-id: <20180723163629.7600-2-stefanha@redhat.com> +Patchwork-id: 81469 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] throttle-groups: fix hang when group member leaves +Bugzilla: 1535914 +RH-Acked-by: Fam Zheng +RH-Acked-by: Laurent Vivier +RH-Acked-by: Miroslav Rezanina + +Throttle groups consist of members sharing one throttling state +(including bps/iops limits). Round-robin scheduling is used to ensure +fairness. If a group member already has a timer pending then other +groups members do not schedule their own timers. The next group member +will have its turn when the existing timer expires. + +A hang may occur when a group member leaves while it had a timer +scheduled. Although the code carefully removes the group member from +the round-robin list, it does not schedule the next member. Therefore +remaining members continue to wait for the removed member's timer to +expire. + +This patch schedules the next request if a timer is pending. +Unfortunately the actual bug is a race condition that I've been unable +to capture in a test case. + +Sometimes drive2 hangs when drive1 is removed from the throttling group: + + $ qemu ... -drive if=none,id=drive1,cache=none,format=qcow2,file=data1.qcow2,iops=100,group=foo \ + -device virtio-blk-pci,id=virtio-blk-pci0,drive=drive1 \ + -drive if=none,id=drive2,cache=none,format=qcow2,file=data2.qcow2,iops=10,group=foo \ + -device virtio-blk-pci,id=virtio-blk-pci1,drive=drive2 + (guest-console1)# fio -filename /dev/vda 4k-seq-read.job + (guest-console2)# fio -filename /dev/vdb 4k-seq-read.job + (qmp) {"execute": "block_set_io_throttle", "arguments": {"device": "drive1","bps": 0,"bps_rd": 0,"bps_wr": 0,"iops": 0,"iops_rd": 0,"iops_wr": 0}} + +Reported-by: Nini Gu +Signed-off-by: Stefan Hajnoczi +Message-id: 20180704145410.794-1-stefanha@redhat.com +Cc: Alberto Garcia +Signed-off-by: Stefan Hajnoczi +Signed-off-by: Miroslav Rezanina +--- + block/throttle-groups.c | 4 ++++ + 1 file changed, 4 insertions(+) + +diff --git a/block/throttle-groups.c b/block/throttle-groups.c +index 36cc043..e297b04 100644 +--- a/block/throttle-groups.c ++++ b/block/throttle-groups.c +@@ -564,6 +564,10 @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm) + + qemu_mutex_lock(&tg->lock); + for (i = 0; i < 2; i++) { ++ if (timer_pending(tgm->throttle_timers.timers[i])) { ++ tg->any_timer_armed[i] = false; ++ schedule_next_request(tgm, i); ++ } + if (tg->tokens[i] == tgm) { + token = throttle_group_next_tgm(tgm); + /* Take care of the case where this is the last tgm in the group */ +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ui-Allow-specifying-rendernode-display-option-for-eg.patch b/SOURCES/kvm-ui-Allow-specifying-rendernode-display-option-for-eg.patch new file mode 100644 index 0000000..a714556 --- /dev/null +++ b/SOURCES/kvm-ui-Allow-specifying-rendernode-display-option-for-eg.patch @@ -0,0 +1,57 @@ +From ba493d2395a3833470e4dd12ef1a7b0f32905772 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Fri, 14 Dec 2018 08:26:41 +0000 +Subject: [PATCH 4/5] ui: Allow specifying 'rendernode' display option for + egl-headless +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20181214082642.21878-5-kraxel@redhat.com> +Patchwork-id: 83506 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 4/5] ui: Allow specifying 'rendernode' display option for egl-headless +Bugzilla: 1652871 +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Erik Skultety + +From: Erik Skultety + +As libvirt can't predict which rendernode QEMU would pick, it +won't adjust the permissions on the device, hence QEMU getting +"Permission denied" when opening the DRI device. Therefore, enable +'rendernode' option for egl-headless display type. + +Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1648236 + +Signed-off-by: Erik Skultety +Message-id: 27f4617f19aa1072114f10f1aa9dd199735ef982.1542362949.git.eskultet@redhat.com +Signed-off-by: Gerd Hoffmann +(cherry picked from commit 91e61947eb2be21b00091d34f5692f89cef41376) +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + ui/egl-headless.c + +Signed-off-by: Danilo C. L. de Paula +--- + ui/egl-headless.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/ui/egl-headless.c b/ui/egl-headless.c +index 7c87712..03fec49 100644 +--- a/ui/egl-headless.c ++++ b/ui/egl-headless.c +@@ -175,7 +175,7 @@ static void egl_headless_init(DisplayState *ds, DisplayOptions *opts) + egl_dpy *edpy; + int idx; + +- if (egl_rendernode_init(NULL) < 0) { ++ if (egl_rendernode_init(opts->u.egl_headless.rendernode) < 0) { + error_report("egl: render node init failed"); + exit(1); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ui-add-qapi-parser-for-display.patch b/SOURCES/kvm-ui-add-qapi-parser-for-display.patch new file mode 100644 index 0000000..c560bc9 --- /dev/null +++ b/SOURCES/kvm-ui-add-qapi-parser-for-display.patch @@ -0,0 +1,92 @@ +From f8d17c68e5d3f5b526968a4790a14681fdc69ecd Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Fri, 14 Dec 2018 08:26:38 +0000 +Subject: [PATCH 1/5] ui: add qapi parser for -display +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20181214082642.21878-2-kraxel@redhat.com> +Patchwork-id: 83507 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/5] ui: add qapi parser for -display +Bugzilla: 1652871 +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Erik Skultety + +Add parse_display_qapi() function which parses the -display command line +using a qapi visitor for DisplayOptions. Wire up as default catch in +parse_display(). + +Improves the error message for unknown display types. + +Also enables json as -display argument, i.e. -display "{ 'type': 'gtk' }" + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Eric Blake +Message-id: 20180507095539.19584-2-kraxel@redhat.com +(cherry picked from commit 776d1344bd0566f412d5bc063e753a6b98530bcf) +Signed-off-by: Danilo C. L. de Paula +--- + vl.c | 24 ++++++++++++++++++++++-- + 1 file changed, 22 insertions(+), 2 deletions(-) + +diff --git a/vl.c b/vl.c +index 74fa8f2..934e402 100644 +--- a/vl.c ++++ b/vl.c +@@ -120,12 +120,14 @@ int main(int argc, char **argv) + #include "ui/qemu-spice.h" + #include "qapi/string-input-visitor.h" + #include "qapi/opts-visitor.h" ++#include "qapi/clone-visitor.h" + #include "qom/object_interfaces.h" + #include "exec/semihost.h" + #include "crypto/init.h" + #include "sysemu/replay.h" + #include "qapi/qapi-events-run-state.h" + #include "qapi/qapi-visit-block-core.h" ++#include "qapi/qapi-visit-ui.h" + #include "qapi/qapi-commands-block-core.h" + #include "qapi/qapi-commands-misc.h" + #include "qapi/qapi-commands-run-state.h" +@@ -2117,6 +2119,25 @@ static void select_vgahw(const char *p) + } + } + ++static void parse_display_qapi(const char *optarg) ++{ ++ Error *err = NULL; ++ DisplayOptions *opts; ++ Visitor *v; ++ ++ v = qobject_input_visitor_new_str(optarg, "type", &err); ++ if (!v) { ++ error_report_err(err); ++ exit(1); ++ } ++ ++ visit_type_DisplayOptions(v, NULL, &opts, &error_fatal); ++ QAPI_CLONE_MEMBERS(DisplayOptions, &dpy, opts); ++ ++ qapi_free_DisplayOptions(opts); ++ visit_free(v); ++} ++ + static void parse_display(const char *p) + { + const char *opts; +@@ -2228,8 +2249,7 @@ static void parse_display(const char *p) + } else if (strstart(p, "none", &opts)) { + dpy.type = DISPLAY_TYPE_NONE; + } else { +- error_report("unknown display type"); +- exit(1); ++ parse_display_qapi(p); + } + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-ui-switch-trivial-displays-to-qapi-parser.patch b/SOURCES/kvm-ui-switch-trivial-displays-to-qapi-parser.patch new file mode 100644 index 0000000..92216d6 --- /dev/null +++ b/SOURCES/kvm-ui-switch-trivial-displays-to-qapi-parser.patch @@ -0,0 +1,56 @@ +From 2bddc16c7c813e6166c9ca8545d5d2aaa1b28991 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Fri, 14 Dec 2018 08:26:39 +0000 +Subject: [PATCH 2/5] ui: switch trivial displays to qapi parser +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20181214082642.21878-3-kraxel@redhat.com> +Patchwork-id: 83508 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 2/5] ui: switch trivial displays to qapi parser +Bugzilla: 1652871 +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Erik Skultety + +Drop the option-less display types (egl-headless, curses, none) from +parse_display(), so they'll be handled by parse_display_qapi(). + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Eric Blake +Message-id: 20180507095539.19584-3-kraxel@redhat.com +(cherry picked from commit 2c9498c3e44cd5574df3baaebfb9d5a095252205) +Signed-off-by: Danilo C. L. de Paula +--- + vl.c | 6 ------ + 1 file changed, 6 deletions(-) + +diff --git a/vl.c b/vl.c +index 934e402..a4d1e3f 100644 +--- a/vl.c ++++ b/vl.c +@@ -2210,10 +2210,6 @@ static void parse_display(const char *p) + error_report("VNC requires a display argument vnc="); + exit(1); + } +- } else if (strstart(p, "egl-headless", &opts)) { +- dpy.type = DISPLAY_TYPE_EGL_HEADLESS; +- } else if (strstart(p, "curses", &opts)) { +- dpy.type = DISPLAY_TYPE_CURSES; + } else if (strstart(p, "gtk", &opts)) { + dpy.type = DISPLAY_TYPE_GTK; + while (*opts) { +@@ -2246,8 +2242,6 @@ static void parse_display(const char *p) + } + opts = nextopt; + } +- } else if (strstart(p, "none", &opts)) { +- dpy.type = DISPLAY_TYPE_NONE; + } else { + parse_display_qapi(p); + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-usb-hcd-xhci-test-add-a-test-for-ccid-hotplug.patch b/SOURCES/kvm-usb-hcd-xhci-test-add-a-test-for-ccid-hotplug.patch new file mode 100644 index 0000000..8bd6668 --- /dev/null +++ b/SOURCES/kvm-usb-hcd-xhci-test-add-a-test-for-ccid-hotplug.patch @@ -0,0 +1,66 @@ +From b5a0b93a8a4bf1074456515a370e5e7a17de7272 Mon Sep 17 00:00:00 2001 +From: Serhii Popovych +Date: Mon, 9 Jul 2018 11:31:17 +0200 +Subject: [PATCH 201/268] usb-hcd-xhci-test: add a test for ccid hotplug +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Serhii Popovych +Message-id: <1531135878-18813-3-git-send-email-spopovyc@redhat.com> +Patchwork-id: 81267 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH v2 2/3] usb-hcd-xhci-test: add a test for ccid hotplug +Bugzilla: 1556678 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Marc-André Lureau +RH-Acked-by: David Gibson + +From: Marc-André Lureau + +Signed-off-by: Marc-André Lureau +Message-id: 20180531195119.22021-5-marcandre.lureau@redhat.com +Signed-off-by: Gerd Hoffmann +(cherry picked from commit 1a3ff20e67330a15d62b00c2916e3541872103c0) +Signed-off-by: Serhii Popovych +Signed-off-by: Miroslav Rezanina + +Conflicts: + tests/usb-hcd-xhci-test.c + +Due to disabled UAS that adds if/endif and comment line. +--- + tests/usb-hcd-xhci-test.c | 11 +++++++++++ + 1 file changed, 11 insertions(+) + +diff --git a/tests/usb-hcd-xhci-test.c b/tests/usb-hcd-xhci-test.c +index 192b7f7..8c3fb36 100644 +--- a/tests/usb-hcd-xhci-test.c ++++ b/tests/usb-hcd-xhci-test.c +@@ -37,6 +37,15 @@ static void test_usb_uas_hotplug(void) + } + #endif + ++static void test_usb_ccid_hotplug(void) ++{ ++ qtest_qmp_device_add("usb-ccid", "ccid", NULL); ++ qtest_qmp_device_del("ccid"); ++ /* check the device can be added again */ ++ qtest_qmp_device_add("usb-ccid", "ccid", NULL); ++ qtest_qmp_device_del("ccid"); ++} ++ + int main(int argc, char **argv) + { + int ret; +@@ -48,6 +57,8 @@ int main(int argc, char **argv) + #if 0 /* Disabled for Red Hat Enterprise Linux 7 */ + qtest_add_func("/xhci/pci/hotplug/usb-uas", test_usb_uas_hotplug); + #endif ++ qtest_add_func("/xhci/pci/hotplug/usb-ccid", test_usb_ccid_hotplug); ++ + qtest_start("-device nec-usb-xhci,id=xhci" + " -drive id=drive0,if=none,file=null-co://,format=raw"); + ret = g_test_run(); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-usb-host-skip-open-on-pending-postload-bh.patch b/SOURCES/kvm-usb-host-skip-open-on-pending-postload-bh.patch new file mode 100644 index 0000000..77e0ebc --- /dev/null +++ b/SOURCES/kvm-usb-host-skip-open-on-pending-postload-bh.patch @@ -0,0 +1,78 @@ +From 020cf375b572ccbda55ad45e34b7cb7ad2a469a2 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 13 Jun 2018 14:07:30 +0200 +Subject: [PATCH 151/268] usb-host: skip open on pending postload bh + +RH-Author: Gerd Hoffmann +Message-id: <20180613140730.16401-2-kraxel@redhat.com> +Patchwork-id: 80667 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] usb-host: skip open on pending postload bh +Bugzilla: 1572851 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Laurent Vivier + +usb-host emulates a device unplug after live migration, because the +device state is unknown and unplug/replug makes sure the guest +re-initializes the device into a working state. This can't be done in +post-load though, so post-load just schedules a bottom half which +executes after vmload is complete. + +It can happen that the device autoscan timer hits the race window +between scheduling and running the bottom half, which in turn can +triggers an assert(). + +Fix that issue by just ignoring the usb_host_open() call in case the +bottom half didn't execute yet. + +Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1572851 +Signed-off-by: Gerd Hoffmann +Message-id: 20180503062932.17233-1-kraxel@redhat.com +(cherry picked from commit 3280ea8edede3814553aa19fa27a58daedd48ad9) +Signed-off-by: Miroslav Rezanina +--- + hw/usb/host-libusb.c | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c +index 1b0be07..0290fb8 100644 +--- a/hw/usb/host-libusb.c ++++ b/hw/usb/host-libusb.c +@@ -102,6 +102,7 @@ struct USBHostDevice { + /* callbacks & friends */ + QEMUBH *bh_nodev; + QEMUBH *bh_postld; ++ bool bh_postld_pending; + Notifier exit; + + /* request queues */ +@@ -866,6 +867,10 @@ static int usb_host_open(USBHostDevice *s, libusb_device *dev) + int rc; + Error *local_err = NULL; + ++ if (s->bh_postld_pending) { ++ return -1; ++ } ++ + trace_usb_host_open_started(bus_num, addr); + + if (s->dh != NULL) { +@@ -1524,6 +1529,7 @@ static void usb_host_post_load_bh(void *opaque) + if (udev->attached) { + usb_device_detach(udev); + } ++ dev->bh_postld_pending = false; + usb_host_auto_check(NULL); + } + +@@ -1535,6 +1541,7 @@ static int usb_host_post_load(void *opaque, int version_id) + dev->bh_postld = qemu_bh_new(usb_host_post_load_bh, dev); + } + qemu_bh_schedule(dev->bh_postld); ++ dev->bh_postld_pending = true; + return 0; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-usb-storage-Add-rerror-werror-properties.patch b/SOURCES/kvm-usb-storage-Add-rerror-werror-properties.patch new file mode 100644 index 0000000..193d322 --- /dev/null +++ b/SOURCES/kvm-usb-storage-Add-rerror-werror-properties.patch @@ -0,0 +1,104 @@ +From 4fa8ef564ee9326ceb0b6a4898dee294fa2b4bb4 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Mon, 2 Jul 2018 12:20:17 +0200 +Subject: [PATCH 180/268] usb-storage: Add rerror/werror properties + +RH-Author: Kevin Wolf +Message-id: <20180702122017.29664-2-kwolf@redhat.com> +Patchwork-id: 81183 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH 1/1] usb-storage: Add rerror/werror properties +Bugzilla: 1595180 +RH-Acked-by: Paolo Bonzini +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow + +The error handling policy was traditionally set with -drive, but with +-blockdev it is no longer possible to set frontend options. scsi-disk +(and other block devices) have long supported qdev properties to +configure the error handling policy, so let's add these options to +usb-storage as well and just forward them to the internal scsi-disk +instance. + +Signed-off-by: Kevin Wolf +Reviewed-by: Markus Armbruster +(cherry picked from commit b8efb36b9e99dbea7370139c0866b97a933f78d4) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + hw/scsi/scsi-bus.c | 11 ++++++++++- + hw/usb/dev-storage.c | 2 ++ + include/hw/scsi/scsi.h | 2 ++ + 3 files changed, 14 insertions(+), 1 deletion(-) + +diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c +index 9646743..5905f6b 100644 +--- a/hw/scsi/scsi-bus.c ++++ b/hw/scsi/scsi-bus.c +@@ -226,6 +226,8 @@ static void scsi_qdev_unrealize(DeviceState *qdev, Error **errp) + SCSIDevice *scsi_bus_legacy_add_drive(SCSIBus *bus, BlockBackend *blk, + int unit, bool removable, int bootindex, + bool share_rw, ++ BlockdevOnError rerror, ++ BlockdevOnError werror, + const char *serial, Error **errp) + { + const char *driver; +@@ -262,6 +264,10 @@ SCSIDevice *scsi_bus_legacy_add_drive(SCSIBus *bus, BlockBackend *blk, + object_unparent(OBJECT(dev)); + return NULL; + } ++ ++ qdev_prop_set_enum(dev, "rerror", rerror); ++ qdev_prop_set_enum(dev, "werror", werror); ++ + object_property_set_bool(OBJECT(dev), true, "realized", &err); + if (err != NULL) { + error_propagate(errp, err); +@@ -285,7 +291,10 @@ void scsi_bus_legacy_handle_cmdline(SCSIBus *bus) + } + qemu_opts_loc_restore(dinfo->opts); + scsi_bus_legacy_add_drive(bus, blk_by_legacy_dinfo(dinfo), +- unit, false, -1, false, NULL, &error_fatal); ++ unit, false, -1, false, ++ BLOCKDEV_ON_ERROR_AUTO, ++ BLOCKDEV_ON_ERROR_AUTO, ++ NULL, &error_fatal); + } + loc_pop(&loc); + } +diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c +index b56c75a..68e2062 100644 +--- a/hw/usb/dev-storage.c ++++ b/hw/usb/dev-storage.c +@@ -634,6 +634,7 @@ static void usb_msd_storage_realize(USBDevice *dev, Error **errp) + &usb_msd_scsi_info_storage, NULL); + scsi_dev = scsi_bus_legacy_add_drive(&s->bus, blk, 0, !!s->removable, + s->conf.bootindex, s->conf.share_rw, ++ s->conf.rerror, s->conf.werror, + dev->serial, + errp); + blk_unref(blk); +@@ -687,6 +688,7 @@ static const VMStateDescription vmstate_usb_msd = { + + static Property msd_properties[] = { + DEFINE_BLOCK_PROPERTIES(MSDState, conf), ++ DEFINE_BLOCK_ERROR_PROPERTIES(MSDState, conf), + DEFINE_PROP_BIT("removable", MSDState, removable, 0, false), + DEFINE_PROP_END_OF_LIST(), + }; +diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h +index e35137e..1a7290d 100644 +--- a/include/hw/scsi/scsi.h ++++ b/include/hw/scsi/scsi.h +@@ -154,6 +154,8 @@ static inline SCSIBus *scsi_bus_from_device(SCSIDevice *d) + SCSIDevice *scsi_bus_legacy_add_drive(SCSIBus *bus, BlockBackend *blk, + int unit, bool removable, int bootindex, + bool share_rw, ++ BlockdevOnError rerror, ++ BlockdevOnError werror, + const char *serial, Error **errp); + void scsi_bus_legacy_handle_cmdline(SCSIBus *bus); + void scsi_legacy_handle_cmdline(void); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-util-async-use-qemu_aio_coroutine_enter-in-co_schedu.patch b/SOURCES/kvm-util-async-use-qemu_aio_coroutine_enter-in-co_schedu.patch new file mode 100644 index 0000000..6f8afa4 --- /dev/null +++ b/SOURCES/kvm-util-async-use-qemu_aio_coroutine_enter-in-co_schedu.patch @@ -0,0 +1,79 @@ +From f29b1e17713739baf416b64eeee9549f07717ea8 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Wed, 10 Oct 2018 20:21:53 +0100 +Subject: [PATCH 27/49] util/async: use qemu_aio_coroutine_enter in + co_schedule_bh_cb + +RH-Author: Kevin Wolf +Message-id: <20181010202213.7372-15-kwolf@redhat.com> +Patchwork-id: 82604 +O-Subject: [RHEL-8 qemu-kvm PATCH 24/44] util/async: use qemu_aio_coroutine_enter in co_schedule_bh_cb +Bugzilla: 1637976 +RH-Acked-by: Max Reitz +RH-Acked-by: John Snow +RH-Acked-by: Thomas Huth + +From: Sergio Lopez + +AIO Coroutines shouldn't by managed by an AioContext different than the +one assigned when they are created. aio_co_enter avoids entering a +coroutine from a different AioContext, calling aio_co_schedule instead. + +Scheduled coroutines are then entered by co_schedule_bh_cb using +qemu_coroutine_enter, which just calls qemu_aio_coroutine_enter with the +current AioContext obtained with qemu_get_current_aio_context. +Eventually, co->ctx will be set to the AioContext passed as an argument +to qemu_aio_coroutine_enter. + +This means that, if an IO Thread's AioConext is being processed by the +Main Thread (due to aio_poll being called with a BDS AioContext, as it +happens in AIO_WAIT_WHILE among other places), the AioContext from some +coroutines may be wrongly replaced with the one from the Main Thread. + +This is the root cause behind some crashes, mainly triggered by the +drain code at block/io.c. The most common are these abort and failed +assertion: + +util/async.c:aio_co_schedule +456 if (scheduled) { +457 fprintf(stderr, +458 "%s: Co-routine was already scheduled in '%s'\n", +459 __func__, scheduled); +460 abort(); +461 } + +util/qemu-coroutine-lock.c: +286 assert(mutex->holder == self); + +But it's also known to cause random errors at different locations, and +even SIGSEGV with broken coroutine backtraces. + +By using qemu_aio_coroutine_enter directly in co_schedule_bh_cb, we can +pass the correct AioContext as an argument, making sure co->ctx is not +wrongly altered. + +Signed-off-by: Sergio Lopez +Signed-off-by: Kevin Wolf +(cherry picked from commit 6808ae0417131f8dbe7b051256dff7a16634dc1d) +Signed-off-by: Kevin Wolf +Signed-off-by: Danilo C. L. de Paula +--- + util/async.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/util/async.c b/util/async.c +index 4dd9d95..5693191 100644 +--- a/util/async.c ++++ b/util/async.c +@@ -391,7 +391,7 @@ static void co_schedule_bh_cb(void *opaque) + + /* Protected by write barrier in qemu_aio_coroutine_enter */ + atomic_set(&co->scheduled, NULL); +- qemu_coroutine_enter(co); ++ qemu_aio_coroutine_enter(ctx, co); + aio_context_release(ctx); + } + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-util-implement-simple-iova-tree.patch b/SOURCES/kvm-util-implement-simple-iova-tree.patch new file mode 100644 index 0000000..71c45be --- /dev/null +++ b/SOURCES/kvm-util-implement-simple-iova-tree.patch @@ -0,0 +1,322 @@ +From 8a24ceae4f758b12ac0cda2559b9cbda41487027 Mon Sep 17 00:00:00 2001 +From: Peter Xu +Date: Fri, 12 Oct 2018 07:58:45 +0100 +Subject: [PATCH 15/17] util: implement simple iova tree + +RH-Author: Peter Xu +Message-id: <20181012075846.25449-9-peterx@redhat.com> +Patchwork-id: 82681 +O-Subject: [RHEL-8 qemu-kvm PATCH 8/9] util: implement simple iova tree +Bugzilla: 1450712 +RH-Acked-by: Auger Eric +RH-Acked-by: Xiao Wang +RH-Acked-by: Michael S. Tsirkin + +Introduce a simplest iova tree implementation based on GTree. + +CC: QEMU Stable +Signed-off-by: Peter Xu +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit eecf5eedbdc0fc04f39abcf3afeedfbf21b25ca4) +Signed-off-by: Peter Xu +Signed-off-by: Danilo C. L. de Paula +--- + MAINTAINERS | 6 +++ + include/qemu/iova-tree.h | 134 +++++++++++++++++++++++++++++++++++++++++++++++ + util/Makefile.objs | 1 + + util/iova-tree.c | 114 ++++++++++++++++++++++++++++++++++++++++ + 4 files changed, 255 insertions(+) + create mode 100644 include/qemu/iova-tree.h + create mode 100644 util/iova-tree.c + +diff --git a/MAINTAINERS b/MAINTAINERS +index 53d5216..1e32116 100644 +--- a/MAINTAINERS ++++ b/MAINTAINERS +@@ -1787,6 +1787,12 @@ F: include/sysemu/replay.h + F: docs/replay.txt + F: stubs/replay.c + ++IOVA Tree ++M: Peter Xu ++S: Maintained ++F: include/qemu/iova-tree.h ++F: util/iova-tree.c ++ + Usermode Emulation + ------------------ + Overall +diff --git a/include/qemu/iova-tree.h b/include/qemu/iova-tree.h +new file mode 100644 +index 0000000..b061932 +--- /dev/null ++++ b/include/qemu/iova-tree.h +@@ -0,0 +1,134 @@ ++/* ++ * An very simplified iova tree implementation based on GTree. ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * ++ * Authors: ++ * Peter Xu ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ */ ++#ifndef IOVA_TREE_H ++#define IOVA_TREE_H ++ ++/* ++ * Currently the iova tree will only allow to keep ranges ++ * information, and no extra user data is allowed for each element. A ++ * benefit is that we can merge adjacent ranges internally within the ++ * tree. It can save a lot of memory when the ranges are splitted but ++ * mostly continuous. ++ * ++ * Note that current implementation does not provide any thread ++ * protections. Callers of the iova tree should be responsible ++ * for the thread safety issue. ++ */ ++ ++#include "qemu/osdep.h" ++#include "exec/memory.h" ++#include "exec/hwaddr.h" ++ ++#define IOVA_OK (0) ++#define IOVA_ERR_INVALID (-1) /* Invalid parameters */ ++#define IOVA_ERR_OVERLAP (-2) /* IOVA range overlapped */ ++ ++typedef struct IOVATree IOVATree; ++typedef struct DMAMap { ++ hwaddr iova; ++ hwaddr translated_addr; ++ hwaddr size; /* Inclusive */ ++ IOMMUAccessFlags perm; ++} QEMU_PACKED DMAMap; ++typedef gboolean (*iova_tree_iterator)(DMAMap *map); ++ ++/** ++ * iova_tree_new: ++ * ++ * Create a new iova tree. ++ * ++ * Returns: the tree pointer when succeeded, or NULL if error. ++ */ ++IOVATree *iova_tree_new(void); ++ ++/** ++ * iova_tree_insert: ++ * ++ * @tree: the iova tree to insert ++ * @map: the mapping to insert ++ * ++ * Insert an iova range to the tree. If there is overlapped ++ * ranges, IOVA_ERR_OVERLAP will be returned. ++ * ++ * Return: 0 if succeeded, or <0 if error. ++ */ ++int iova_tree_insert(IOVATree *tree, DMAMap *map); ++ ++/** ++ * iova_tree_remove: ++ * ++ * @tree: the iova tree to remove range from ++ * @map: the map range to remove ++ * ++ * Remove mappings from the tree that are covered by the map range ++ * provided. The range does not need to be exactly what has inserted, ++ * all the mappings that are included in the provided range will be ++ * removed from the tree. Here map->translated_addr is meaningless. ++ * ++ * Return: 0 if succeeded, or <0 if error. ++ */ ++int iova_tree_remove(IOVATree *tree, DMAMap *map); ++ ++/** ++ * iova_tree_find: ++ * ++ * @tree: the iova tree to search from ++ * @map: the mapping to search ++ * ++ * Search for a mapping in the iova tree that overlaps with the ++ * mapping range specified. Only the first found mapping will be ++ * returned. ++ * ++ * Return: DMAMap pointer if found, or NULL if not found. Note that ++ * the returned DMAMap pointer is maintained internally. User should ++ * only read the content but never modify or free the content. Also, ++ * user is responsible to make sure the pointer is valid (say, no ++ * concurrent deletion in progress). ++ */ ++DMAMap *iova_tree_find(IOVATree *tree, DMAMap *map); ++ ++/** ++ * iova_tree_find_address: ++ * ++ * @tree: the iova tree to search from ++ * @iova: the iova address to find ++ * ++ * Similar to iova_tree_find(), but it tries to find mapping with ++ * range iova=iova & size=0. ++ * ++ * Return: same as iova_tree_find(). ++ */ ++DMAMap *iova_tree_find_address(IOVATree *tree, hwaddr iova); ++ ++/** ++ * iova_tree_foreach: ++ * ++ * @tree: the iova tree to iterate on ++ * @iterator: the interator for the mappings, return true to stop ++ * ++ * Iterate over the iova tree. ++ * ++ * Return: 1 if found any overlap, 0 if not, <0 if error. ++ */ ++void iova_tree_foreach(IOVATree *tree, iova_tree_iterator iterator); ++ ++/** ++ * iova_tree_destroy: ++ * ++ * @tree: the iova tree to destroy ++ * ++ * Destroy an existing iova tree. ++ * ++ * Return: None. ++ */ ++void iova_tree_destroy(IOVATree *tree); ++ ++#endif +diff --git a/util/Makefile.objs b/util/Makefile.objs +index 728c354..e1c3fed 100644 +--- a/util/Makefile.objs ++++ b/util/Makefile.objs +@@ -47,4 +47,5 @@ util-obj-y += qht.o + util-obj-y += range.o + util-obj-y += stats64.o + util-obj-y += systemd.o ++util-obj-y += iova-tree.o + util-obj-$(CONFIG_LINUX) += vfio-helpers.o +diff --git a/util/iova-tree.c b/util/iova-tree.c +new file mode 100644 +index 0000000..2d9cebf +--- /dev/null ++++ b/util/iova-tree.c +@@ -0,0 +1,114 @@ ++/* ++ * IOVA tree implementation based on GTree. ++ * ++ * Copyright 2018 Red Hat, Inc. ++ * ++ * Authors: ++ * Peter Xu ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2 or later. ++ */ ++ ++#include ++#include "qemu/iova-tree.h" ++ ++struct IOVATree { ++ GTree *tree; ++}; ++ ++static int iova_tree_compare(gconstpointer a, gconstpointer b, gpointer data) ++{ ++ const DMAMap *m1 = a, *m2 = b; ++ ++ if (m1->iova > m2->iova + m2->size) { ++ return 1; ++ } ++ ++ if (m1->iova + m1->size < m2->iova) { ++ return -1; ++ } ++ ++ /* Overlapped */ ++ return 0; ++} ++ ++IOVATree *iova_tree_new(void) ++{ ++ IOVATree *iova_tree = g_new0(IOVATree, 1); ++ ++ /* We don't have values actually, no need to free */ ++ iova_tree->tree = g_tree_new_full(iova_tree_compare, NULL, g_free, NULL); ++ ++ return iova_tree; ++} ++ ++DMAMap *iova_tree_find(IOVATree *tree, DMAMap *map) ++{ ++ return g_tree_lookup(tree->tree, map); ++} ++ ++DMAMap *iova_tree_find_address(IOVATree *tree, hwaddr iova) ++{ ++ DMAMap map = { .iova = iova, .size = 0 }; ++ ++ return iova_tree_find(tree, &map); ++} ++ ++static inline void iova_tree_insert_internal(GTree *gtree, DMAMap *range) ++{ ++ /* Key and value are sharing the same range data */ ++ g_tree_insert(gtree, range, range); ++} ++ ++int iova_tree_insert(IOVATree *tree, DMAMap *map) ++{ ++ DMAMap *new; ++ ++ if (map->iova + map->size < map->iova || map->perm == IOMMU_NONE) { ++ return IOVA_ERR_INVALID; ++ } ++ ++ /* We don't allow to insert range that overlaps with existings */ ++ if (iova_tree_find(tree, map)) { ++ return IOVA_ERR_OVERLAP; ++ } ++ ++ new = g_new0(DMAMap, 1); ++ memcpy(new, map, sizeof(*new)); ++ iova_tree_insert_internal(tree->tree, new); ++ ++ return IOVA_OK; ++} ++ ++static gboolean iova_tree_traverse(gpointer key, gpointer value, ++ gpointer data) ++{ ++ iova_tree_iterator iterator = data; ++ DMAMap *map = key; ++ ++ g_assert(key == value); ++ ++ return iterator(map); ++} ++ ++void iova_tree_foreach(IOVATree *tree, iova_tree_iterator iterator) ++{ ++ g_tree_foreach(tree->tree, iova_tree_traverse, iterator); ++} ++ ++int iova_tree_remove(IOVATree *tree, DMAMap *map) ++{ ++ DMAMap *overlap; ++ ++ while ((overlap = iova_tree_find(tree, map))) { ++ g_tree_remove(tree->tree, overlap); ++ } ++ ++ return IOVA_OK; ++} ++ ++void iova_tree_destroy(IOVATree *tree) ++{ ++ g_tree_destroy(tree->tree); ++ g_free(tree); ++} +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vdi-Fix-vdi_co_do_create-return-value.patch b/SOURCES/kvm-vdi-Fix-vdi_co_do_create-return-value.patch new file mode 100644 index 0000000..f4bca26 --- /dev/null +++ b/SOURCES/kvm-vdi-Fix-vdi_co_do_create-return-value.patch @@ -0,0 +1,42 @@ +From 1a88053b4231756c8543b60cc7029e4ff570caaf Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:37 +0200 +Subject: [PATCH 129/268] vdi: Fix vdi_co_do_create() return value + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-55-kwolf@redhat.com> +Patchwork-id: 81126 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 54/73] vdi: Fix vdi_co_do_create() return value +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +.bdrv_co_create() is supposed to return 0 on success, but vdi could +return a positive value instead. Fix this. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 53618dd83885cc551a3833e228cf714494602142) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/vdi.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/block/vdi.c b/block/vdi.c +index 41859a8..1d8ed67 100644 +--- a/block/vdi.c ++++ b/block/vdi.c +@@ -865,6 +865,7 @@ static int coroutine_fn vdi_co_do_create(BlockdevCreateOptions *create_options, + } + } + ++ ret = 0; + exit: + blk_unref(blk); + bdrv_unref(bs_file); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-Inhibit-ballooning-based-on-group-attachment-to.patch b/SOURCES/kvm-vfio-Inhibit-ballooning-based-on-group-attachment-to.patch new file mode 100644 index 0000000..dea760a --- /dev/null +++ b/SOURCES/kvm-vfio-Inhibit-ballooning-based-on-group-attachment-to.patch @@ -0,0 +1,129 @@ +From f9416fd5d1232f47af1366c8099003a88dab4a21 Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:01:48 +0000 +Subject: [PATCH 12/16] vfio: Inhibit ballooning based on group attachment to a + container + +RH-Author: Alex Williamson +Message-id: <154387450879.27651.3509144221336190827.stgit@gimli.home> +Patchwork-id: 83238 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 3/7] vfio: Inhibit ballooning based on group attachment to a container +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +We use a VFIOContainer to associate an AddressSpace to one or more +VFIOGroups. The VFIOContainer represents the DMA context for that +AdressSpace for those VFIOGroups and is synchronized to changes in +that AddressSpace via a MemoryListener. For IOMMU backed devices, +maintaining the DMA context for a VFIOGroup generally involves +pinning a host virtual address in order to create a stable host +physical address and then mapping a translation from the associated +guest physical address to that host physical address into the IOMMU. + +While the above maintains the VFIOContainer synchronized to the QEMU +memory API of the VM, memory ballooning occurs outside of that API. +Inflating the memory balloon (ie. cooperatively capturing pages from +the guest for use by the host) simply uses MADV_DONTNEED to "zap" +pages from QEMU's host virtual address space. The page pinning and +IOMMU mapping above remains in place, negating the host's ability to +reuse the page, but the host virtual to host physical mapping of the +page is invalidated outside of QEMU's memory API. + +When the balloon is later deflated, attempting to cooperatively +return pages to the guest, the page is simply freed by the guest +balloon driver, allowing it to be used in the guest and incurring a +page fault when that occurs. The page fault maps a new host physical +page backing the existing host virtual address, meanwhile the +VFIOContainer still maintains the translation to the original host +physical address. At this point the guest vCPU and any assigned +devices will map different host physical addresses to the same guest +physical address. Badness. + +The IOMMU typically does not have page level granularity with which +it can track this mapping without also incurring inefficiencies in +using page size mappings throughout. MMU notifiers in the host +kernel also provide indicators for invalidating the mapping on +balloon inflation, not for updating the mapping when the balloon is +deflated. For these reasons we assume a default behavior that the +mapping of each VFIOGroup into the VFIOContainer is incompatible +with memory ballooning and increment the balloon inhibitor to match +the attached VFIOGroups. + +Reviewed-by: Peter Xu +Signed-off-by: Alex Williamson +(cherry picked from commit c65ee433153b5925e183a00ebf568e160077c694) +Signed-off-by: Danilo C. L. de Paula +--- + hw/vfio/common.c | 30 ++++++++++++++++++++++++++++++ + 1 file changed, 30 insertions(+) + +diff --git a/hw/vfio/common.c b/hw/vfio/common.c +index 07ffa0b..7e8f289 100644 +--- a/hw/vfio/common.c ++++ b/hw/vfio/common.c +@@ -32,6 +32,7 @@ + #include "hw/hw.h" + #include "qemu/error-report.h" + #include "qemu/range.h" ++#include "sysemu/balloon.h" + #include "sysemu/kvm.h" + #include "trace.h" + #include "qapi/error.h" +@@ -1039,6 +1040,33 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, + + space = vfio_get_address_space(as); + ++ /* ++ * VFIO is currently incompatible with memory ballooning insofar as the ++ * madvise to purge (zap) the page from QEMU's address space does not ++ * interact with the memory API and therefore leaves stale virtual to ++ * physical mappings in the IOMMU if the page was previously pinned. We ++ * therefore add a balloon inhibit for each group added to a container, ++ * whether the container is used individually or shared. This provides ++ * us with options to allow devices within a group to opt-in and allow ++ * ballooning, so long as it is done consistently for a group (for instance ++ * if the device is an mdev device where it is known that the host vendor ++ * driver will never pin pages outside of the working set of the guest ++ * driver, which would thus not be ballooning candidates). ++ * ++ * The first opportunity to induce pinning occurs here where we attempt to ++ * attach the group to existing containers within the AddressSpace. If any ++ * pages are already zapped from the virtual address space, such as from a ++ * previous ballooning opt-in, new pinning will cause valid mappings to be ++ * re-established. Likewise, when the overall MemoryListener for a new ++ * container is registered, a replay of mappings within the AddressSpace ++ * will occur, re-establishing any previously zapped pages as well. ++ * ++ * NB. Balloon inhibiting does not currently block operation of the ++ * balloon driver or revoke previously pinned pages, it only prevents ++ * calling madvise to modify the virtual mapping of ballooned pages. ++ */ ++ qemu_balloon_inhibit(true); ++ + QLIST_FOREACH(container, &space->containers, next) { + if (!ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &container->fd)) { + group->container = container; +@@ -1227,6 +1255,7 @@ close_fd_exit: + close(fd); + + put_space_exit: ++ qemu_balloon_inhibit(false); + vfio_put_address_space(space); + + return ret; +@@ -1347,6 +1376,7 @@ void vfio_put_group(VFIOGroup *group) + return; + } + ++ qemu_balloon_inhibit(false); + vfio_kvm_device_del_group(group); + vfio_disconnect_container(group); + QLIST_REMOVE(group, next); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-ccw-pci-Allow-devices-to-opt-in-for-ballooning.patch b/SOURCES/kvm-vfio-ccw-pci-Allow-devices-to-opt-in-for-ballooning.patch new file mode 100644 index 0000000..b94e9a1 --- /dev/null +++ b/SOURCES/kvm-vfio-ccw-pci-Allow-devices-to-opt-in-for-ballooning.patch @@ -0,0 +1,204 @@ +From f37a1e337dd62c873f18aabd31863c8df144c7ea Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:01:54 +0000 +Subject: [PATCH 13/16] vfio/ccw/pci: Allow devices to opt-in for ballooning + +RH-Author: Alex Williamson +Message-id: <154387451469.27651.8657130146789267501.stgit@gimli.home> +Patchwork-id: 83236 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 4/7] vfio/ccw/pci: Allow devices to opt-in for ballooning +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +If a vfio assigned device makes use of a physical IOMMU, then memory +ballooning is necessarily inhibited due to the page pinning, lack of +page level granularity at the IOMMU, and sufficient notifiers to both +remove the page on balloon inflation and add it back on deflation. +However, not all devices are backed by a physical IOMMU. In the case +of mediated devices, if a vendor driver is well synchronized with the +guest driver, such that only pages actively used by the guest driver +are pinned by the host mdev vendor driver, then there should be no +overlap between pages available for the balloon driver and pages +actively in use by the device. Under these conditions, ballooning +should be safe. + +vfio-ccw devices are always mediated devices and always operate under +the constraints above. Therefore we can consider all vfio-ccw devices +as balloon compatible. + +The situation is far from straightforward with vfio-pci. These +devices can be physical devices with physical IOMMU backing or +mediated devices where it is unknown whether a physical IOMMU is in +use or whether the vendor driver is well synchronized to the working +set of the guest driver. The safest approach is therefore to assume +all vfio-pci devices are incompatible with ballooning, but allow user +opt-in should they have further insight into mediated devices. + +Signed-off-by: Alex Williamson +(cherry picked from commit 238e91728503d400e1c4e644e3a9b80f9e621682) +Signed-off-by: Danilo C. L. de Paula +--- + hw/vfio/ccw.c | 9 +++++++++ + hw/vfio/common.c | 23 ++++++++++++++++++++++- + hw/vfio/pci.c | 26 +++++++++++++++++++++++++- + hw/vfio/trace-events | 1 + + include/hw/vfio/vfio-common.h | 2 ++ + 5 files changed, 59 insertions(+), 2 deletions(-) + +diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c +index fe34b50..0c74dda 100644 +--- a/hw/vfio/ccw.c ++++ b/hw/vfio/ccw.c +@@ -362,6 +362,15 @@ static void vfio_ccw_realize(DeviceState *dev, Error **errp) + } + } + ++ /* ++ * All vfio-ccw devices are believed to operate in a way compatible with ++ * memory ballooning, ie. pages pinned in the host are in the current ++ * working set of the guest driver and therefore never overlap with pages ++ * available to the guest balloon driver. This needs to be set before ++ * vfio_get_device() for vfio common to handle the balloon inhibitor. ++ */ ++ vcdev->vdev.balloon_allowed = true; ++ + if (vfio_get_device(group, cdev->mdevid, &vcdev->vdev, &err)) { + g_free(vcdev->vdev.name); + goto out_device_err; +diff --git a/hw/vfio/common.c b/hw/vfio/common.c +index 7e8f289..cda2d1f 100644 +--- a/hw/vfio/common.c ++++ b/hw/vfio/common.c +@@ -1376,7 +1376,9 @@ void vfio_put_group(VFIOGroup *group) + return; + } + +- qemu_balloon_inhibit(false); ++ if (!group->balloon_allowed) { ++ qemu_balloon_inhibit(false); ++ } + vfio_kvm_device_del_group(group); + vfio_disconnect_container(group); + QLIST_REMOVE(group, next); +@@ -1412,6 +1414,25 @@ int vfio_get_device(VFIOGroup *group, const char *name, + return ret; + } + ++ /* ++ * Clear the balloon inhibitor for this group if the driver knows the ++ * device operates compatibly with ballooning. Setting must be consistent ++ * per group, but since compatibility is really only possible with mdev ++ * currently, we expect singleton groups. ++ */ ++ if (vbasedev->balloon_allowed != group->balloon_allowed) { ++ if (!QLIST_EMPTY(&group->device_list)) { ++ error_setg(errp, ++ "Inconsistent device balloon setting within group"); ++ return -1; ++ } ++ ++ if (!group->balloon_allowed) { ++ group->balloon_allowed = true; ++ qemu_balloon_inhibit(false); ++ } ++ } ++ + vbasedev->fd = fd; + vbasedev->group = group; + QLIST_INSERT_HEAD(&group->device_list, vbasedev, next); +diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c +index 4683eb4..d43727f 100644 +--- a/hw/vfio/pci.c ++++ b/hw/vfio/pci.c +@@ -2803,12 +2803,13 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) + VFIOPCIDevice *vdev = DO_UPCAST(VFIOPCIDevice, pdev, pdev); + VFIODevice *vbasedev_iter; + VFIOGroup *group; +- char *tmp, group_path[PATH_MAX], *group_name; ++ char *tmp, *subsys, group_path[PATH_MAX], *group_name; + Error *err = NULL; + ssize_t len; + struct stat st; + int groupid; + int ret, i = 0; ++ bool is_mdev; + + QLIST_FOREACH(group, &vfio_group_list, next) { + QLIST_FOREACH(vbasedev_iter, &group->device_list, next) { +@@ -2880,6 +2881,27 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) + } + } + ++ /* ++ * Mediated devices *might* operate compatibly with memory ballooning, but ++ * we cannot know for certain, it depends on whether the mdev vendor driver ++ * stays in sync with the active working set of the guest driver. Prevent ++ * the x-balloon-allowed option unless this is minimally an mdev device. ++ */ ++ tmp = g_strdup_printf("%s/subsystem", vdev->vbasedev.sysfsdev); ++ subsys = realpath(tmp, NULL); ++ g_free(tmp); ++ is_mdev = (strcmp(subsys, "/sys/bus/mdev") == 0); ++ free(subsys); ++ ++ trace_vfio_mdev(vdev->vbasedev.name, is_mdev); ++ ++ if (vdev->vbasedev.balloon_allowed && !is_mdev) { ++ error_setg(errp, "x-balloon-allowed only potentially compatible " ++ "with mdev devices"); ++ vfio_put_group(group); ++ goto error; ++ } ++ + ret = vfio_get_device(group, vdev->vbasedev.name, &vdev->vbasedev, errp); + if (ret) { + vfio_put_group(group); +@@ -3177,6 +3199,8 @@ static Property vfio_pci_dev_properties[] = { + DEFINE_PROP_BIT("x-igd-opregion", VFIOPCIDevice, features, + VFIO_FEATURE_ENABLE_IGD_OPREGION_BIT, false), + DEFINE_PROP_BOOL("x-no-mmap", VFIOPCIDevice, vbasedev.no_mmap, false), ++ DEFINE_PROP_BOOL("x-balloon-allowed", VFIOPCIDevice, ++ vbasedev.balloon_allowed, false), + DEFINE_PROP_BOOL("x-no-kvm-intx", VFIOPCIDevice, no_kvm_intx, false), + DEFINE_PROP_BOOL("x-no-kvm-msi", VFIOPCIDevice, no_kvm_msi, false), + DEFINE_PROP_BOOL("x-no-kvm-msix", VFIOPCIDevice, no_kvm_msix, false), +diff --git a/hw/vfio/trace-events b/hw/vfio/trace-events +index 20109cb..9487887 100644 +--- a/hw/vfio/trace-events ++++ b/hw/vfio/trace-events +@@ -39,6 +39,7 @@ vfio_pci_hot_reset_result(const char *name, const char *result) "%s hot reset: % + vfio_populate_device_config(const char *name, unsigned long size, unsigned long offset, unsigned long flags) "Device %s config:\n size: 0x%lx, offset: 0x%lx, flags: 0x%lx" + vfio_populate_device_get_irq_info_failure(void) "VFIO_DEVICE_GET_IRQ_INFO failure: %m" + vfio_realize(const char *name, int group_id) " (%s) group %d" ++vfio_mdev(const char *name, bool is_mdev) " (%s) is_mdev %d" + vfio_add_ext_cap_dropped(const char *name, uint16_t cap, uint16_t offset) "%s 0x%x@0x%x" + vfio_pci_reset(const char *name) " (%s)" + vfio_pci_reset_flr(const char *name) "%s FLR/VFIO_DEVICE_RESET" +diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h +index f29df6e..36ee657 100644 +--- a/include/hw/vfio/vfio-common.h ++++ b/include/hw/vfio/vfio-common.h +@@ -123,6 +123,7 @@ typedef struct VFIODevice { + bool reset_works; + bool needs_reset; + bool no_mmap; ++ bool balloon_allowed; + VFIODeviceOps *ops; + unsigned int num_irqs; + unsigned int num_regions; +@@ -142,6 +143,7 @@ typedef struct VFIOGroup { + QLIST_HEAD(, VFIODevice) device_list; + QLIST_ENTRY(VFIOGroup) next; + QLIST_ENTRY(VFIOGroup) container_next; ++ bool balloon_allowed; + } VFIOGroup; + + typedef struct VFIODMABuf { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-helpers-Fix-qemu_vfio_open_pci-crash.patch b/SOURCES/kvm-vfio-helpers-Fix-qemu_vfio_open_pci-crash.patch new file mode 100644 index 0000000..bbb96ab --- /dev/null +++ b/SOURCES/kvm-vfio-helpers-Fix-qemu_vfio_open_pci-crash.patch @@ -0,0 +1,56 @@ +From 83b492d20071da90611d0ba5b028a80a49211eb3 Mon Sep 17 00:00:00 2001 +From: Markus Armbruster +Date: Mon, 3 Dec 2018 16:12:10 +0000 +Subject: [PATCH 09/16] vfio-helpers: Fix qemu_vfio_open_pci() crash + +RH-Author: Markus Armbruster +Message-id: <20181203161210.8577-2-armbru@redhat.com> +Patchwork-id: 83214 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] vfio-helpers: Fix qemu_vfio_open_pci() crash +Bugzilla: 1645840 +RH-Acked-by: Cornelia Huck +RH-Acked-by: John Snow +RH-Acked-by: Stefan Hajnoczi + +qemu_vfio_open_common() initializes s->lock only after passing s to +qemu_vfio_dma_map() via qemu_vfio_init_ramblock(). +qemu_vfio_dma_map() tries to lock the uninitialized lock and crashes. + +Fix by initializing s->lock first. + +RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=1645840 +Fixes: 418026ca43bc2626db092d7558258f9594366f28 +Cc: qemu-stable@nongnu.org +Signed-off-by: Markus Armbruster +Reviewed-by: Cornelia Huck +Reviewed-by: Stefan Hajnoczi +Message-id: 20181127084143.1113-1-armbru@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit 549b50a31d28f2687a47e827a1e17300784a2c44) +Signed-off-by: Danilo C. L. de Paula +--- + util/vfio-helpers.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c +index 006674c..c595faf 100644 +--- a/util/vfio-helpers.c ++++ b/util/vfio-helpers.c +@@ -411,13 +411,13 @@ static int qemu_vfio_init_ramblock(const char *block_name, void *host_addr, + + static void qemu_vfio_open_common(QEMUVFIOState *s) + { ++ qemu_mutex_init(&s->lock); + s->ram_notifier.ram_block_added = qemu_vfio_ram_block_added; + s->ram_notifier.ram_block_removed = qemu_vfio_ram_block_removed; + ram_block_notifier_add(&s->ram_notifier); + s->low_water_mark = QEMU_VFIO_IOVA_MIN; + s->high_water_mark = QEMU_VFIO_IOVA_MAX; + qemu_ram_foreach_block(qemu_vfio_init_ramblock, s); +- qemu_mutex_init(&s->lock); + } + + /** +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-pci-Default-display-option-to-off.patch b/SOURCES/kvm-vfio-pci-Default-display-option-to-off.patch new file mode 100644 index 0000000..ec637cc --- /dev/null +++ b/SOURCES/kvm-vfio-pci-Default-display-option-to-off.patch @@ -0,0 +1,50 @@ +From c766165206c5618768075d1d37204b2500ad2420 Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Tue, 12 Jun 2018 19:04:07 +0100 +Subject: [PATCH 01/15] vfio/pci: Default display option to "off" + +RH-Author: Alex Williamson +Message-id: <20180612190329.21103.32803.stgit@gimli.home> +Patchwork-id: 80646 +O-Subject: [virt212 qemu-kvm PATCH] vfio/pci: Default display option to "off" +Bugzilla: 1590511 +RH-Acked-by: Auger Eric +RH-Acked-by: Gerd Hoffmann +RH-Acked-by: Laszlo Ersek + +Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1590511 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=16695604 +Testing: RHEL-7.6 version tested by QE + +Commit a9994687cb9b ("vfio/display: core & wireup") added display +support to vfio-pci with the default being "auto", which breaks +existing VMs when the vGPU requires GL support but had no previous +requirement for a GL compatible configuration. "Off" is the safer +default as we impose no new requirements to VM configurations. + +Fixes: a9994687cb9b ("vfio/display: core & wireup") +Cc: qemu-stable@nongnu.org +Cc: Gerd Hoffmann +Signed-off-by: Alex Williamson +(cherry picked from commit 8151a9c56d31eeeea872b8103c8b86d03c411667) +Signed-off-by: Danilo C. L. de Paula +--- + hw/vfio/pci.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c +index 34b9d19..4683eb4 100644 +--- a/hw/vfio/pci.c ++++ b/hw/vfio/pci.c +@@ -3167,7 +3167,7 @@ static Property vfio_pci_dev_properties[] = { + DEFINE_PROP_PCI_HOST_DEVADDR("host", VFIOPCIDevice, host), + DEFINE_PROP_STRING("sysfsdev", VFIOPCIDevice, vbasedev.sysfsdev), + DEFINE_PROP_ON_OFF_AUTO("display", VFIOPCIDevice, +- display, ON_OFF_AUTO_AUTO), ++ display, ON_OFF_AUTO_OFF), + DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIOPCIDevice, + intx.mmap_timeout, 1100), + DEFINE_PROP_BIT("x-vga", VFIOPCIDevice, features, +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-pci-Fix-failure-to-close-file-descriptor-on-err.patch b/SOURCES/kvm-vfio-pci-Fix-failure-to-close-file-descriptor-on-err.patch new file mode 100644 index 0000000..33e282a --- /dev/null +++ b/SOURCES/kvm-vfio-pci-Fix-failure-to-close-file-descriptor-on-err.patch @@ -0,0 +1,44 @@ +From 25e634bffca2048bc0df66ff00a9c46f00d1c3c2 Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:02:16 +0000 +Subject: [PATCH 15/16] vfio/pci: Fix failure to close file descriptor on error + +RH-Author: Alex Williamson +Message-id: <154387453688.27651.6584558037969560928.stgit@gimli.home> +Patchwork-id: 83240 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 6/7] vfio/pci: Fix failure to close file descriptor on error +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +A new error path fails to close the device file descriptor when +triggered by a ballooning incompatibility within the group. Fix it. + +Fixes: 238e91728503 ("vfio/ccw/pci: Allow devices to opt-in for ballooning") +Reviewed-by: Peter Xu +Signed-off-by: Alex Williamson +(cherry picked from commit 8709b3954d4161bad30ccc435408ec50e10f53cc) +Signed-off-by: Danilo C. L. de Paula +--- + hw/vfio/common.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/hw/vfio/common.c b/hw/vfio/common.c +index cda2d1f..3ab92bd 100644 +--- a/hw/vfio/common.c ++++ b/hw/vfio/common.c +@@ -1424,6 +1424,7 @@ int vfio_get_device(VFIOGroup *group, const char *name, + if (!QLIST_EMPTY(&group->device_list)) { + error_setg(errp, + "Inconsistent device balloon setting within group"); ++ close(fd); + return -1; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-pci-Handle-subsystem-realpath-returning-NULL.patch b/SOURCES/kvm-vfio-pci-Handle-subsystem-realpath-returning-NULL.patch new file mode 100644 index 0000000..20cb204 --- /dev/null +++ b/SOURCES/kvm-vfio-pci-Handle-subsystem-realpath-returning-NULL.patch @@ -0,0 +1,46 @@ +From 7a807e50f62f045b13a5b6e28ee02e36e42ad201 Mon Sep 17 00:00:00 2001 +From: Alex Williamson +Date: Mon, 3 Dec 2018 22:02:00 +0000 +Subject: [PATCH 14/16] vfio/pci: Handle subsystem realpath() returning NULL + +RH-Author: Alex Williamson +Message-id: <154387452062.27651.8506633913988228901.stgit@gimli.home> +Patchwork-id: 83239 +O-Subject: [RHEL-8.0 qemu-kvm PATCH 5/7] vfio/pci: Handle subsystem realpath() returning NULL +Bugzilla: 1650272 +RH-Acked-by: Peter Xu +RH-Acked-by: Auger Eric +RH-Acked-by: Cornelia Huck +RH-Acked-by: David Hildenbrand + +Bugzilla: 1650272 + +Fix error reported by Coverity where realpath can return NULL, +resulting in a segfault in strcmp(). This should never happen given +that we're working through regularly structured sysfs paths, but +trivial enough to easily avoid. + +Fixes: 238e91728503 ("vfio/ccw/pci: Allow devices to opt-in for ballooning") +Signed-off-by: Alex Williamson +(cherry picked from commit a1c0f886496cfb4c336f8eb4155ed424567d653e) +Signed-off-by: Danilo C. L. de Paula +--- + hw/vfio/pci.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c +index d43727f..b463661 100644 +--- a/hw/vfio/pci.c ++++ b/hw/vfio/pci.c +@@ -2890,7 +2890,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) + tmp = g_strdup_printf("%s/subsystem", vdev->vbasedev.sysfsdev); + subsys = realpath(tmp, NULL); + g_free(tmp); +- is_mdev = (strcmp(subsys, "/sys/bus/mdev") == 0); ++ is_mdev = subsys && (strcmp(subsys, "/sys/bus/mdev") == 0); + free(subsys); + + trace_vfio_mdev(vdev->vbasedev.name, is_mdev); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vfio-pci-do-not-set-the-PCIDevice-has_rom-attribute.patch b/SOURCES/kvm-vfio-pci-do-not-set-the-PCIDevice-has_rom-attribute.patch new file mode 100644 index 0000000..7350f50 --- /dev/null +++ b/SOURCES/kvm-vfio-pci-do-not-set-the-PCIDevice-has_rom-attribute.patch @@ -0,0 +1,67 @@ +From 85d3fdbf408b0d96ff27c38d09770685c6bac4ec Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Mon, 7 Jan 2019 17:02:15 +0000 +Subject: [PATCH 14/22] vfio/pci: do not set the PCIDevice 'has_rom' attribute +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: plai@redhat.com +Message-id: <1546880543-24860-3-git-send-email-plai@redhat.com> +Patchwork-id: 83888 +O-Subject: [RHEL8.0 qemu-kvm PATCH v7 02/10] vfio/pci: do not set the PCIDevice 'has_rom' attribute +Bugzilla: 1539285 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Eduardo Habkost + +From: Cédric Le Goater + +PCI devices needing a ROM allocate an optional MemoryRegion with +pci_add_option_rom(). pci_del_option_rom() does the cleanup when the +device is destroyed. The only action taken by this routine is to call +vmstate_unregister_ram() which clears the id string of the optional +ROM RAMBlock and now, also flags the RAMBlock as non-migratable. This +was recently added by commit b895de502717 ("migration: discard +non-migratable RAMBlocks"), . + +VFIO devices do their own loading of the PCI option ROM in +vfio_pci_size_rom(). The memory region is switched to an I/O region +and the PCI attribute 'has_rom' is set but the RAMBlock of the ROM +region is not allocated. When the associated PCI device is deleted, +pci_del_option_rom() calls vmstate_unregister_ram() which tries to +flag a NULL RAMBlock, leading to a SEGV. + +It seems that 'has_rom' was set to have memory_region_destroy() +called, but since commit 469b046ead06 ("memory: remove +memory_region_destroy") this is not necessary anymore as the +MemoryRegion is freed automagically. + +Remove the PCIDevice 'has_rom' attribute setting in vfio. + +Fixes: b895de502717 ("migration: discard non-migratable RAMBlocks") +Signed-off-by: Cédric Le Goater +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Alex Williamson +(cherry picked from commit 26c0ae56386edacc8b0da40264748f59afedb1bb) +Signed-off-by: Paul Lai +Signed-off-by: Danilo C. L. de Paula +--- + hw/vfio/pci.c | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c +index b463661..ba3a393 100644 +--- a/hw/vfio/pci.c ++++ b/hw/vfio/pci.c +@@ -990,7 +990,6 @@ static void vfio_pci_size_rom(VFIOPCIDevice *vdev) + pci_register_bar(&vdev->pdev, PCI_ROM_SLOT, + PCI_BASE_ADDRESS_SPACE_MEMORY, &vdev->pdev.rom); + +- vdev->pdev.has_rom = true; + vdev->rom_read_failed = false; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vga-catch-depth-0.patch b/SOURCES/kvm-vga-catch-depth-0.patch new file mode 100644 index 0000000..91c5e07 --- /dev/null +++ b/SOURCES/kvm-vga-catch-depth-0.patch @@ -0,0 +1,82 @@ +From 9fbf6794bed827af4d1248b25d175f014a47201d Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Tue, 29 May 2018 10:57:04 +0200 +Subject: [PATCH 004/268] vga: catch depth 0 + +RH-Author: Gerd Hoffmann +Message-id: <20180529105704.21419-2-kraxel@redhat.com> +Patchwork-id: 80500 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] vga: catch depth 0 +Bugzilla: 1575541 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Laszlo Ersek +RH-Acked-by: Stefan Hajnoczi + +depth == 0 is used to indicate 256 color modes. Our region calculation +goes wrong in that case. So detect that and just take the safe code +path we already have for the wraparound case. + +While being at it also catch depth == 15 (where our region size +calculation goes wrong too). And make the comment more verbose, +explaining what is going on here. + +Without this windows guest install might trigger an assert due to trying +to check dirty bitmap outside the snapshot region. + +Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1575541 +Signed-off-by: Gerd Hoffmann +Message-id: 20180514103117.21059-1-kraxel@redhat.com +(cherry picked from commit a89fe6c329799e47aaa1663650f076b28808e186) +Signed-off-by: Miroslav Rezanina +--- + hw/display/vga.c | 23 ++++++++++++++++++----- + 1 file changed, 18 insertions(+), 5 deletions(-) + +diff --git a/hw/display/vga.c b/hw/display/vga.c +index 7218133..a7794f6 100644 +--- a/hw/display/vga.c ++++ b/hw/display/vga.c +@@ -1480,13 +1480,28 @@ static void vga_draw_graphic(VGACommonState *s, int full_update) + + s->get_resolution(s, &width, &height); + disp_width = width; ++ depth = s->get_bpp(s); + + region_start = (s->start_addr * 4); + region_end = region_start + (ram_addr_t)s->line_offset * height; +- region_end += width * s->get_bpp(s) / 8; /* scanline length */ ++ region_end += width * depth / 8; /* scanline length */ + region_end -= s->line_offset; +- if (region_end > s->vbe_size) { +- /* wraps around (can happen with cirrus vbe modes) */ ++ if (region_end > s->vbe_size || depth == 0 || depth == 15) { ++ /* ++ * We land here on: ++ * - wraps around (can happen with cirrus vbe modes) ++ * - depth == 0 (256 color palette video mode) ++ * - depth == 15 ++ * ++ * Take the safe and slow route: ++ * - create a dirty bitmap snapshot for all vga memory. ++ * - force shadowing (so all vga memory access goes ++ * through vga_read_*() helpers). ++ * ++ * Given this affects only vga features which are pretty much ++ * unused by modern guests there should be no performance ++ * impact. ++ */ + region_start = 0; + region_end = s->vbe_size; + force_shadow = true; +@@ -1520,8 +1535,6 @@ static void vga_draw_graphic(VGACommonState *s, int full_update) + } + } + +- depth = s->get_bpp(s); +- + /* + * Check whether we can share the surface with the backend + * or whether we need a shadow surface. We share native +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhdx-Fix-vhdx_co_create-return-value.patch b/SOURCES/kvm-vhdx-Fix-vhdx_co_create-return-value.patch new file mode 100644 index 0000000..5ca6035 --- /dev/null +++ b/SOURCES/kvm-vhdx-Fix-vhdx_co_create-return-value.patch @@ -0,0 +1,43 @@ +From 1db9da574c842c61e80523d9bdb8bf258a8bc883 Mon Sep 17 00:00:00 2001 +From: Kevin Wolf +Date: Tue, 26 Jun 2018 09:48:38 +0200 +Subject: [PATCH 130/268] vhdx: Fix vhdx_co_create() return value + +RH-Author: Kevin Wolf +Message-id: <20180626094856.6924-56-kwolf@redhat.com> +Patchwork-id: 81127 +O-Subject: [RHV-7.6 qemu-kvm-rhev PATCH v2 55/73] vhdx: Fix vhdx_co_create() return value +Bugzilla: 1513543 +RH-Acked-by: Jeffrey Cody +RH-Acked-by: Max Reitz +RH-Acked-by: Fam Zheng + +.bdrv_co_create() is supposed to return 0 on success, but vhdx could +return a positive value instead. Fix this. + +Signed-off-by: Kevin Wolf +Reviewed-by: Max Reitz +Reviewed-by: Jeff Cody +(cherry picked from commit 4a5f2779bad769184550869931937acd0707ec3b) +Signed-off-by: Kevin Wolf +Signed-off-by: Miroslav Rezanina +--- + block/vhdx.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/block/vhdx.c b/block/vhdx.c +index 6731298..31ac53b 100644 +--- a/block/vhdx.c ++++ b/block/vhdx.c +@@ -1950,7 +1950,7 @@ static int coroutine_fn vhdx_co_create(BlockdevCreateOptions *opts, + goto delete_and_exit; + } + +- ++ ret = 0; + delete_and_exit: + blk_unref(blk); + bdrv_unref(bs); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-allow-backends-to-filter-memory-sections.patch b/SOURCES/kvm-vhost-allow-backends-to-filter-memory-sections.patch new file mode 100644 index 0000000..5985a82 --- /dev/null +++ b/SOURCES/kvm-vhost-allow-backends-to-filter-memory-sections.patch @@ -0,0 +1,117 @@ +From 1a875754c500068de9c210b9f57621843c78e8b3 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:41 +0200 +Subject: [PATCH 162/268] vhost: allow backends to filter memory sections + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-7-git-send-email-plai@redhat.com> +Patchwork-id: 80941 +O-Subject: [RHEL7.6 PATCH BZ 1526645 06/10] vhost: allow backends to filter memory sections +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +This patch introduces a vhost op for vhost backends to allow +them to filter the memory sections that they can handle. + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 988a27754bbbc45698f7acb54352e5a1ae699514) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + hw/virtio/vhost-user.c | 11 +++++++++++ + hw/virtio/vhost.c | 9 +++++++-- + include/hw/virtio/vhost-backend.h | 4 ++++ + 3 files changed, 22 insertions(+), 2 deletions(-) + +diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c +index 85d8fd2..ebb946a 100644 +--- a/hw/virtio/vhost-user.c ++++ b/hw/virtio/vhost-user.c +@@ -1620,6 +1620,16 @@ vhost_user_crypto_close_session(struct vhost_dev *dev, uint64_t session_id) + return 0; + } + ++static bool vhost_user_mem_section_filter(struct vhost_dev *dev, ++ MemoryRegionSection *section) ++{ ++ bool result; ++ ++ result = memory_region_get_fd(section->mr) >= 0; ++ ++ return result; ++} ++ + const VhostOps user_ops = { + .backend_type = VHOST_BACKEND_TYPE_USER, + .vhost_backend_init = vhost_user_init, +@@ -1650,4 +1660,5 @@ const VhostOps user_ops = { + .vhost_set_config = vhost_user_set_config, + .vhost_crypto_create_session = vhost_user_crypto_create_session, + .vhost_crypto_close_session = vhost_user_crypto_close_session, ++ .vhost_backend_mem_section_filter = vhost_user_mem_section_filter, + }; +diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c +index 9d5850a..1ae68ff 100644 +--- a/hw/virtio/vhost.c ++++ b/hw/virtio/vhost.c +@@ -386,7 +386,7 @@ static int vhost_verify_ring_mappings(struct vhost_dev *dev, + return r; + } + +-static bool vhost_section(MemoryRegionSection *section) ++static bool vhost_section(struct vhost_dev *dev, MemoryRegionSection *section) + { + bool result; + bool log_dirty = memory_region_get_dirty_log_mask(section->mr) & +@@ -399,6 +399,11 @@ static bool vhost_section(MemoryRegionSection *section) + */ + result &= !log_dirty; + ++ if (result && dev->vhost_ops->vhost_backend_mem_section_filter) { ++ result &= ++ dev->vhost_ops->vhost_backend_mem_section_filter(dev, section); ++ } ++ + trace_vhost_section(section->mr->name, result); + return result; + } +@@ -632,7 +637,7 @@ static void vhost_region_addnop(MemoryListener *listener, + struct vhost_dev *dev = container_of(listener, struct vhost_dev, + memory_listener); + +- if (!vhost_section(section)) { ++ if (!vhost_section(dev, section)) { + return; + } + vhost_region_add_section(dev, section); +diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h +index 5dac61f..81283ec 100644 +--- a/include/hw/virtio/vhost-backend.h ++++ b/include/hw/virtio/vhost-backend.h +@@ -101,6 +101,9 @@ typedef int (*vhost_crypto_create_session_op)(struct vhost_dev *dev, + typedef int (*vhost_crypto_close_session_op)(struct vhost_dev *dev, + uint64_t session_id); + ++typedef bool (*vhost_backend_mem_section_filter_op)(struct vhost_dev *dev, ++ MemoryRegionSection *section); ++ + typedef struct VhostOps { + VhostBackendType backend_type; + vhost_backend_init vhost_backend_init; +@@ -138,6 +141,7 @@ typedef struct VhostOps { + vhost_set_config_op vhost_set_config; + vhost_crypto_create_session_op vhost_crypto_create_session; + vhost_crypto_close_session_op vhost_crypto_close_session; ++ vhost_backend_mem_section_filter_op vhost_backend_mem_section_filter; + } VhostOps; + + extern const VhostOps user_ops; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-user-add-Net-prefix-to-internal-state-structur.patch b/SOURCES/kvm-vhost-user-add-Net-prefix-to-internal-state-structur.patch new file mode 100644 index 0000000..53c6dbd --- /dev/null +++ b/SOURCES/kvm-vhost-user-add-Net-prefix-to-internal-state-structur.patch @@ -0,0 +1,201 @@ +From b9f8e65b1304d2d22ddadb2389a285225d2cf3f2 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:36 +0200 +Subject: [PATCH 157/268] vhost-user: add Net prefix to internal state + structure + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-2-git-send-email-plai@redhat.com> +Patchwork-id: 80943 +O-Subject: [RHEL7.6 PATCH BZ 1526645 01/10] vhost-user: add Net prefix to internal state structure +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +We are going to introduce a shared vhost user state which +will be named as 'VhostUserState'. So add 'Net' prefix to +the existing internal state structure in the vhost-user +netdev to avoid conflict. + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 703878e2e0975123b1a89a006c0204c469333278) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + net/vhost-user.c | 38 +++++++++++++++++++------------------- + 1 file changed, 19 insertions(+), 19 deletions(-) + +diff --git a/net/vhost-user.c b/net/vhost-user.c +index e0f16c8..fa28aad 100644 +--- a/net/vhost-user.c ++++ b/net/vhost-user.c +@@ -20,38 +20,38 @@ + #include "qemu/option.h" + #include "trace.h" + +-typedef struct VhostUserState { ++typedef struct NetVhostUserState { + NetClientState nc; + CharBackend chr; /* only queue index 0 */ + VHostNetState *vhost_net; + guint watch; + uint64_t acked_features; + bool started; +-} VhostUserState; ++} NetVhostUserState; + + VHostNetState *vhost_user_get_vhost_net(NetClientState *nc) + { +- VhostUserState *s = DO_UPCAST(VhostUserState, nc, nc); ++ NetVhostUserState *s = DO_UPCAST(NetVhostUserState, nc, nc); + assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_USER); + return s->vhost_net; + } + + uint64_t vhost_user_get_acked_features(NetClientState *nc) + { +- VhostUserState *s = DO_UPCAST(VhostUserState, nc, nc); ++ NetVhostUserState *s = DO_UPCAST(NetVhostUserState, nc, nc); + assert(nc->info->type == NET_CLIENT_DRIVER_VHOST_USER); + return s->acked_features; + } + + static void vhost_user_stop(int queues, NetClientState *ncs[]) + { +- VhostUserState *s; ++ NetVhostUserState *s; + int i; + + for (i = 0; i < queues; i++) { + assert(ncs[i]->info->type == NET_CLIENT_DRIVER_VHOST_USER); + +- s = DO_UPCAST(VhostUserState, nc, ncs[i]); ++ s = DO_UPCAST(NetVhostUserState, nc, ncs[i]); + + if (s->vhost_net) { + /* save acked features */ +@@ -68,7 +68,7 @@ static int vhost_user_start(int queues, NetClientState *ncs[], CharBackend *be) + { + VhostNetOptions options; + struct vhost_net *net = NULL; +- VhostUserState *s; ++ NetVhostUserState *s; + int max_queues; + int i; + +@@ -77,7 +77,7 @@ static int vhost_user_start(int queues, NetClientState *ncs[], CharBackend *be) + for (i = 0; i < queues; i++) { + assert(ncs[i]->info->type == NET_CLIENT_DRIVER_VHOST_USER); + +- s = DO_UPCAST(VhostUserState, nc, ncs[i]); ++ s = DO_UPCAST(NetVhostUserState, nc, ncs[i]); + + options.net_backend = ncs[i]; + options.opaque = be; +@@ -123,7 +123,7 @@ static ssize_t vhost_user_receive(NetClientState *nc, const uint8_t *buf, + without GUEST_ANNOUNCE capability. + */ + if (size == 60) { +- VhostUserState *s = DO_UPCAST(VhostUserState, nc, nc); ++ NetVhostUserState *s = DO_UPCAST(NetVhostUserState, nc, nc); + int r; + static int display_rarp_failure = 1; + char mac_addr[6]; +@@ -146,7 +146,7 @@ static ssize_t vhost_user_receive(NetClientState *nc, const uint8_t *buf, + + static void vhost_user_cleanup(NetClientState *nc) + { +- VhostUserState *s = DO_UPCAST(VhostUserState, nc, nc); ++ NetVhostUserState *s = DO_UPCAST(NetVhostUserState, nc, nc); + + if (s->vhost_net) { + vhost_net_cleanup(s->vhost_net); +@@ -180,7 +180,7 @@ static bool vhost_user_has_ufo(NetClientState *nc) + + static NetClientInfo net_vhost_user_info = { + .type = NET_CLIENT_DRIVER_VHOST_USER, +- .size = sizeof(VhostUserState), ++ .size = sizeof(NetVhostUserState), + .receive = vhost_user_receive, + .cleanup = vhost_user_cleanup, + .has_vnet_hdr = vhost_user_has_vnet_hdr, +@@ -190,7 +190,7 @@ static NetClientInfo net_vhost_user_info = { + static gboolean net_vhost_user_watch(GIOChannel *chan, GIOCondition cond, + void *opaque) + { +- VhostUserState *s = opaque; ++ NetVhostUserState *s = opaque; + + qemu_chr_fe_disconnect(&s->chr); + +@@ -203,7 +203,7 @@ static void chr_closed_bh(void *opaque) + { + const char *name = opaque; + NetClientState *ncs[MAX_QUEUE_NUM]; +- VhostUserState *s; ++ NetVhostUserState *s; + Error *err = NULL; + int queues; + +@@ -212,7 +212,7 @@ static void chr_closed_bh(void *opaque) + MAX_QUEUE_NUM); + assert(queues < MAX_QUEUE_NUM); + +- s = DO_UPCAST(VhostUserState, nc, ncs[0]); ++ s = DO_UPCAST(NetVhostUserState, nc, ncs[0]); + + qmp_set_link(name, false, &err); + vhost_user_stop(queues, ncs); +@@ -229,7 +229,7 @@ static void net_vhost_user_event(void *opaque, int event) + { + const char *name = opaque; + NetClientState *ncs[MAX_QUEUE_NUM]; +- VhostUserState *s; ++ NetVhostUserState *s; + Chardev *chr; + Error *err = NULL; + int queues; +@@ -239,7 +239,7 @@ static void net_vhost_user_event(void *opaque, int event) + MAX_QUEUE_NUM); + assert(queues < MAX_QUEUE_NUM); + +- s = DO_UPCAST(VhostUserState, nc, ncs[0]); ++ s = DO_UPCAST(NetVhostUserState, nc, ncs[0]); + chr = qemu_chr_fe_get_driver(&s->chr); + trace_vhost_user_event(chr->label, event); + switch (event) { +@@ -283,7 +283,7 @@ static int net_vhost_user_init(NetClientState *peer, const char *device, + { + Error *err = NULL; + NetClientState *nc, *nc0 = NULL; +- VhostUserState *s; ++ NetVhostUserState *s; + int i; + + assert(name); +@@ -296,7 +296,7 @@ static int net_vhost_user_init(NetClientState *peer, const char *device, + nc->queue_index = i; + if (!nc0) { + nc0 = nc; +- s = DO_UPCAST(VhostUserState, nc, nc); ++ s = DO_UPCAST(NetVhostUserState, nc, nc); + if (!qemu_chr_fe_init(&s->chr, chr, &err)) { + error_report_err(err); + return -1; +@@ -305,7 +305,7 @@ static int net_vhost_user_init(NetClientState *peer, const char *device, + + } + +- s = DO_UPCAST(VhostUserState, nc, nc0); ++ s = DO_UPCAST(NetVhostUserState, nc, nc0); + do { + if (qemu_chr_fe_wait_connected(&s->chr, &err) < 0) { + error_report_err(err); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-user-allow-slave-to-send-fds-via-slave-channel.patch b/SOURCES/kvm-vhost-user-allow-slave-to-send-fds-via-slave-channel.patch new file mode 100644 index 0000000..95095bc --- /dev/null +++ b/SOURCES/kvm-vhost-user-allow-slave-to-send-fds-via-slave-channel.patch @@ -0,0 +1,144 @@ +From 6fceacad0e3d27cec6cef4e105c1dabacccb875c Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:42 +0200 +Subject: [PATCH 163/268] vhost-user: allow slave to send fds via slave channel + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-8-git-send-email-plai@redhat.com> +Patchwork-id: 80935 +O-Subject: [RHEL7.6 PATCH BZ 1526645 07/10] vhost-user: allow slave to send fds via slave channel +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +Introduce VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD protocol +feature to allow slave to send at most 8 descriptors +in each message to master via ancillary data using the +slave channel. + +Suggested-by: Michael S. Tsirkin +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 5f57fbeaaf7c4cd33152d7f2e449caab4d4209d9) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + docs/interop/vhost-user.txt | 5 +++++ + hw/virtio/vhost-user.c | 27 +++++++++++++++++---------- + 2 files changed, 22 insertions(+), 10 deletions(-) + +diff --git a/docs/interop/vhost-user.txt b/docs/interop/vhost-user.txt +index 534caab..682a683 100644 +--- a/docs/interop/vhost-user.txt ++++ b/docs/interop/vhost-user.txt +@@ -367,6 +367,10 @@ The fd is provided via VHOST_USER_SET_SLAVE_REQ_FD ancillary data. + A slave may then send VHOST_USER_SLAVE_* messages to the master + using this fd communication channel. + ++If VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD protocol feature is negotiated, ++slave can send file descriptors (at most 8 descriptors in each message) ++to master via ancillary data using this fd communication channel. ++ + Protocol features + ----------------- + +@@ -380,6 +384,7 @@ Protocol features + #define VHOST_USER_PROTOCOL_F_CRYPTO_SESSION 7 + #define VHOST_USER_PROTOCOL_F_PAGEFAULT 8 + #define VHOST_USER_PROTOCOL_F_CONFIG 9 ++#define VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD 10 + + Master message types + -------------------- +diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c +index ebb946a..e8027ad 100644 +--- a/hw/virtio/vhost-user.c ++++ b/hw/virtio/vhost-user.c +@@ -30,6 +30,7 @@ + + #define VHOST_MEMORY_MAX_NREGIONS 8 + #define VHOST_USER_F_PROTOCOL_FEATURES 30 ++#define VHOST_USER_SLAVE_MAX_FDS 8 + + /* + * Maximum size of virtio device config space +@@ -47,6 +48,7 @@ enum VhostUserProtocolFeature { + VHOST_USER_PROTOCOL_F_CRYPTO_SESSION = 7, + VHOST_USER_PROTOCOL_F_PAGEFAULT = 8, + VHOST_USER_PROTOCOL_F_CONFIG = 9, ++ VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD = 10, + VHOST_USER_PROTOCOL_F_MAX + }; + +@@ -854,10 +856,10 @@ static void slave_read(void *opaque) + int size, ret = 0; + struct iovec iov; + struct msghdr msgh; +- int fd = -1; ++ int fd[VHOST_USER_SLAVE_MAX_FDS]; + char control[CMSG_SPACE(sizeof(fd))]; + struct cmsghdr *cmsg; +- size_t fdsize; ++ int i, fdsize = 0; + + memset(&msgh, 0, sizeof(msgh)); + msgh.msg_iov = &iov; +@@ -865,6 +867,8 @@ static void slave_read(void *opaque) + msgh.msg_control = control; + msgh.msg_controllen = sizeof(control); + ++ memset(fd, -1, sizeof(fd)); ++ + /* Read header */ + iov.iov_base = &hdr; + iov.iov_len = VHOST_USER_HDR_SIZE; +@@ -885,7 +889,7 @@ static void slave_read(void *opaque) + if (cmsg->cmsg_level == SOL_SOCKET && + cmsg->cmsg_type == SCM_RIGHTS) { + fdsize = cmsg->cmsg_len - CMSG_LEN(0); +- memcpy(&fd, CMSG_DATA(cmsg), fdsize); ++ memcpy(fd, CMSG_DATA(cmsg), fdsize); + break; + } + } +@@ -913,14 +917,15 @@ static void slave_read(void *opaque) + break; + default: + error_report("Received unexpected msg type."); +- if (fd != -1) { +- close(fd); +- } + ret = -EINVAL; + } + +- /* Message handlers need to make sure that fd will be consumed. */ +- fd = -1; ++ /* Close the remaining file descriptors. */ ++ for (i = 0; i < fdsize; i++) { ++ if (fd[i] != -1) { ++ close(fd[i]); ++ } ++ } + + /* + * REPLY_ACK feature handling. Other reply types has to be managed +@@ -954,8 +959,10 @@ err: + qemu_set_fd_handler(u->slave_fd, NULL, NULL, NULL); + close(u->slave_fd); + u->slave_fd = -1; +- if (fd != -1) { +- close(fd); ++ for (i = 0; i < fdsize; i++) { ++ if (fd[i] != -1) { ++ close(fd[i]); ++ } + } + return; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-user-bridge-support-host-notifier.patch b/SOURCES/kvm-vhost-user-bridge-support-host-notifier.patch new file mode 100644 index 0000000..013cb13 --- /dev/null +++ b/SOURCES/kvm-vhost-user-bridge-support-host-notifier.patch @@ -0,0 +1,214 @@ +From d32439f6eb5f0d628c1a9d559007dda3083ec7b8 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:40 +0200 +Subject: [PATCH 161/268] vhost-user-bridge: support host notifier + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-6-git-send-email-plai@redhat.com> +Patchwork-id: 80934 +O-Subject: [RHEL7.6 PATCH BZ 1526645 05/10] vhost-user-bridge: support host notifier +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +This patch introduces the host notifier support in +vhost-user-bridge. A new option (-H) is added to use +the host notifier. This is mainly used to test the +host notifier implementation in vhost user. + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit e3af2928f8270498097af163bb023b633c0d8d1c) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + tests/vhost-user-bridge.c | 98 +++++++++++++++++++++++++++++++++++++++++++++-- + 1 file changed, 95 insertions(+), 3 deletions(-) + +diff --git a/tests/vhost-user-bridge.c b/tests/vhost-user-bridge.c +index e0605a5..0884294 100644 +--- a/tests/vhost-user-bridge.c ++++ b/tests/vhost-user-bridge.c +@@ -29,6 +29,7 @@ + + #define _FILE_OFFSET_BITS 64 + ++#include "qemu/atomic.h" + #include "qemu/osdep.h" + #include "qemu/iov.h" + #include "standard-headers/linux/virtio_net.h" +@@ -65,6 +66,11 @@ typedef struct VubrDev { + int sock; + int ready; + int quit; ++ struct { ++ int fd; ++ void *addr; ++ pthread_t thread; ++ } notifier; + } VubrDev; + + static void +@@ -445,14 +451,22 @@ static uint64_t + vubr_get_features(VuDev *dev) + { + return 1ULL << VIRTIO_NET_F_GUEST_ANNOUNCE | +- 1ULL << VIRTIO_NET_F_MRG_RXBUF; ++ 1ULL << VIRTIO_NET_F_MRG_RXBUF | ++ 1ULL << VIRTIO_F_VERSION_1; + } + + static void + vubr_queue_set_started(VuDev *dev, int qidx, bool started) + { ++ VubrDev *vubr = container_of(dev, VubrDev, vudev); + VuVirtq *vq = vu_get_queue(dev, qidx); + ++ if (started && vubr->notifier.fd >= 0) { ++ vu_set_queue_host_notifier(dev, vq, vubr->notifier.fd, ++ getpagesize(), ++ qidx * getpagesize()); ++ } ++ + if (qidx % 2 == 1) { + vu_set_queue_handler(dev, vq, started ? vubr_handle_tx : NULL); + } +@@ -522,6 +536,8 @@ vubr_new(const char *path, bool client) + vubr_die("socket"); + } + ++ dev->notifier.fd = -1; ++ + un.sun_family = AF_UNIX; + strcpy(un.sun_path, path); + len = sizeof(un.sun_family) + strlen(path); +@@ -559,6 +575,73 @@ vubr_new(const char *path, bool client) + return dev; + } + ++static void *notifier_thread(void *arg) ++{ ++ VuDev *dev = (VuDev *)arg; ++ VubrDev *vubr = container_of(dev, VubrDev, vudev); ++ int pagesize = getpagesize(); ++ int qidx; ++ ++ while (true) { ++ for (qidx = 0; qidx < VHOST_MAX_NR_VIRTQUEUE; qidx++) { ++ uint16_t *n = vubr->notifier.addr + pagesize * qidx; ++ ++ if (*n == qidx) { ++ *n = 0xffff; ++ /* We won't miss notifications if we reset ++ * the memory first. */ ++ smp_mb(); ++ ++ DPRINT("Got a notification for queue%d via host notifier.\n", ++ qidx); ++ ++ if (qidx % 2 == 1) { ++ vubr_handle_tx(dev, qidx); ++ } ++ } ++ usleep(1000); ++ } ++ } ++ ++ return NULL; ++} ++ ++static void ++vubr_host_notifier_setup(VubrDev *dev) ++{ ++ char template[] = "/tmp/vubr-XXXXXX"; ++ pthread_t thread; ++ size_t length; ++ void *addr; ++ int fd; ++ ++ length = getpagesize() * VHOST_MAX_NR_VIRTQUEUE; ++ ++ fd = mkstemp(template); ++ if (fd < 0) { ++ vubr_die("mkstemp()"); ++ } ++ ++ if (posix_fallocate(fd, 0, length) != 0) { ++ vubr_die("posix_fallocate()"); ++ } ++ ++ addr = mmap(NULL, length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); ++ if (addr == MAP_FAILED) { ++ vubr_die("mmap()"); ++ } ++ ++ memset(addr, 0xff, length); ++ ++ if (pthread_create(&thread, NULL, notifier_thread, &dev->vudev) != 0) { ++ vubr_die("pthread_create()"); ++ } ++ ++ dev->notifier.fd = fd; ++ dev->notifier.addr = addr; ++ dev->notifier.thread = thread; ++} ++ + static void + vubr_set_host(struct sockaddr_in *saddr, const char *host) + { +@@ -673,8 +756,9 @@ main(int argc, char *argv[]) + VubrDev *dev; + int opt; + bool client = false; ++ bool host_notifier = false; + +- while ((opt = getopt(argc, argv, "l:r:u:c")) != -1) { ++ while ((opt = getopt(argc, argv, "l:r:u:cH")) != -1) { + + switch (opt) { + case 'l': +@@ -693,6 +777,9 @@ main(int argc, char *argv[]) + case 'c': + client = true; + break; ++ case 'H': ++ host_notifier = true; ++ break; + default: + goto out; + } +@@ -708,6 +795,10 @@ main(int argc, char *argv[]) + return 1; + } + ++ if (host_notifier) { ++ vubr_host_notifier_setup(dev); ++ } ++ + vubr_backend_udp_setup(dev, lhost, lport, rhost, rport); + vubr_run(dev); + +@@ -717,7 +808,7 @@ main(int argc, char *argv[]) + + out: + fprintf(stderr, "Usage: %s ", argv[0]); +- fprintf(stderr, "[-c] [-u ud_socket_path] [-l lhost:lport] [-r rhost:rport]\n"); ++ fprintf(stderr, "[-c] [-H] [-u ud_socket_path] [-l lhost:lport] [-r rhost:rport]\n"); + fprintf(stderr, "\t-u path to unix doman socket. default: %s\n", + DEFAULT_UD_SOCKET); + fprintf(stderr, "\t-l local host and port. default: %s:%s\n", +@@ -725,6 +816,7 @@ out: + fprintf(stderr, "\t-r remote host and port. default: %s:%s\n", + DEFAULT_RHOST, DEFAULT_RPORT); + fprintf(stderr, "\t-c client mode\n"); ++ fprintf(stderr, "\t-H use host notifier\n"); + + return 1; + } +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-user-introduce-shared-vhost-user-state.patch b/SOURCES/kvm-vhost-user-introduce-shared-vhost-user-state.patch new file mode 100644 index 0000000..8a5febe --- /dev/null +++ b/SOURCES/kvm-vhost-user-introduce-shared-vhost-user-state.patch @@ -0,0 +1,560 @@ +From 080469ac5a5f9b0ab1d3001001b9939425023f93 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:43 +0200 +Subject: [PATCH 164/268] vhost-user: introduce shared vhost-user state + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-9-git-send-email-plai@redhat.com> +Patchwork-id: 80942 +O-Subject: [RHEL7.6 PATCH BZ 1526645 08/10] vhost-user: introduce shared vhost-user state +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +When multi queue is enabled e.g. for a virtio-net device, +each queue pair will have a vhost_dev, and the only thing +shared between vhost devs currently is the chardev. This +patch introduces a vhost-user state structure which will +be shared by all vhost devs of the same virtio device. + +Signed-off-by: Tiwei Bie +Signed-off-by: Michael S. Tsirkin +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 4d0cf552d3a9585f380e8abdc313e4d416a56aa0) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + backends/cryptodev-vhost-user.c | 20 ++++++++++++++++- + hw/block/vhost-user-blk.c | 22 ++++++++++++++++++- + hw/scsi/vhost-user-scsi.c | 20 ++++++++++++++++- + hw/virtio/Makefile.objs | 2 +- + hw/virtio/vhost-stub.c | 10 +++++++++ + hw/virtio/vhost-user.c | 31 ++++++++++++++++++-------- + include/hw/virtio/vhost-user-blk.h | 2 ++ + include/hw/virtio/vhost-user-scsi.h | 2 ++ + include/hw/virtio/vhost-user.h | 20 +++++++++++++++++ + net/vhost-user.c | 44 ++++++++++++++++++++++++++++++------- + 10 files changed, 152 insertions(+), 21 deletions(-) + create mode 100644 include/hw/virtio/vhost-user.h + +diff --git a/backends/cryptodev-vhost-user.c b/backends/cryptodev-vhost-user.c +index 862d4f2..d52dacc 100644 +--- a/backends/cryptodev-vhost-user.c ++++ b/backends/cryptodev-vhost-user.c +@@ -26,6 +26,7 @@ + #include "qapi/error.h" + #include "qapi/qmp/qerror.h" + #include "qemu/error-report.h" ++#include "hw/virtio/vhost-user.h" + #include "standard-headers/linux/virtio_crypto.h" + #include "sysemu/cryptodev-vhost.h" + #include "chardev/char-fe.h" +@@ -46,6 +47,7 @@ + typedef struct CryptoDevBackendVhostUser { + CryptoDevBackend parent_obj; + ++ VhostUserState *vhost_user; + CharBackend chr; + char *chr_name; + bool opened; +@@ -102,7 +104,7 @@ cryptodev_vhost_user_start(int queues, + continue; + } + +- options.opaque = &s->chr; ++ options.opaque = s->vhost_user; + options.backend_type = VHOST_BACKEND_TYPE_USER; + options.cc = b->conf.peers.ccs[i]; + s->vhost_crypto[i] = cryptodev_vhost_init(&options); +@@ -185,6 +187,7 @@ static void cryptodev_vhost_user_init( + size_t i; + Error *local_err = NULL; + Chardev *chr; ++ VhostUserState *user; + CryptoDevBackendClient *cc; + CryptoDevBackendVhostUser *s = + CRYPTODEV_BACKEND_VHOST_USER(backend); +@@ -215,6 +218,15 @@ static void cryptodev_vhost_user_init( + } + } + ++ user = vhost_user_init(); ++ if (!user) { ++ error_setg(errp, "Failed to init vhost_user"); ++ return; ++ } ++ ++ user->chr = &s->chr; ++ s->vhost_user = user; ++ + qemu_chr_fe_set_handlers(&s->chr, NULL, NULL, + cryptodev_vhost_user_event, NULL, s, NULL, true); + +@@ -299,6 +311,12 @@ static void cryptodev_vhost_user_cleanup( + backend->conf.peers.ccs[i] = NULL; + } + } ++ ++ if (s->vhost_user) { ++ vhost_user_cleanup(s->vhost_user); ++ g_free(s->vhost_user); ++ s->vhost_user = NULL; ++ } + } + + static void cryptodev_vhost_user_set_chardev(Object *obj, +diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c +index 262baca..4021d71 100644 +--- a/hw/block/vhost-user-blk.c ++++ b/hw/block/vhost-user-blk.c +@@ -229,6 +229,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) + { + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBlk *s = VHOST_USER_BLK(vdev); ++ VhostUserState *user; + int i, ret; + + if (!s->chardev.chr) { +@@ -246,6 +247,15 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) + return; + } + ++ user = vhost_user_init(); ++ if (!user) { ++ error_setg(errp, "vhost-user-blk: failed to init vhost_user"); ++ return; ++ } ++ ++ user->chr = &s->chardev; ++ s->vhost_user = user; ++ + virtio_init(vdev, "virtio-blk", VIRTIO_ID_BLOCK, + sizeof(struct virtio_blk_config)); + +@@ -261,7 +271,7 @@ static void vhost_user_blk_device_realize(DeviceState *dev, Error **errp) + + vhost_dev_set_config_notifier(&s->dev, &blk_ops); + +- ret = vhost_dev_init(&s->dev, &s->chardev, VHOST_BACKEND_TYPE_USER, 0); ++ ret = vhost_dev_init(&s->dev, s->vhost_user, VHOST_BACKEND_TYPE_USER, 0); + if (ret < 0) { + error_setg(errp, "vhost-user-blk: vhost initialization failed: %s", + strerror(-ret)); +@@ -286,6 +296,10 @@ vhost_err: + virtio_err: + g_free(s->dev.vqs); + virtio_cleanup(vdev); ++ ++ vhost_user_cleanup(user); ++ g_free(user); ++ s->vhost_user = NULL; + } + + static void vhost_user_blk_device_unrealize(DeviceState *dev, Error **errp) +@@ -297,6 +311,12 @@ static void vhost_user_blk_device_unrealize(DeviceState *dev, Error **errp) + vhost_dev_cleanup(&s->dev); + g_free(s->dev.vqs); + virtio_cleanup(vdev); ++ ++ if (s->vhost_user) { ++ vhost_user_cleanup(s->vhost_user); ++ g_free(s->vhost_user); ++ s->vhost_user = NULL; ++ } + } + + static void vhost_user_blk_instance_init(Object *obj) +diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c +index 9389ed4..9355cfd 100644 +--- a/hw/scsi/vhost-user-scsi.c ++++ b/hw/scsi/vhost-user-scsi.c +@@ -69,6 +69,7 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) + VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(dev); + VHostUserSCSI *s = VHOST_USER_SCSI(dev); + VHostSCSICommon *vsc = VHOST_SCSI_COMMON(s); ++ VhostUserState *user; + Error *err = NULL; + int ret; + +@@ -85,19 +86,30 @@ static void vhost_user_scsi_realize(DeviceState *dev, Error **errp) + return; + } + ++ user = vhost_user_init(); ++ if (!user) { ++ error_setg(errp, "vhost-user-scsi: failed to init vhost_user"); ++ return; ++ } ++ user->chr = &vs->conf.chardev; ++ + vsc->dev.nvqs = 2 + vs->conf.num_queues; + vsc->dev.vqs = g_new(struct vhost_virtqueue, vsc->dev.nvqs); + vsc->dev.vq_index = 0; + vsc->dev.backend_features = 0; + +- ret = vhost_dev_init(&vsc->dev, (void *)&vs->conf.chardev, ++ ret = vhost_dev_init(&vsc->dev, user, + VHOST_BACKEND_TYPE_USER, 0); + if (ret < 0) { + error_setg(errp, "vhost-user-scsi: vhost initialization failed: %s", + strerror(-ret)); ++ vhost_user_cleanup(user); ++ g_free(user); + return; + } + ++ s->vhost_user = user; ++ + /* Channel and lun both are 0 for bootable vhost-user-scsi disk */ + vsc->channel = 0; + vsc->lun = 0; +@@ -117,6 +129,12 @@ static void vhost_user_scsi_unrealize(DeviceState *dev, Error **errp) + g_free(vsc->dev.vqs); + + virtio_scsi_common_unrealize(dev, errp); ++ ++ if (s->vhost_user) { ++ vhost_user_cleanup(s->vhost_user); ++ g_free(s->vhost_user); ++ s->vhost_user = NULL; ++ } + } + + static uint64_t vhost_user_scsi_get_features(VirtIODevice *vdev, +diff --git a/hw/virtio/Makefile.objs b/hw/virtio/Makefile.objs +index a5a0936..00c6696 100644 +--- a/hw/virtio/Makefile.objs ++++ b/hw/virtio/Makefile.objs +@@ -12,5 +12,5 @@ obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock.o + #obj-$(CONFIG_VIRTIO_PCI) += virtio-crypto-pci.o + endif + +-common-obj-$(call lnot,$(CONFIG_LINUX)) += vhost-stub.o ++common-obj-$(call lnot,$(call land,$(CONFIG_VIRTIO),$(CONFIG_LINUX))) += vhost-stub.o + common-obj-$(CONFIG_ALL) += vhost-stub.o +diff --git a/hw/virtio/vhost-stub.c b/hw/virtio/vhost-stub.c +index 2d76cde..049089b 100644 +--- a/hw/virtio/vhost-stub.c ++++ b/hw/virtio/vhost-stub.c +@@ -1,7 +1,17 @@ + #include "qemu/osdep.h" + #include "hw/virtio/vhost.h" ++#include "hw/virtio/vhost-user.h" + + bool vhost_has_free_slot(void) + { + return true; + } ++ ++VhostUserState *vhost_user_init(void) ++{ ++ return NULL; ++} ++ ++void vhost_user_cleanup(VhostUserState *user) ++{ ++} +diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c +index e8027ad..a715c5c 100644 +--- a/hw/virtio/vhost-user.c ++++ b/hw/virtio/vhost-user.c +@@ -11,6 +11,7 @@ + #include "qemu/osdep.h" + #include "qapi/error.h" + #include "hw/virtio/vhost.h" ++#include "hw/virtio/vhost-user.h" + #include "hw/virtio/vhost-backend.h" + #include "hw/virtio/virtio-net.h" + #include "chardev/char-fe.h" +@@ -175,7 +176,8 @@ static VhostUserMsg m __attribute__ ((unused)); + + struct vhost_user { + struct vhost_dev *dev; +- CharBackend *chr; ++ /* Shared between vhost devs of the same virtio device */ ++ VhostUserState *user; + int slave_fd; + NotifierWithReturn postcopy_notifier; + struct PostCopyFD postcopy_fd; +@@ -201,7 +203,7 @@ static bool ioeventfd_enabled(void) + static int vhost_user_read(struct vhost_dev *dev, VhostUserMsg *msg) + { + struct vhost_user *u = dev->opaque; +- CharBackend *chr = u->chr; ++ CharBackend *chr = u->user->chr; + uint8_t *p = (uint8_t *) msg; + int r, size = VHOST_USER_HDR_SIZE; + +@@ -287,7 +289,7 @@ static int vhost_user_write(struct vhost_dev *dev, VhostUserMsg *msg, + int *fds, int fd_num) + { + struct vhost_user *u = dev->opaque; +- CharBackend *chr = u->chr; ++ CharBackend *chr = u->user->chr; + int ret, size = VHOST_USER_HDR_SIZE + msg->hdr.size; + + /* +@@ -1090,7 +1092,7 @@ static int vhost_user_postcopy_waker(struct PostCopyFD *pcfd, RAMBlock *rb, + static int vhost_user_postcopy_advise(struct vhost_dev *dev, Error **errp) + { + struct vhost_user *u = dev->opaque; +- CharBackend *chr = u->chr; ++ CharBackend *chr = u->user->chr; + int ufd; + VhostUserMsg msg = { + .hdr.request = VHOST_USER_POSTCOPY_ADVISE, +@@ -1228,7 +1230,7 @@ static int vhost_user_postcopy_notifier(NotifierWithReturn *notifier, + return 0; + } + +-static int vhost_user_init(struct vhost_dev *dev, void *opaque) ++static int vhost_user_backend_init(struct vhost_dev *dev, void *opaque) + { + uint64_t features, protocol_features; + struct vhost_user *u; +@@ -1237,7 +1239,7 @@ static int vhost_user_init(struct vhost_dev *dev, void *opaque) + assert(dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER); + + u = g_new0(struct vhost_user, 1); +- u->chr = opaque; ++ u->user = opaque; + u->slave_fd = -1; + u->dev = dev; + dev->opaque = u; +@@ -1313,7 +1315,7 @@ static int vhost_user_init(struct vhost_dev *dev, void *opaque) + return 0; + } + +-static int vhost_user_cleanup(struct vhost_dev *dev) ++static int vhost_user_backend_cleanup(struct vhost_dev *dev) + { + struct vhost_user *u; + +@@ -1637,10 +1639,21 @@ static bool vhost_user_mem_section_filter(struct vhost_dev *dev, + return result; + } + ++VhostUserState *vhost_user_init(void) ++{ ++ VhostUserState *user = g_new0(struct VhostUserState, 1); ++ ++ return user; ++} ++ ++void vhost_user_cleanup(VhostUserState *user) ++{ ++} ++ + const VhostOps user_ops = { + .backend_type = VHOST_BACKEND_TYPE_USER, +- .vhost_backend_init = vhost_user_init, +- .vhost_backend_cleanup = vhost_user_cleanup, ++ .vhost_backend_init = vhost_user_backend_init, ++ .vhost_backend_cleanup = vhost_user_backend_cleanup, + .vhost_backend_memslots_limit = vhost_user_memslots_limit, + .vhost_set_log_base = vhost_user_set_log_base, + .vhost_set_mem_table = vhost_user_set_mem_table, +diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-user-blk.h +index 5804cc9..f1258ae 100644 +--- a/include/hw/virtio/vhost-user-blk.h ++++ b/include/hw/virtio/vhost-user-blk.h +@@ -21,6 +21,7 @@ + #include "hw/block/block.h" + #include "chardev/char-fe.h" + #include "hw/virtio/vhost.h" ++#include "hw/virtio/vhost-user.h" + + #define TYPE_VHOST_USER_BLK "vhost-user-blk" + #define VHOST_USER_BLK(obj) \ +@@ -36,6 +37,7 @@ typedef struct VHostUserBlk { + uint32_t config_wce; + uint32_t config_ro; + struct vhost_dev dev; ++ VhostUserState *vhost_user; + } VHostUserBlk; + + #endif +diff --git a/include/hw/virtio/vhost-user-scsi.h b/include/hw/virtio/vhost-user-scsi.h +index 01861f7..3ec34ae 100644 +--- a/include/hw/virtio/vhost-user-scsi.h ++++ b/include/hw/virtio/vhost-user-scsi.h +@@ -21,6 +21,7 @@ + #include "hw/qdev.h" + #include "hw/virtio/virtio-scsi.h" + #include "hw/virtio/vhost.h" ++#include "hw/virtio/vhost-user.h" + #include "hw/virtio/vhost-scsi-common.h" + + #define TYPE_VHOST_USER_SCSI "vhost-user-scsi" +@@ -30,6 +31,7 @@ + typedef struct VHostUserSCSI { + VHostSCSICommon parent_obj; + uint64_t host_features; ++ VhostUserState *vhost_user; + } VHostUserSCSI; + + #endif /* VHOST_USER_SCSI_H */ +diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h +new file mode 100644 +index 0000000..eb8bc0d +--- /dev/null ++++ b/include/hw/virtio/vhost-user.h +@@ -0,0 +1,20 @@ ++/* ++ * Copyright (c) 2017-2018 Intel Corporation ++ * ++ * This work is licensed under the terms of the GNU GPL, version 2. ++ * See the COPYING file in the top-level directory. ++ */ ++ ++#ifndef HW_VIRTIO_VHOST_USER_H ++#define HW_VIRTIO_VHOST_USER_H ++ ++#include "chardev/char-fe.h" ++ ++typedef struct VhostUserState { ++ CharBackend *chr; ++} VhostUserState; ++ ++VhostUserState *vhost_user_init(void); ++void vhost_user_cleanup(VhostUserState *user); ++ ++#endif +diff --git a/net/vhost-user.c b/net/vhost-user.c +index fa28aad..608b837 100644 +--- a/net/vhost-user.c ++++ b/net/vhost-user.c +@@ -12,6 +12,7 @@ + #include "clients.h" + #include "net/vhost_net.h" + #include "net/vhost-user.h" ++#include "hw/virtio/vhost-user.h" + #include "chardev/char-fe.h" + #include "qapi/error.h" + #include "qapi/qapi-commands-net.h" +@@ -23,6 +24,7 @@ + typedef struct NetVhostUserState { + NetClientState nc; + CharBackend chr; /* only queue index 0 */ ++ VhostUserState *vhost_user; + VHostNetState *vhost_net; + guint watch; + uint64_t acked_features; +@@ -64,7 +66,8 @@ static void vhost_user_stop(int queues, NetClientState *ncs[]) + } + } + +-static int vhost_user_start(int queues, NetClientState *ncs[], CharBackend *be) ++static int vhost_user_start(int queues, NetClientState *ncs[], ++ VhostUserState *be) + { + VhostNetOptions options; + struct vhost_net *net = NULL; +@@ -144,7 +147,7 @@ static ssize_t vhost_user_receive(NetClientState *nc, const uint8_t *buf, + return size; + } + +-static void vhost_user_cleanup(NetClientState *nc) ++static void net_vhost_user_cleanup(NetClientState *nc) + { + NetVhostUserState *s = DO_UPCAST(NetVhostUserState, nc, nc); + +@@ -159,6 +162,11 @@ static void vhost_user_cleanup(NetClientState *nc) + s->watch = 0; + } + qemu_chr_fe_deinit(&s->chr, true); ++ if (s->vhost_user) { ++ vhost_user_cleanup(s->vhost_user); ++ g_free(s->vhost_user); ++ s->vhost_user = NULL; ++ } + } + + qemu_purge_queued_packets(nc); +@@ -182,7 +190,7 @@ static NetClientInfo net_vhost_user_info = { + .type = NET_CLIENT_DRIVER_VHOST_USER, + .size = sizeof(NetVhostUserState), + .receive = vhost_user_receive, +- .cleanup = vhost_user_cleanup, ++ .cleanup = net_vhost_user_cleanup, + .has_vnet_hdr = vhost_user_has_vnet_hdr, + .has_ufo = vhost_user_has_ufo, + }; +@@ -244,7 +252,7 @@ static void net_vhost_user_event(void *opaque, int event) + trace_vhost_user_event(chr->label, event); + switch (event) { + case CHR_EVENT_OPENED: +- if (vhost_user_start(queues, ncs, &s->chr) < 0) { ++ if (vhost_user_start(queues, ncs, s->vhost_user) < 0) { + qemu_chr_fe_disconnect(&s->chr); + return; + } +@@ -283,12 +291,19 @@ static int net_vhost_user_init(NetClientState *peer, const char *device, + { + Error *err = NULL; + NetClientState *nc, *nc0 = NULL; +- NetVhostUserState *s; ++ VhostUserState *user = NULL; ++ NetVhostUserState *s = NULL; + int i; + + assert(name); + assert(queues > 0); + ++ user = vhost_user_init(); ++ if (!user) { ++ error_report("failed to init vhost_user"); ++ goto err; ++ } ++ + for (i = 0; i < queues; i++) { + nc = qemu_new_net_client(&net_vhost_user_info, peer, device, name); + snprintf(nc->info_str, sizeof(nc->info_str), "vhost-user%d to %s", +@@ -299,17 +314,19 @@ static int net_vhost_user_init(NetClientState *peer, const char *device, + s = DO_UPCAST(NetVhostUserState, nc, nc); + if (!qemu_chr_fe_init(&s->chr, chr, &err)) { + error_report_err(err); +- return -1; ++ goto err; + } ++ user->chr = &s->chr; + } +- ++ s = DO_UPCAST(NetVhostUserState, nc, nc); ++ s->vhost_user = user; + } + + s = DO_UPCAST(NetVhostUserState, nc, nc0); + do { + if (qemu_chr_fe_wait_connected(&s->chr, &err) < 0) { + error_report_err(err); +- return -1; ++ goto err; + } + qemu_chr_fe_set_handlers(&s->chr, NULL, NULL, + net_vhost_user_event, NULL, nc0->name, NULL, +@@ -319,6 +336,17 @@ static int net_vhost_user_init(NetClientState *peer, const char *device, + assert(s->vhost_net); + + return 0; ++ ++err: ++ if (user) { ++ vhost_user_cleanup(user); ++ g_free(user); ++ if (s) { ++ s->vhost_user = NULL; ++ } ++ } ++ ++ return -1; + } + + static Chardev *net_vhost_claim_chardev( +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-user-support-receiving-file-descriptors-in-sla.patch b/SOURCES/kvm-vhost-user-support-receiving-file-descriptors-in-sla.patch new file mode 100644 index 0000000..8406c4f --- /dev/null +++ b/SOURCES/kvm-vhost-user-support-receiving-file-descriptors-in-sla.patch @@ -0,0 +1,106 @@ +From 426cb49d423f5386b2d869009e29d7061e51b998 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:38 +0200 +Subject: [PATCH 159/268] vhost-user: support receiving file descriptors in + slave_read + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-4-git-send-email-plai@redhat.com> +Patchwork-id: 80933 +O-Subject: [RHEL7.6 PATCH BZ 1526645 03/10] vhost-user: support receiving file descriptors in slave_read +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 1f3a4519b1c107b5db2434b30638353978366b4d) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + hw/virtio/vhost-user.c | 41 ++++++++++++++++++++++++++++++++++++++++- + 1 file changed, 40 insertions(+), 1 deletion(-) + +diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c +index 38da869..85d8fd2 100644 +--- a/hw/virtio/vhost-user.c ++++ b/hw/virtio/vhost-user.c +@@ -852,14 +852,44 @@ static void slave_read(void *opaque) + VhostUserHeader hdr = { 0, }; + VhostUserPayload payload = { 0, }; + int size, ret = 0; ++ struct iovec iov; ++ struct msghdr msgh; ++ int fd = -1; ++ char control[CMSG_SPACE(sizeof(fd))]; ++ struct cmsghdr *cmsg; ++ size_t fdsize; ++ ++ memset(&msgh, 0, sizeof(msgh)); ++ msgh.msg_iov = &iov; ++ msgh.msg_iovlen = 1; ++ msgh.msg_control = control; ++ msgh.msg_controllen = sizeof(control); + + /* Read header */ +- size = read(u->slave_fd, &hdr, VHOST_USER_HDR_SIZE); ++ iov.iov_base = &hdr; ++ iov.iov_len = VHOST_USER_HDR_SIZE; ++ ++ size = recvmsg(u->slave_fd, &msgh, 0); + if (size != VHOST_USER_HDR_SIZE) { + error_report("Failed to read from slave."); + goto err; + } + ++ if (msgh.msg_flags & MSG_CTRUNC) { ++ error_report("Truncated message."); ++ goto err; ++ } ++ ++ for (cmsg = CMSG_FIRSTHDR(&msgh); cmsg != NULL; ++ cmsg = CMSG_NXTHDR(&msgh, cmsg)) { ++ if (cmsg->cmsg_level == SOL_SOCKET && ++ cmsg->cmsg_type == SCM_RIGHTS) { ++ fdsize = cmsg->cmsg_len - CMSG_LEN(0); ++ memcpy(&fd, CMSG_DATA(cmsg), fdsize); ++ break; ++ } ++ } ++ + if (hdr.size > VHOST_USER_PAYLOAD_SIZE) { + error_report("Failed to read msg header." + " Size %d exceeds the maximum %zu.", hdr.size, +@@ -883,9 +913,15 @@ static void slave_read(void *opaque) + break; + default: + error_report("Received unexpected msg type."); ++ if (fd != -1) { ++ close(fd); ++ } + ret = -EINVAL; + } + ++ /* Message handlers need to make sure that fd will be consumed. */ ++ fd = -1; ++ + /* + * REPLY_ACK feature handling. Other reply types has to be managed + * directly in their request handlers. +@@ -918,6 +954,9 @@ err: + qemu_set_fd_handler(u->slave_fd, NULL, NULL, NULL); + close(u->slave_fd); + u->slave_fd = -1; ++ if (fd != -1) { ++ close(fd); ++ } + return; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vhost-user-support-registering-external-host-notifie.patch b/SOURCES/kvm-vhost-user-support-registering-external-host-notifie.patch new file mode 100644 index 0000000..db2ab74 --- /dev/null +++ b/SOURCES/kvm-vhost-user-support-registering-external-host-notifie.patch @@ -0,0 +1,315 @@ +From 40b67cbd00745cb4aeca063619d4b8e94e926a31 Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:44 +0200 +Subject: [PATCH 165/268] vhost-user: support registering external host + notifiers + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-10-git-send-email-plai@redhat.com> +Patchwork-id: 80940 +O-Subject: [RHEL7.6 PATCH BZ 1526645 09/10] vhost-user: support registering external host notifiers +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +This patch introduces VHOST_USER_PROTOCOL_F_HOST_NOTIFIER. +With this feature negotiated, vhost-user backend can register +memory region based host notifiers. And it will allow the guest +driver in the VM to notify the hardware accelerator at the +vhost-user backend directly. + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 44866521bd6ea8e5152a87664dea1eee90c9438b) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + docs/interop/vhost-user.txt | 33 ++++++++++++ + hw/virtio/vhost-user.c | 113 +++++++++++++++++++++++++++++++++++++++++ + include/hw/virtio/vhost-user.h | 8 +++ + 3 files changed, 154 insertions(+) + +diff --git a/docs/interop/vhost-user.txt b/docs/interop/vhost-user.txt +index 682a683..d51fd58 100644 +--- a/docs/interop/vhost-user.txt ++++ b/docs/interop/vhost-user.txt +@@ -132,6 +132,16 @@ Depending on the request type, payload can be: + Payload: Size bytes array holding the contents of the virtio + device's configuration space + ++ * Vring area description ++ ----------------------- ++ | u64 | size | offset | ++ ----------------------- ++ ++ u64: a 64-bit integer contains vring index and flags ++ Size: a 64-bit size of this area ++ Offset: a 64-bit offset of this area from the start of the ++ supplied file descriptor ++ + In QEMU the vhost-user message is implemented with the following struct: + + typedef struct VhostUserMsg { +@@ -146,6 +156,7 @@ typedef struct VhostUserMsg { + VhostUserLog log; + struct vhost_iotlb_msg iotlb; + VhostUserConfig config; ++ VhostUserVringArea area; + }; + } QEMU_PACKED VhostUserMsg; + +@@ -385,6 +396,7 @@ Protocol features + #define VHOST_USER_PROTOCOL_F_PAGEFAULT 8 + #define VHOST_USER_PROTOCOL_F_CONFIG 9 + #define VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD 10 ++#define VHOST_USER_PROTOCOL_F_HOST_NOTIFIER 11 + + Master message types + -------------------- +@@ -782,6 +794,27 @@ Slave message types + the VHOST_USER_NEED_REPLY flag, master must respond with zero when + operation is successfully completed, or non-zero otherwise. + ++ * VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG ++ ++ Id: 3 ++ Equivalent ioctl: N/A ++ Slave payload: vring area description ++ Master payload: N/A ++ ++ Sets host notifier for a specified queue. The queue index is contained ++ in the u64 field of the vring area description. The host notifier is ++ described by the file descriptor (typically it's a VFIO device fd) which ++ is passed as ancillary data and the size (which is mmap size and should ++ be the same as host page size) and offset (which is mmap offset) carried ++ in the vring area description. QEMU can mmap the file descriptor based ++ on the size and offset to get a memory range. Registering a host notifier ++ means mapping this memory range to the VM as the specified queue's notify ++ MMIO region. Slave sends this request to tell QEMU to de-register the ++ existing notifier if any and register the new notifier if the request is ++ sent with a file descriptor. ++ This request should be sent only when VHOST_USER_PROTOCOL_F_HOST_NOTIFIER ++ protocol feature has been successfully negotiated. ++ + VHOST_USER_PROTOCOL_F_REPLY_ACK: + ------------------------------- + The original vhost-user specification only demands replies for certain +diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c +index a715c5c..c30fc3d 100644 +--- a/hw/virtio/vhost-user.c ++++ b/hw/virtio/vhost-user.c +@@ -13,6 +13,7 @@ + #include "hw/virtio/vhost.h" + #include "hw/virtio/vhost-user.h" + #include "hw/virtio/vhost-backend.h" ++#include "hw/virtio/virtio.h" + #include "hw/virtio/virtio-net.h" + #include "chardev/char-fe.h" + #include "sysemu/kvm.h" +@@ -50,6 +51,7 @@ enum VhostUserProtocolFeature { + VHOST_USER_PROTOCOL_F_PAGEFAULT = 8, + VHOST_USER_PROTOCOL_F_CONFIG = 9, + VHOST_USER_PROTOCOL_F_SLAVE_SEND_FD = 10, ++ VHOST_USER_PROTOCOL_F_HOST_NOTIFIER = 11, + VHOST_USER_PROTOCOL_F_MAX + }; + +@@ -94,6 +96,7 @@ typedef enum VhostUserSlaveRequest { + VHOST_USER_SLAVE_NONE = 0, + VHOST_USER_SLAVE_IOTLB_MSG = 1, + VHOST_USER_SLAVE_CONFIG_CHANGE_MSG = 2, ++ VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG = 3, + VHOST_USER_SLAVE_MAX + } VhostUserSlaveRequest; + +@@ -138,6 +141,12 @@ static VhostUserConfig c __attribute__ ((unused)); + + sizeof(c.size) \ + + sizeof(c.flags)) + ++typedef struct VhostUserVringArea { ++ uint64_t u64; ++ uint64_t size; ++ uint64_t offset; ++} VhostUserVringArea; ++ + typedef struct { + VhostUserRequest request; + +@@ -159,6 +168,7 @@ typedef union { + struct vhost_iotlb_msg iotlb; + VhostUserConfig config; + VhostUserCryptoSession session; ++ VhostUserVringArea area; + } VhostUserPayload; + + typedef struct VhostUserMsg { +@@ -640,9 +650,37 @@ static int vhost_user_set_vring_num(struct vhost_dev *dev, + return vhost_set_vring(dev, VHOST_USER_SET_VRING_NUM, ring); + } + ++static void vhost_user_host_notifier_restore(struct vhost_dev *dev, ++ int queue_idx) ++{ ++ struct vhost_user *u = dev->opaque; ++ VhostUserHostNotifier *n = &u->user->notifier[queue_idx]; ++ VirtIODevice *vdev = dev->vdev; ++ ++ if (n->addr && !n->set) { ++ virtio_queue_set_host_notifier_mr(vdev, queue_idx, &n->mr, true); ++ n->set = true; ++ } ++} ++ ++static void vhost_user_host_notifier_remove(struct vhost_dev *dev, ++ int queue_idx) ++{ ++ struct vhost_user *u = dev->opaque; ++ VhostUserHostNotifier *n = &u->user->notifier[queue_idx]; ++ VirtIODevice *vdev = dev->vdev; ++ ++ if (n->addr && n->set) { ++ virtio_queue_set_host_notifier_mr(vdev, queue_idx, &n->mr, false); ++ n->set = false; ++ } ++} ++ + static int vhost_user_set_vring_base(struct vhost_dev *dev, + struct vhost_vring_state *ring) + { ++ vhost_user_host_notifier_restore(dev, ring->index); ++ + return vhost_set_vring(dev, VHOST_USER_SET_VRING_BASE, ring); + } + +@@ -676,6 +714,8 @@ static int vhost_user_get_vring_base(struct vhost_dev *dev, + .hdr.size = sizeof(msg.payload.state), + }; + ++ vhost_user_host_notifier_remove(dev, ring->index); ++ + if (vhost_user_write(dev, &msg, NULL, 0) < 0) { + return -1; + } +@@ -849,6 +889,66 @@ static int vhost_user_slave_handle_config_change(struct vhost_dev *dev) + return ret; + } + ++static int vhost_user_slave_handle_vring_host_notifier(struct vhost_dev *dev, ++ VhostUserVringArea *area, ++ int fd) ++{ ++ int queue_idx = area->u64 & VHOST_USER_VRING_IDX_MASK; ++ size_t page_size = qemu_real_host_page_size; ++ struct vhost_user *u = dev->opaque; ++ VhostUserState *user = u->user; ++ VirtIODevice *vdev = dev->vdev; ++ VhostUserHostNotifier *n; ++ void *addr; ++ char *name; ++ ++ if (!virtio_has_feature(dev->protocol_features, ++ VHOST_USER_PROTOCOL_F_HOST_NOTIFIER) || ++ vdev == NULL || queue_idx >= virtio_get_num_queues(vdev)) { ++ return -1; ++ } ++ ++ n = &user->notifier[queue_idx]; ++ ++ if (n->addr) { ++ virtio_queue_set_host_notifier_mr(vdev, queue_idx, &n->mr, false); ++ object_unparent(OBJECT(&n->mr)); ++ munmap(n->addr, page_size); ++ n->addr = NULL; ++ } ++ ++ if (area->u64 & VHOST_USER_VRING_NOFD_MASK) { ++ return 0; ++ } ++ ++ /* Sanity check. */ ++ if (area->size != page_size) { ++ return -1; ++ } ++ ++ addr = mmap(NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, ++ fd, area->offset); ++ if (addr == MAP_FAILED) { ++ return -1; ++ } ++ ++ name = g_strdup_printf("vhost-user/host-notifier@%p mmaps[%d]", ++ user, queue_idx); ++ memory_region_init_ram_device_ptr(&n->mr, OBJECT(vdev), name, ++ page_size, addr); ++ g_free(name); ++ ++ if (virtio_queue_set_host_notifier_mr(vdev, queue_idx, &n->mr, true)) { ++ munmap(addr, page_size); ++ return -1; ++ } ++ ++ n->addr = addr; ++ n->set = true; ++ ++ return 0; ++} ++ + static void slave_read(void *opaque) + { + struct vhost_dev *dev = opaque; +@@ -917,6 +1017,10 @@ static void slave_read(void *opaque) + case VHOST_USER_SLAVE_CONFIG_CHANGE_MSG : + ret = vhost_user_slave_handle_config_change(dev); + break; ++ case VHOST_USER_SLAVE_VRING_HOST_NOTIFIER_MSG: ++ ret = vhost_user_slave_handle_vring_host_notifier(dev, &payload.area, ++ fd[0]); ++ break; + default: + error_report("Received unexpected msg type."); + ret = -EINVAL; +@@ -1648,6 +1752,15 @@ VhostUserState *vhost_user_init(void) + + void vhost_user_cleanup(VhostUserState *user) + { ++ int i; ++ ++ for (i = 0; i < VIRTIO_QUEUE_MAX; i++) { ++ if (user->notifier[i].addr) { ++ object_unparent(OBJECT(&user->notifier[i].mr)); ++ munmap(user->notifier[i].addr, qemu_real_host_page_size); ++ user->notifier[i].addr = NULL; ++ } ++ } + } + + const VhostOps user_ops = { +diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h +index eb8bc0d..fd66039 100644 +--- a/include/hw/virtio/vhost-user.h ++++ b/include/hw/virtio/vhost-user.h +@@ -9,9 +9,17 @@ + #define HW_VIRTIO_VHOST_USER_H + + #include "chardev/char-fe.h" ++#include "hw/virtio/virtio.h" ++ ++typedef struct VhostUserHostNotifier { ++ MemoryRegion mr; ++ void *addr; ++ bool set; ++} VhostUserHostNotifier; + + typedef struct VhostUserState { + CharBackend *chr; ++ VhostUserHostNotifier notifier[VIRTIO_QUEUE_MAX]; + } VhostUserState; + + VhostUserState *vhost_user_init(void); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-gpu-disable-scanout-when-backing-resource-is-.patch b/SOURCES/kvm-virtio-gpu-disable-scanout-when-backing-resource-is-.patch new file mode 100644 index 0000000..dd130bd --- /dev/null +++ b/SOURCES/kvm-virtio-gpu-disable-scanout-when-backing-resource-is-.patch @@ -0,0 +1,51 @@ +From 590e0651a901f793df8aeee41968b0b2f838e2e8 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 4 Jul 2018 09:54:09 +0200 +Subject: [PATCH 189/268] virtio-gpu: disable scanout when backing resource is + destroyed +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20180704095409.14514-4-kraxel@redhat.com> +Patchwork-id: 81225 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 3/3] virtio-gpu: disable scanout when backing resource is destroyed +Bugzilla: 1589634 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Laszlo Ersek + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Marc-André Lureau +Message-id: 20180702162443.16796-4-kraxel@redhat.com +(cherry picked from commit 1fccd7c5a9a722a9cbf1bc91693f4618034f01ac) +Signed-off-by: Miroslav Rezanina +--- + hw/display/virtio-gpu.c | 10 ++++++++++ + 1 file changed, 10 insertions(+) + +diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c +index 336dc59..08cd567 100644 +--- a/hw/display/virtio-gpu.c ++++ b/hw/display/virtio-gpu.c +@@ -430,6 +430,16 @@ static void virtio_gpu_disable_scanout(VirtIOGPU *g, int scanout_id) + static void virtio_gpu_resource_destroy(VirtIOGPU *g, + struct virtio_gpu_simple_resource *res) + { ++ int i; ++ ++ if (res->scanout_bitmask) { ++ for (i = 0; i < g->conf.max_outputs; i++) { ++ if (res->scanout_bitmask & (1 << i)) { ++ virtio_gpu_disable_scanout(g, i); ++ } ++ } ++ } ++ + pixman_image_unref(res->image); + virtio_gpu_cleanup_mapping(res); + QTAILQ_REMOVE(&g->reslist, res, next); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-gpu-tweak-scanout-disable.patch b/SOURCES/kvm-virtio-gpu-tweak-scanout-disable.patch new file mode 100644 index 0000000..236e26c --- /dev/null +++ b/SOURCES/kvm-virtio-gpu-tweak-scanout-disable.patch @@ -0,0 +1,98 @@ +From e41c578615d2b52db633214891631ca61973f34b Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 4 Jul 2018 09:54:07 +0200 +Subject: [PATCH 187/268] virtio-gpu: tweak scanout disable. +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20180704095409.14514-2-kraxel@redhat.com> +Patchwork-id: 81226 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/3] virtio-gpu: tweak scanout disable. +Bugzilla: 1589634 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Laszlo Ersek + +- Factor out the code to virtio_gpu_disable_scanout(). +- Allow disable scanout 0, show a message then. +- Clear scanout->resource_id. + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Marc-André Lureau +Message-id: 20180702162443.16796-2-kraxel@redhat.com +(cherry picked from commit da566a18a7799e5a655cd9f87e222b672cc93e7b) +Signed-off-by: Miroslav Rezanina +--- + hw/display/virtio-gpu.c | 47 +++++++++++++++++++++++++++++------------------ + 1 file changed, 29 insertions(+), 18 deletions(-) + +diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c +index 2dd3c34..054ec73 100644 +--- a/hw/display/virtio-gpu.c ++++ b/hw/display/virtio-gpu.c +@@ -399,6 +399,34 @@ static void virtio_gpu_resource_create_2d(VirtIOGPU *g, + g->hostmem += res->hostmem; + } + ++static void virtio_gpu_disable_scanout(VirtIOGPU *g, int scanout_id) ++{ ++ struct virtio_gpu_scanout *scanout = &g->scanout[scanout_id]; ++ struct virtio_gpu_simple_resource *res; ++ DisplaySurface *ds = NULL; ++ ++ if (scanout->resource_id == 0) { ++ return; ++ } ++ ++ res = virtio_gpu_find_resource(g, scanout->resource_id); ++ if (res) { ++ res->scanout_bitmask &= ~(1 << scanout_id); ++ } ++ ++ if (scanout_id == 0) { ++ /* primary head */ ++ ds = qemu_create_message_surface(scanout->width ?: 640, ++ scanout->height ?: 480, ++ "Guest disabled display."); ++ } ++ dpy_gfx_replace_surface(scanout->con, ds); ++ scanout->resource_id = 0; ++ scanout->ds = NULL; ++ scanout->width = 0; ++ scanout->height = 0; ++} ++ + static void virtio_gpu_resource_destroy(VirtIOGPU *g, + struct virtio_gpu_simple_resource *res) + { +@@ -583,24 +611,7 @@ static void virtio_gpu_set_scanout(VirtIOGPU *g, + + g->enable = 1; + if (ss.resource_id == 0) { +- scanout = &g->scanout[ss.scanout_id]; +- if (scanout->resource_id) { +- res = virtio_gpu_find_resource(g, scanout->resource_id); +- if (res) { +- res->scanout_bitmask &= ~(1 << ss.scanout_id); +- } +- } +- if (ss.scanout_id == 0) { +- qemu_log_mask(LOG_GUEST_ERROR, +- "%s: illegal scanout id specified %d", +- __func__, ss.scanout_id); +- cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_SCANOUT_ID; +- return; +- } +- dpy_gfx_replace_surface(g->scanout[ss.scanout_id].con, NULL); +- scanout->ds = NULL; +- scanout->width = 0; +- scanout->height = 0; ++ virtio_gpu_disable_scanout(g, ss.scanout_id); + return; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-gpu-update-old-resource-too.patch b/SOURCES/kvm-virtio-gpu-update-old-resource-too.patch new file mode 100644 index 0000000..4c7ab55 --- /dev/null +++ b/SOURCES/kvm-virtio-gpu-update-old-resource-too.patch @@ -0,0 +1,58 @@ +From 6da14e1532783c95c18db157cd8f9b7dada31220 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 4 Jul 2018 09:54:08 +0200 +Subject: [PATCH 188/268] virtio-gpu: update old resource too. +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Gerd Hoffmann +Message-id: <20180704095409.14514-3-kraxel@redhat.com> +Patchwork-id: 81227 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/3] virtio-gpu: update old resource too. +Bugzilla: 1589634 +RH-Acked-by: Laurent Vivier +RH-Acked-by: Marc-André Lureau +RH-Acked-by: Laszlo Ersek + +When switching scanout from one resource to another we must update the +scanout_bitmask field for both new (set bit) and old (clear bit) +resource. + +Signed-off-by: Gerd Hoffmann +Reviewed-by: Marc-André Lureau +Message-id: 20180702162443.16796-3-kraxel@redhat.com +(cherry picked from commit c806cfa036a7ec991170eb9899f3a676bfcc3253) +Signed-off-by: Miroslav Rezanina +--- + hw/display/virtio-gpu.c | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c +index 054ec73..336dc59 100644 +--- a/hw/display/virtio-gpu.c ++++ b/hw/display/virtio-gpu.c +@@ -590,7 +590,7 @@ static void virtio_unref_resource(pixman_image_t *image, void *data) + static void virtio_gpu_set_scanout(VirtIOGPU *g, + struct virtio_gpu_ctrl_command *cmd) + { +- struct virtio_gpu_simple_resource *res; ++ struct virtio_gpu_simple_resource *res, *ores; + struct virtio_gpu_scanout *scanout; + pixman_format_code_t format; + uint32_t offset; +@@ -664,6 +664,11 @@ static void virtio_gpu_set_scanout(VirtIOGPU *g, + dpy_gfx_replace_surface(g->scanout[ss.scanout_id].con, scanout->ds); + } + ++ ores = virtio_gpu_find_resource(g, scanout->resource_id); ++ if (ores) { ++ ores->scanout_bitmask &= ~(1 << ss.scanout_id); ++ } ++ + res->scanout_bitmask |= (1 << ss.scanout_id); + scanout->resource_id = ss.resource_id; + scanout->x = ss.r.x; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-net-test-accept-variable-length-argument-in-p.patch b/SOURCES/kvm-virtio-net-test-accept-variable-length-argument-in-p.patch new file mode 100644 index 0000000..d56309a --- /dev/null +++ b/SOURCES/kvm-virtio-net-test-accept-variable-length-argument-in-p.patch @@ -0,0 +1,78 @@ +From 40e52779c8876bc453e5d23e6b36e6f0d7302fa4 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:59:02 +0000 +Subject: [PATCH 07/11] virtio-net-test: accept variable length argument in + pci_test_start() +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-8-jasowang@redhat.com> +Patchwork-id: 83981 +O-Subject: [RHEL8 qemu-kvm PATCH 7/9] virtio-net-test: accept variable length argument in pci_test_start() +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +This allows flexibility to be reused for all kinds of command line +used by other tests. + +Reviewed-by: Eric Blake +Reviewed-by: Thomas Huth +Signed-off-by: Jason Wang +Message-id: 20181204035347.6148-4-jasowang@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit ae4c445c6f38a01504b7040b1e01a69945137b0c) +Signed-off-by: Danilo C. L. de Paula +--- + tests/virtio-net-test.c | 17 +++++++++++------ + 1 file changed, 11 insertions(+), 6 deletions(-) + +diff --git a/tests/virtio-net-test.c b/tests/virtio-net-test.c +index b285a26..ad6968c 100644 +--- a/tests/virtio-net-test.c ++++ b/tests/virtio-net-test.c +@@ -52,17 +52,21 @@ static QVirtioPCIDevice *virtio_net_pci_init(QPCIBus *bus, int slot) + return dev; + } + +-static QOSState *pci_test_start(int socket) ++GCC_FMT_ATTR(1, 2) ++static QOSState *pci_test_start(const char *cmd, ...) + { + QOSState *qs; ++ va_list ap; + const char *arch = qtest_get_arch(); +- const char *cmd = "-netdev socket,fd=%d,id=hs0 -device " +- "virtio-net-pci,netdev=hs0"; + + if (strcmp(arch, "i386") == 0 || strcmp(arch, "x86_64") == 0) { +- qs = qtest_pc_boot(cmd, socket); ++ va_start(ap, cmd); ++ qs = qtest_pc_vboot(cmd, ap); ++ va_end(ap); + } else if (strcmp(arch, "ppc64") == 0) { +- qs = qtest_spapr_boot(cmd, socket); ++ va_start(ap, cmd); ++ qs = qtest_spapr_vboot(cmd, ap); ++ va_end(ap); + } else { + g_printerr("virtio-net tests are only available on x86 or ppc64\n"); + exit(EXIT_FAILURE); +@@ -223,7 +227,8 @@ static void pci_basic(gconstpointer data) + ret = socketpair(PF_UNIX, SOCK_STREAM, 0, sv); + g_assert_cmpint(ret, !=, -1); + +- qs = pci_test_start(sv[1]); ++ qs = pci_test_start("-netdev socket,fd=%d,id=hs0 -device " ++ "virtio-net-pci,netdev=hs0", sv[1]); + dev = virtio_net_pci_init(qs->pcibus, PCI_SLOT); + + rx = (QVirtQueuePCI *)qvirtqueue_setup(&dev->vdev, qs->alloc, 0); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-net-test-add-large-tx-buffer-test.patch b/SOURCES/kvm-virtio-net-test-add-large-tx-buffer-test.patch new file mode 100644 index 0000000..cd3fe1c --- /dev/null +++ b/SOURCES/kvm-virtio-net-test-add-large-tx-buffer-test.patch @@ -0,0 +1,100 @@ +From a7db0339ff184a34168de5c1faff523e180fec03 Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:59:04 +0000 +Subject: [PATCH 09/11] virtio-net-test: add large tx buffer test +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-10-jasowang@redhat.com> +Patchwork-id: 83980 +O-Subject: [RHEL8 qemu-kvm PATCH 9/9] virtio-net-test: add large tx buffer test +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +This test tries to build a packet whose size is greater than INT_MAX +which tries to trigger integer overflow in qemu_net_queue_append_iov() +which may result OOB. + +Signed-off-by: Jason Wang +Reviewed-by: Thomas Huth +Message-id: 20181204035347.6148-6-jasowang@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit 118cafff251318d16e1cfdef9cbf6b7d1e74cdb5) +Signed-off-by: Danilo C. L. de Paula +--- + tests/virtio-net-test.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 46 insertions(+) + +diff --git a/tests/virtio-net-test.c b/tests/virtio-net-test.c +index 9da8f3d..99bc605 100644 +--- a/tests/virtio-net-test.c ++++ b/tests/virtio-net-test.c +@@ -245,6 +245,48 @@ static void pci_basic(gconstpointer data) + g_free(dev); + qtest_shutdown(qs); + } ++ ++static void large_tx(gconstpointer data) ++{ ++ QVirtioPCIDevice *dev; ++ QOSState *qs; ++ QVirtQueuePCI *tx, *rx; ++ QVirtQueue *vq; ++ uint64_t req_addr; ++ uint32_t free_head; ++ size_t alloc_size = (size_t)data / 64; ++ int i; ++ ++ qs = pci_test_start("-netdev hubport,id=hp0,hubid=0 " ++ "-device virtio-net-pci,netdev=hp0"); ++ dev = virtio_net_pci_init(qs->pcibus, PCI_SLOT); ++ ++ rx = (QVirtQueuePCI *)qvirtqueue_setup(&dev->vdev, qs->alloc, 0); ++ tx = (QVirtQueuePCI *)qvirtqueue_setup(&dev->vdev, qs->alloc, 1); ++ ++ driver_init(&dev->vdev); ++ vq = &tx->vq; ++ ++ /* Bypass the limitation by pointing several descriptors to a single ++ * smaller area */ ++ req_addr = guest_alloc(qs->alloc, alloc_size); ++ free_head = qvirtqueue_add(vq, req_addr, alloc_size, false, true); ++ ++ for (i = 0; i < 64; i++) { ++ qvirtqueue_add(vq, req_addr, alloc_size, false, i != 63); ++ } ++ qvirtqueue_kick(&dev->vdev, vq, free_head); ++ ++ qvirtio_wait_used_elem(&dev->vdev, vq, free_head, NULL, ++ QVIRTIO_NET_TIMEOUT_US); ++ ++ qvirtqueue_cleanup(dev->vdev.bus, &tx->vq, qs->alloc); ++ qvirtqueue_cleanup(dev->vdev.bus, &rx->vq, qs->alloc); ++ qvirtio_pci_device_disable(dev); ++ g_free(dev->pdev); ++ g_free(dev); ++ qtest_shutdown(qs); ++} + #endif + + static void hotplug(void) +@@ -269,6 +311,10 @@ int main(int argc, char **argv) + qtest_add_data_func("/virtio/net/pci/basic", send_recv_test, pci_basic); + qtest_add_data_func("/virtio/net/pci/rx_stop_cont", + stop_cont_test, pci_basic); ++ qtest_add_data_func("/virtio/net/pci/large_tx_uint_max", ++ (gconstpointer)UINT_MAX, large_tx); ++ qtest_add_data_func("/virtio/net/pci/large_tx_net_bufsize", ++ (gconstpointer)NET_BUFSIZE, large_tx); + #endif + qtest_add_func("/virtio/net/pci/hotplug", hotplug); + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-net-test-remove-unused-macro.patch b/SOURCES/kvm-virtio-net-test-remove-unused-macro.patch new file mode 100644 index 0000000..093a6a7 --- /dev/null +++ b/SOURCES/kvm-virtio-net-test-remove-unused-macro.patch @@ -0,0 +1,45 @@ +From d6ae7e8d693de7865fe31e58daeabf9cb9a62e8d Mon Sep 17 00:00:00 2001 +From: Xiao Wang +Date: Fri, 11 Jan 2019 07:59:03 +0000 +Subject: [PATCH 08/11] virtio-net-test: remove unused macro +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Xiao Wang +Message-id: <20190111075904.2030-9-jasowang@redhat.com> +Patchwork-id: 83983 +O-Subject: [RHEL8 qemu-kvm PATCH 8/9] virtio-net-test: remove unused macro +Bugzilla: 1636784 +RH-Acked-by: Thomas Huth +RH-Acked-by: Philippe Mathieu-Daudé +RH-Acked-by: Jens Freimann +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Michael S. Tsirkin + +Reviewed-by: Thomas Huth +Reviewed-by: Eric Blake +Signed-off-by: Jason Wang +Message-id: 20181204035347.6148-5-jasowang@redhat.com +Signed-off-by: Peter Maydell +(cherry picked from commit 432a82d72720f71cb965b64836672f19d57ceedb) +Signed-off-by: Danilo C. L. de Paula +--- + tests/virtio-net-test.c | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/tests/virtio-net-test.c b/tests/virtio-net-test.c +index ad6968c..9da8f3d 100644 +--- a/tests/virtio-net-test.c ++++ b/tests/virtio-net-test.c +@@ -24,7 +24,6 @@ + + #define PCI_SLOT_HP 0x06 + #define PCI_SLOT 0x04 +-#define PCI_FN 0x00 + + #define QVIRTIO_NET_TIMEOUT_US (30 * 1000 * 1000) + #define VNET_HDR_SIZE sizeof(struct virtio_net_hdr_mrg_rxbuf) +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-rng-process-pending-requests-on-DRIVER_OK.patch b/SOURCES/kvm-virtio-rng-process-pending-requests-on-DRIVER_OK.patch new file mode 100644 index 0000000..b1e7bc3 --- /dev/null +++ b/SOURCES/kvm-virtio-rng-process-pending-requests-on-DRIVER_OK.patch @@ -0,0 +1,70 @@ +From 28648492db9520d5914ea20eaa8425aae3d1627d Mon Sep 17 00:00:00 2001 +From: Pankaj Gupta +Date: Fri, 13 Jul 2018 12:52:28 +0200 +Subject: [PATCH 216/268] virtio-rng: process pending requests on DRIVER_OK + +RH-Author: Pankaj Gupta +Message-id: <20180713125228.14458-1-pagupta@redhat.com> +Patchwork-id: 81347 +O-Subject: [RHEL7.6 qemu-kvm-rhev PATCH] virtio-rng: process pending requests on DRIVER_OK +Bugzilla: 1576743 +RH-Acked-by: Stefan Hajnoczi +RH-Acked-by: Cornelia Huck +RH-Acked-by: Miroslav Rezanina + +virtio-rng device causes old guest kernels(2.6.32) to hang on latest qemu. +The driver attempts to read from the virtio-rng device too early in it's +initialization. Qemu detects guest is not ready and returns, resulting in +hang. + +To fix handle pending requests when guest is running and driver status is +set to 'VIRTIO_CONFIG_S_DRIVER_OK'. + +CC: qemu-stable@nongnu.org +Reported-by: Sergio lopez +Signed-off-by: Stefan Hajnoczi +Signed-off-by: Pankaj Gupta +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 5d9c9ea22ab4f3b3ee497523e34b6f4d3281f62d) +Signed-off-by: Pankaj Gupta +Signed-off-by: Miroslav Rezanina +--- + hw/virtio/virtio-rng.c | 14 ++++++++++++++ + 1 file changed, 14 insertions(+) + +diff --git a/hw/virtio/virtio-rng.c b/hw/virtio/virtio-rng.c +index 289bbca..855f1b4 100644 +--- a/hw/virtio/virtio-rng.c ++++ b/hw/virtio/virtio-rng.c +@@ -156,6 +156,19 @@ static void check_rate_limit(void *opaque) + vrng->activate_timer = true; + } + ++static void virtio_rng_set_status(VirtIODevice *vdev, uint8_t status) ++{ ++ VirtIORNG *vrng = VIRTIO_RNG(vdev); ++ ++ if (!vdev->vm_running) { ++ return; ++ } ++ vdev->status = status; ++ ++ /* Something changed, try to process buffers */ ++ virtio_rng_process(vrng); ++} ++ + static void virtio_rng_device_realize(DeviceState *dev, Error **errp) + { + VirtIODevice *vdev = VIRTIO_DEVICE(dev); +@@ -261,6 +274,7 @@ static void virtio_rng_class_init(ObjectClass *klass, void *data) + vdc->realize = virtio_rng_device_realize; + vdc->unrealize = virtio_rng_device_unrealize; + vdc->get_features = get_features; ++ vdc->set_status = virtio_rng_set_status; + } + + static const TypeInfo virtio_rng_info = { +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-scsi-fix-hotplug-reset-vs-event-race.patch b/SOURCES/kvm-virtio-scsi-fix-hotplug-reset-vs-event-race.patch new file mode 100644 index 0000000..5bd00cc --- /dev/null +++ b/SOURCES/kvm-virtio-scsi-fix-hotplug-reset-vs-event-race.patch @@ -0,0 +1,77 @@ +From b872baa52f5e8e8c84b3a19bb6cf5b6e92106fb9 Mon Sep 17 00:00:00 2001 +From: Stefan Hajnoczi +Date: Tue, 24 Jul 2018 15:13:08 +0200 +Subject: [PATCH 264/268] virtio-scsi: fix hotplug ->reset() vs event race + +RH-Author: Stefan Hajnoczi +Message-id: <20180724151308.20500-3-stefanha@redhat.com> +Patchwork-id: 81486 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 2/2] virtio-scsi: fix hotplug ->reset() vs event race +Bugzilla: 1607891 +RH-Acked-by: Igor Mammedov +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Cornelia Huck + +There is a race condition during hotplug when iothread is used. It +occurs because virtio-scsi may be processing command queues in the +iothread while the monitor performs SCSI device hotplug. + +When a SCSI device is hotplugged the HotplugHandler->plug() callback is +invoked and virtio-scsi emits a rescan event to the guest. + +If the guest submits a SCSI command at this point then it may be +cancelled before hotplug completes. This happens because ->reset() is +called by hw/core/qdev.c:device_set_realized() after +HotplugHandler->plug() has been called and +hw/scsi/scsi-disk.c:scsi_disk_reset() purges all requests. + +This patch uses the new HotplugHandler->post_plug() callback to emit the +rescan event after ->reset(). This eliminates the race conditions where +requests could be cancelled. + +Reported-by: l00284672 +Cc: Paolo Bonzini +Cc: Fam Zheng +Signed-off-by: Stefan Hajnoczi +Message-Id: <20180716083732.3347-3-stefanha@redhat.com> +Signed-off-by: Paolo Bonzini +(cherry picked from commit 8449bcf94986156a1476d6647c75ec1ce3db64d0) +Signed-off-by: Stefan Hajnoczi +Signed-off-by: Miroslav Rezanina +--- + hw/scsi/virtio-scsi.c | 11 ++++++++++- + 1 file changed, 10 insertions(+), 1 deletion(-) + +diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c +index 9f754c4..52a3c1d 100644 +--- a/hw/scsi/virtio-scsi.c ++++ b/hw/scsi/virtio-scsi.c +@@ -806,8 +806,16 @@ static void virtio_scsi_hotplug(HotplugHandler *hotplug_dev, DeviceState *dev, + virtio_scsi_acquire(s); + blk_set_aio_context(sd->conf.blk, s->ctx); + virtio_scsi_release(s); +- + } ++} ++ ++/* Announce the new device after it has been plugged */ ++static void virtio_scsi_post_hotplug(HotplugHandler *hotplug_dev, ++ DeviceState *dev) ++{ ++ VirtIODevice *vdev = VIRTIO_DEVICE(hotplug_dev); ++ VirtIOSCSI *s = VIRTIO_SCSI(vdev); ++ SCSIDevice *sd = SCSI_DEVICE(dev); + + if (virtio_vdev_has_feature(vdev, VIRTIO_SCSI_F_HOTPLUG)) { + virtio_scsi_acquire(s); +@@ -977,6 +985,7 @@ static void virtio_scsi_class_init(ObjectClass *klass, void *data) + vdc->start_ioeventfd = virtio_scsi_dataplane_start; + vdc->stop_ioeventfd = virtio_scsi_dataplane_stop; + hc->plug = virtio_scsi_hotplug; ++ hc->post_plug = virtio_scsi_post_hotplug; + hc->unplug = virtio_scsi_hotunplug; + } + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-virtio-support-setting-memory-region-based-host-noti.patch b/SOURCES/kvm-virtio-support-setting-memory-region-based-host-noti.patch new file mode 100644 index 0000000..3c455d9 --- /dev/null +++ b/SOURCES/kvm-virtio-support-setting-memory-region-based-host-noti.patch @@ -0,0 +1,129 @@ +From cc1a69d28dd21da529d0d2f7a34547c03277243c Mon Sep 17 00:00:00 2001 +From: "plai@redhat.com" +Date: Thu, 21 Jun 2018 18:54:37 +0200 +Subject: [PATCH 158/268] virtio: support setting memory region based host + notifier + +RH-Author: plai@redhat.com +Message-id: <1529607285-9942-3-git-send-email-plai@redhat.com> +Patchwork-id: 80937 +O-Subject: [RHEL7.6 PATCH BZ 1526645 02/10] virtio: support setting memory region based host notifier +Bugzilla: 1526645 +RH-Acked-by: Michael S. Tsirkin +RH-Acked-by: Maxime Coquelin +RH-Acked-by: Laurent Vivier + +From: Tiwei Bie + +This patch introduces the support for setting memory region +based host notifiers for virtio device. This is helpful when +using a hardware accelerator for a virtio device, because +hardware heavily depends on the notification, this will allow +the guest driver in the VM to notify the hardware directly. + +Signed-off-by: Tiwei Bie +Reviewed-by: Michael S. Tsirkin +Signed-off-by: Michael S. Tsirkin +(cherry picked from commit 6f80e6170ede13605817e5c0ca73db0de7bdf261) +Signed-off-by: Paul Lai +Signed-off-by: Miroslav Rezanina +--- + hw/virtio/virtio-pci.c | 22 ++++++++++++++++++++++ + hw/virtio/virtio.c | 13 +++++++++++++ + include/hw/virtio/virtio-bus.h | 2 ++ + include/hw/virtio/virtio.h | 2 ++ + 4 files changed, 39 insertions(+) + +diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c +index 92bdc9e..070ad0f 100644 +--- a/hw/virtio/virtio-pci.c ++++ b/hw/virtio/virtio-pci.c +@@ -1037,6 +1037,27 @@ assign_error: + return r; + } + ++static int virtio_pci_set_host_notifier_mr(DeviceState *d, int n, ++ MemoryRegion *mr, bool assign) ++{ ++ VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d); ++ int offset; ++ ++ if (n >= VIRTIO_QUEUE_MAX || !virtio_pci_modern(proxy) || ++ virtio_pci_queue_mem_mult(proxy) != memory_region_size(mr)) { ++ return -1; ++ } ++ ++ if (assign) { ++ offset = virtio_pci_queue_mem_mult(proxy) * n; ++ memory_region_add_subregion_overlap(&proxy->notify.mr, offset, mr, 1); ++ } else { ++ memory_region_del_subregion(&proxy->notify.mr, mr); ++ } ++ ++ return 0; ++} ++ + static void virtio_pci_vmstate_change(DeviceState *d, bool running) + { + VirtIOPCIProxy *proxy = to_virtio_pci_proxy(d); +@@ -2652,6 +2673,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data) + k->has_extra_state = virtio_pci_has_extra_state; + k->query_guest_notifiers = virtio_pci_query_guest_notifiers; + k->set_guest_notifiers = virtio_pci_set_guest_notifiers; ++ k->set_host_notifier_mr = virtio_pci_set_host_notifier_mr; + k->vmstate_change = virtio_pci_vmstate_change; + k->pre_plugged = virtio_pci_pre_plugged; + k->device_plugged = virtio_pci_device_plugged; +diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c +index 006d3d1..1debb01 100644 +--- a/hw/virtio/virtio.c ++++ b/hw/virtio/virtio.c +@@ -2454,6 +2454,19 @@ EventNotifier *virtio_queue_get_host_notifier(VirtQueue *vq) + return &vq->host_notifier; + } + ++int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, ++ MemoryRegion *mr, bool assign) ++{ ++ BusState *qbus = qdev_get_parent_bus(DEVICE(vdev)); ++ VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); ++ ++ if (k->set_host_notifier_mr) { ++ return k->set_host_notifier_mr(qbus->parent, n, mr, assign); ++ } ++ ++ return -1; ++} ++ + void virtio_device_set_child_bus_name(VirtIODevice *vdev, char *bus_name) + { + g_free(vdev->bus_name); +diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h +index ced3d2d..7fec9dc 100644 +--- a/include/hw/virtio/virtio-bus.h ++++ b/include/hw/virtio/virtio-bus.h +@@ -52,6 +52,8 @@ typedef struct VirtioBusClass { + bool (*has_extra_state)(DeviceState *d); + bool (*query_guest_notifiers)(DeviceState *d); + int (*set_guest_notifiers)(DeviceState *d, int nvqs, bool assign); ++ int (*set_host_notifier_mr)(DeviceState *d, int n, ++ MemoryRegion *mr, bool assign); + void (*vmstate_change)(DeviceState *d, bool running); + /* + * Expose the features the transport layer supports before +diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h +index 098bdaa..9c1fa07 100644 +--- a/include/hw/virtio/virtio.h ++++ b/include/hw/virtio/virtio.h +@@ -239,6 +239,8 @@ void virtio_queue_set_align(VirtIODevice *vdev, int n, int align); + void virtio_queue_notify(VirtIODevice *vdev, int n); + uint16_t virtio_queue_vector(VirtIODevice *vdev, int n); + void virtio_queue_set_vector(VirtIODevice *vdev, int n, uint16_t vector); ++int virtio_queue_set_host_notifier_mr(VirtIODevice *vdev, int n, ++ MemoryRegion *mr, bool assign); + int virtio_set_status(VirtIODevice *vdev, uint8_t val); + void virtio_reset(void *opaque); + void virtio_update_irq(VirtIODevice *vdev); +-- +1.8.3.1 + diff --git a/SOURCES/kvm-vnc-call-sasl_server_init-only-when-required.patch b/SOURCES/kvm-vnc-call-sasl_server_init-only-when-required.patch new file mode 100644 index 0000000..e5418b1 --- /dev/null +++ b/SOURCES/kvm-vnc-call-sasl_server_init-only-when-required.patch @@ -0,0 +1,89 @@ +From 7bc411803771ae9c18290eeb55480bab5cd66b93 Mon Sep 17 00:00:00 2001 +From: =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= +Date: Tue, 2 Oct 2018 12:34:03 +0100 +Subject: [PATCH 01/49] vnc: call sasl_server_init() only when required +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +RH-Author: Marc-André Lureau +Message-id: <20181002123403.20747-2-marcandre.lureau@redhat.com> +Patchwork-id: 82356 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/1] vnc: call sasl_server_init() only when required +Bugzilla: 1609327 +RH-Acked-by: Daniel P. Berrange +RH-Acked-by: Thomas Huth +RH-Acked-by: Danilo de Paula + +VNC server is calling sasl_server_init() during startup of QEMU, even +if SASL auth has not been enabled. + +This may create undesirable warnings like "Could not find keytab file: +/etc/qemu/krb5.tab" when the user didn't configure SASL on host and +started VNC server. + +Instead, only initialize SASL when needed. Note that HMP/QMP "change +vnc" calls vnc_display_open() again, which will initialize SASL if +needed. + +Fix assignment in if condition, while touching this code. + +Related to: +https://bugzilla.redhat.com/show_bug.cgi?id=1609327 + +Signed-off-by: Marc-André Lureau +Reviewed-by: Daniel P. Berrangé +Message-id: 20180907063634.359-1-marcandre.lureau@redhat.com +Signed-off-by: Gerd Hoffmann + +(cherry picked from commit b5dc0d7d565048fcf2767060261d8385805aced1) + +BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1609327 +Brew: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=18601393 +Signed-off-by: Danilo C. L. de Paula + +Conflicts: + ui/vnc.c + Due to "qemu"->"qemu-kvm" rename. + +Signed-off-by: Marc-André Lureau +--- + ui/vnc.c | 15 ++++++++------- + 1 file changed, 8 insertions(+), 7 deletions(-) + +diff --git a/ui/vnc.c b/ui/vnc.c +index 0c3011b..86c6762 100644 +--- a/ui/vnc.c ++++ b/ui/vnc.c +@@ -3869,9 +3869,6 @@ void vnc_display_open(const char *id, Error **errp) + bool reverse = false; + const char *credid; + bool sasl = false; +-#ifdef CONFIG_VNC_SASL +- int saslErr; +-#endif + int acl = 0; + int lock_key_sync = 1; + int key_delay_ms; +@@ -4045,10 +4042,14 @@ void vnc_display_open(const char *id, Error **errp) + trace_vnc_auth_init(vd, 1, vd->ws_auth, vd->ws_subauth); + + #ifdef CONFIG_VNC_SASL +- if ((saslErr = sasl_server_init(NULL, "qemu-kvm")) != SASL_OK) { +- error_setg(errp, "Failed to initialize SASL auth: %s", +- sasl_errstring(saslErr, NULL, NULL)); +- goto fail; ++ if (sasl) { ++ int saslErr = sasl_server_init(NULL, "qemu-kvm"); ++ ++ if (saslErr != SASL_OK) { ++ error_setg(errp, "Failed to initialize SASL auth: %s", ++ sasl_errstring(saslErr, NULL, NULL)); ++ goto fail; ++ } + } + #endif + vd->lock_key_sync = lock_key_sync; +-- +1.8.3.1 + diff --git a/SOURCES/kvm-x86-host-phys-bits-limit-option.patch b/SOURCES/kvm-x86-host-phys-bits-limit-option.patch new file mode 100644 index 0000000..3b973bc --- /dev/null +++ b/SOURCES/kvm-x86-host-phys-bits-limit-option.patch @@ -0,0 +1,93 @@ +From b6a062c64f9639558a88f46edc3dd76b54b26bb5 Mon Sep 17 00:00:00 2001 +From: Eduardo Habkost +Date: Thu, 13 Dec 2018 15:51:59 +0000 +Subject: [PATCH 1/5] x86: host-phys-bits-limit option + +RH-Author: Eduardo Habkost +Message-id: <20181213155200.20300-2-ehabkost@redhat.com> +Patchwork-id: 83479 +O-Subject: [RHEL8/rhel qemu-kvm PATCH 1/2] x86: host-phys-bits-limit option +Bugzilla: 1598284 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Pankaj Gupta +RH-Acked-by: Bandan Das +RH-Acked-by: Paolo Bonzini + +Some downstream distributions of QEMU set host-phys-bits=on by +default. This worked very well for most use cases, because +phys-bits really didn't have huge consequences. The only +difference was on the CPUID data seen by guests, and on the +handling of reserved bits. + +This changed in KVM commit 855feb673640 ("KVM: MMU: Add 5 level +EPT & Shadow page table support"). Now choosing a large +phys-bits value for a VM has bigger impact: it will make KVM use +5-level EPT even when it's not really necessary. This means +using the host phys-bits value may not be the best choice. + +Management software could address this problem by manually +configuring phys-bits depending on the size of the VM and the +amount of MMIO address space required for hotplug. But this is +not trivial to implement. + +However, there's another workaround that would work for most +cases: keep using the host phys-bits value, but only if it's +smaller than 48. This patch makes this possible by introducing a +new "-cpu" option: "host-phys-bits-limit". Management software +or users can make sure they will always use 4-level EPT using: +"host-phys-bits=on,host-phys-bits-limit=48". + +This behavior is still not enabled by default because QEMU +doesn't enable host-phys-bits=on by default. But users, +management software, or downstream distributions may choose to +change their defaults using the new option. + +Signed-off-by: Eduardo Habkost +Message-Id: <20181211192527.13254-1-ehabkost@redhat.com> +Signed-off-by: Eduardo Habkost +Signed-off-by: Danilo C. L. de Paula +--- + target/i386/cpu.c | 5 +++++ + target/i386/cpu.h | 3 +++ + 2 files changed, 8 insertions(+) + +diff --git a/target/i386/cpu.c b/target/i386/cpu.c +index a44912c..c37cd1e 100644 +--- a/target/i386/cpu.c ++++ b/target/i386/cpu.c +@@ -4826,6 +4826,10 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp) + if (cpu->host_phys_bits) { + /* The user asked for us to use the host physical bits */ + cpu->phys_bits = host_phys_bits; ++ if (cpu->host_phys_bits_limit && ++ cpu->phys_bits > cpu->host_phys_bits_limit) { ++ cpu->phys_bits = cpu->host_phys_bits_limit; ++ } + } + + /* Print a warning if the user set it to a value that's not the +@@ -5377,6 +5381,7 @@ static Property x86_cpu_properties[] = { + DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true), + DEFINE_PROP_UINT32("phys-bits", X86CPU, phys_bits, 0), + DEFINE_PROP_BOOL("host-phys-bits", X86CPU, host_phys_bits, false), ++ DEFINE_PROP_UINT8("host-phys-bits-limit", X86CPU, host_phys_bits_limit, 0), + DEFINE_PROP_BOOL("fill-mtrr-mask", X86CPU, fill_mtrr_mask, true), + DEFINE_PROP_UINT32("level", X86CPU, env.cpuid_level, UINT32_MAX), + DEFINE_PROP_UINT32("xlevel", X86CPU, env.cpuid_xlevel, UINT32_MAX), +diff --git a/target/i386/cpu.h b/target/i386/cpu.h +index 4a3ef4b..58d5430 100644 +--- a/target/i386/cpu.h ++++ b/target/i386/cpu.h +@@ -1418,6 +1418,9 @@ struct X86CPU { + /* if true override the phys_bits value with a value read from the host */ + bool host_phys_bits; + ++ /* if set, limit maximum value for phys_bits when host_phys_bits is true */ ++ uint8_t host_phys_bits_limit; ++ + /* Stop SMI delivery for migration compatibility with old machines */ + bool kvm_no_smi_migration; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm-x86.conf b/SOURCES/kvm-x86.conf new file mode 100644 index 0000000..3f7842a --- /dev/null +++ b/SOURCES/kvm-x86.conf @@ -0,0 +1,12 @@ +# Setting modprobe kvm_intel/kvm_amd nested = 1 +# only enables Nested Virtualization until the next reboot or +# module reload. Uncomment the option applicable +# to your system below to enable the feature permanently. +# +# User changes in this file are preserved across upgrades. +# +# For Intel +#options kvm_intel nested=1 +# +# For AMD +#options kvm_amd nested=1 diff --git a/SOURCES/kvm-xhci-fix-guest-triggerable-assert.patch b/SOURCES/kvm-xhci-fix-guest-triggerable-assert.patch new file mode 100644 index 0000000..4af6eba --- /dev/null +++ b/SOURCES/kvm-xhci-fix-guest-triggerable-assert.patch @@ -0,0 +1,45 @@ +From d6ab4b2c2b8dac019452a9237def64ac7c9803a1 Mon Sep 17 00:00:00 2001 +From: Gerd Hoffmann +Date: Wed, 4 Jul 2018 09:00:54 +0200 +Subject: [PATCH 186/268] xhci: fix guest-triggerable assert + +RH-Author: Gerd Hoffmann +Message-id: <20180704090054.5101-2-kraxel@redhat.com> +Patchwork-id: 81224 +O-Subject: [RHEL-7.6 qemu-kvm-rhev PATCH 1/1] xhci: fix guest-triggerable assert +Bugzilla: 1594135 +RH-Acked-by: Dr. David Alan Gilbert +RH-Acked-by: Laurent Vivier +RH-Acked-by: Markus Armbruster + +Set xhci into error state instead of throwing a core dump. + +Signed-off-by: Gerd Hoffmann +Message-id: 20180702162752.29233-1-kraxel@redhat.com +(cherry picked from commit 8f36ec708834dfad58af6feb0b69bb0be6077f0f) +Signed-off-by: Miroslav Rezanina +--- + hw/usb/hcd-xhci.c | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/hw/usb/hcd-xhci.c b/hw/usb/hcd-xhci.c +index 181e803..45fcce3 100644 +--- a/hw/usb/hcd-xhci.c ++++ b/hw/usb/hcd-xhci.c +@@ -1954,7 +1954,12 @@ static void xhci_kick_epctx(XHCIEPContext *epctx, unsigned int streamid) + for (i = 0; i < length; i++) { + TRBType type; + type = xhci_ring_fetch(xhci, ring, &xfer->trbs[i], NULL); +- assert(type); ++ if (!type) { ++ xhci_die(xhci); ++ xhci_ep_free_xfer(xfer); ++ epctx->kick_active--; ++ return; ++ } + } + xfer->streamid = streamid; + +-- +1.8.3.1 + diff --git a/SOURCES/kvm.conf b/SOURCES/kvm.conf new file mode 100644 index 0000000..24e60e9 --- /dev/null +++ b/SOURCES/kvm.conf @@ -0,0 +1,3 @@ +# +# User changes in this file are preserved across upgrades. +# diff --git a/SOURCES/qemu-ga.sysconfig b/SOURCES/qemu-ga.sysconfig new file mode 100644 index 0000000..67bad0c --- /dev/null +++ b/SOURCES/qemu-ga.sysconfig @@ -0,0 +1,19 @@ +# This is a systemd environment file, not a shell script. +# It provides settings for "/lib/systemd/system/qemu-guest-agent.service". + +# Comma-separated blacklist of RPCs to disable, or empty list to enable all. +# +# You can get the list of RPC commands using "qemu-ga --blacklist='?'". +# There should be no spaces between commas and commands in the blacklist. +BLACKLIST_RPC=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-file-seek,guest-file-flush,guest-exec,guest-exec-status + +# Fsfreeze hook script specification. +# +# FSFREEZE_HOOK_PATHNAME=/dev/null : disables the feature. +# +# FSFREEZE_HOOK_PATHNAME=/path/to/executable : enables the feature with the +# specified binary or shell script. +# +# FSFREEZE_HOOK_PATHNAME= : enables the feature with the +# default value (invoke "qemu-ga --help" to interrogate). +FSFREEZE_HOOK_PATHNAME=/etc/qemu-ga/fsfreeze-hook diff --git a/SOURCES/qemu-guest-agent.service b/SOURCES/qemu-guest-agent.service new file mode 100644 index 0000000..b33e951 --- /dev/null +++ b/SOURCES/qemu-guest-agent.service @@ -0,0 +1,20 @@ +[Unit] +Description=QEMU Guest Agent +BindsTo=dev-virtio\x2dports-org.qemu.guest_agent.0.device +After=dev-virtio\x2dports-org.qemu.guest_agent.0.device +IgnoreOnIsolate=True + +[Service] +UMask=0077 +EnvironmentFile=/etc/sysconfig/qemu-ga +ExecStart=/usr/bin/qemu-ga \ + --method=virtio-serial \ + --path=/dev/virtio-ports/org.qemu.guest_agent.0 \ + --blacklist=${BLACKLIST_RPC} \ + -F${FSFREEZE_HOOK_PATHNAME} +StandardError=syslog +Restart=always +RestartSec=0 + +[Install] +WantedBy=dev-virtio\x2dports-org.qemu.guest_agent.0.device diff --git a/SOURCES/qemu-pr-helper.service b/SOURCES/qemu-pr-helper.service new file mode 100644 index 0000000..a1d27b0 --- /dev/null +++ b/SOURCES/qemu-pr-helper.service @@ -0,0 +1,15 @@ +[Unit] +Description=Persistent Reservation Daemon for QEMU + +[Service] +WorkingDirectory=/tmp +Type=simple +ExecStart=/usr/bin/qemu-pr-helper +PrivateTmp=yes +ProtectSystem=strict +ReadWritePaths=/var/run +RestrictAddressFamilies=AF_UNIX +Restart=always +RestartSec=0 + +[Install] diff --git a/SOURCES/qemu-pr-helper.socket b/SOURCES/qemu-pr-helper.socket new file mode 100644 index 0000000..9d7c3e5 --- /dev/null +++ b/SOURCES/qemu-pr-helper.socket @@ -0,0 +1,9 @@ +[Unit] +Description=Persistent Reservation Daemon for QEMU + +[Socket] +ListenStream=/run/qemu-pr-helper.sock +SocketMode=0600 + +[Install] +WantedBy=multi-user.target diff --git a/SOURCES/udev-kvm-check.c b/SOURCES/udev-kvm-check.c new file mode 100644 index 0000000..cb0ecba --- /dev/null +++ b/SOURCES/udev-kvm-check.c @@ -0,0 +1,172 @@ +/* + * udev-kvm-check.c + * + * Copyright 2018 Red Hat, Inc. + * + * This is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software Foundation, + * Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + * + */ + +#include +#include +#include +#include +#include + +#define DEFAULT 0 +#define FACILITY "kvm" +#define SYSCONFIG_KVM "/etc/sysconfig/kvm" + +#define COUNT_MSG \ + "%d %s now active" + +#define SUBSCRIPTION_MSG \ + "%d %s now active; your Red Hat Enterprise Linux subscription" \ + " limit is %d guests. Please review your Red Hat Enterprise Linux" \ + " subscription agreement or contact your Red Hat" \ + " support representative for more information. You" \ + " may review the Red Hat Enterprise subscription" \ + " limits at http://www.redhat.com/rhel-virt-limits" + +int get_threshold_from_file(FILE *fp) +{ + static const char key[] = "THRESHOLD="; + int pos = 0; + int thres; + int ch; + +start: + /* State START - at beginning of line, search for beginning of "THRESHOLD=" + * string. + */ + ch = getc(fp); + if (ch == EOF) { + return DEFAULT; + } + if (isspace(ch)) { + goto start; + } + if (ch == 'T') { + pos = 1; + goto key; + } + goto eol; + +eol: + /* State EOL - loop until end of line */ + ch = getc(fp); + if (ch == EOF) { + return DEFAULT; + } + if (ch == '\n') { + goto start; + } + goto eol; + +key: + /* State KEY - match "THRESHOLD=" string, go to THRESHOLD if found */ + ch = getc(fp); + if (ch == EOF) { + return DEFAULT; + } + if (ch == key[pos]) { + pos++; + if (key[pos] == 0) { + goto threshold; + } else { + goto key; + } + } + goto eol; + +threshold: + /* State THRESHOLD - parse number using fscanf, expect comment or space + * or EOL. + */ + ch = getc(fp); + if (ch == EOF) { + return DEFAULT; + } + if (!isdigit(ch)) { + goto eol; + } + ungetc(ch, fp); + if (fscanf(fp, "%d", &thres) != 1) { + return DEFAULT; + } + ch = getc(fp); + if (ch == '#' || ch == EOF || ch == '\n' || isspace(ch)) { + return thres; + } + goto eol; +} + +int get_threshold() +{ + FILE *fp = fopen(SYSCONFIG_KVM, "r"); + int val; + + if (!fp) { + return DEFAULT; + } + + val = get_threshold_from_file(fp); + fclose (fp); + return val; +} + +const char *guest(int count) +{ + return (count == 1 ? "guest" : "guests"); +} + +void emit_count_message(int count) +{ + openlog(FACILITY, LOG_CONS, LOG_USER); + syslog(LOG_INFO, COUNT_MSG, count, guest(count)); + closelog(); +} + +void emit_subscription_message(int count, int threshold) +{ + openlog(FACILITY, LOG_CONS, LOG_USER); + syslog(LOG_WARNING, SUBSCRIPTION_MSG, count, guest(count), threshold); + closelog(); +} + +int main(int argc, char **argv) +{ + int count, threshold; + + if (argc < 3) + exit(1); + + count = atoi(argv[1]); + threshold = get_threshold(); + + if (!strcmp(argv[2], "create")) { + if (threshold == 0) { + emit_count_message(count); + } else if (count > threshold) { + emit_subscription_message(count, threshold); + } + } else { + if (count >= threshold) { + emit_count_message(count); + } + } + + return 0; +} diff --git a/SOURCES/vhost.conf b/SOURCES/vhost.conf new file mode 100644 index 0000000..68d6d7f --- /dev/null +++ b/SOURCES/vhost.conf @@ -0,0 +1,3 @@ +# Increase default vhost memory map limit to match +# KVM's memory slot limit +options vhost max_mem_regions=509 diff --git a/SPECS/qemu-kvm.spec b/SPECS/qemu-kvm.spec new file mode 100644 index 0000000..a240b64 --- /dev/null +++ b/SPECS/qemu-kvm.spec @@ -0,0 +1,3276 @@ +%global SLOF_gittagdate 20170724 +%global SLOF_gittagcommit 89f519f + +%global have_usbredir 1 +%global have_spice 1 +%global have_opengl 1 +%global have_fdt 0 +%global have_gluster 1 +%global have_kvm_setup 0 +%global have_memlock_limits 0 +%global have_vxhs 0 + +%ifnarch %{ix86} x86_64 + %global have_usbredir 0 +%endif + +%ifnarch s390x + %global have_librdma 1 +%else + %global have_librdma 0 +%endif + +%ifarch %{ix86} + %global kvm_target i386 +%endif +%ifarch x86_64 + %global kvm_target x86_64 + %global have_vxhs 1 +%else + %global have_spice 0 + %global have_opengl 0 + %global have_gluster 0 +%endif +%ifarch %{power64} + %global kvm_target ppc64 + %global have_fdt 1 + %global have_kvm_setup 1 + %global have_memlock_limits 1 +%endif +%ifarch s390x + %global kvm_target s390x +%endif +%ifarch ppc + %global kvm_target ppc + %global have_fdt 1 +%endif +%ifarch aarch64 + %global kvm_target aarch64 + %global have_fdt 1 +%endif + +#Versions of various parts: + +%global requires_all_modules \ +Requires: %{name}-block-curl = %{epoch}:%{version}-%{release} \ +%if %{have_gluster} \ +Requires: %{name}-block-gluster = %{epoch}:%{version}-%{release} \ +%endif \ +Requires: %{name}-block-iscsi = %{epoch}:%{version}-%{release} \ +Requires: %{name}-block-rbd = %{epoch}:%{version}-%{release} \ +Requires: %{name}-block-ssh = %{epoch}:%{version}-%{release} + +# Macro to properly setup RHEL/RHEV conflict handling +%define rhev_ma_conflicts() \ +Obsoletes: %1-ma \ +Obsoletes: %1-rhev + +Summary: QEMU is a machine emulator and virtualizer +Name: qemu-kvm +Version: 2.12.0 +Release: 63%{?dist} +# Epoch because we pushed a qemu-1.0 package. AIUI this can't ever be dropped +Epoch: 15 +License: GPLv2 and GPLv2+ and CC-BY +Group: Development/Tools +URL: http://www.qemu.org/ +ExclusiveArch: x86_64 %{power64} aarch64 s390x + + +Source0: http://wiki.qemu.org/download/qemu-2.12.0.tar.xz + +# KSM control scripts +Source4: ksm.service +Source5: ksm.sysconfig +Source6: ksmctl.c +Source7: ksmtuned.service +Source8: ksmtuned +Source9: ksmtuned.conf +Source10: qemu-guest-agent.service +Source11: 99-qemu-guest-agent.rules +Source12: bridge.conf +Source13: qemu-ga.sysconfig +Source21: kvm-setup +Source22: kvm-setup.service +Source23: 85-kvm.preset +Source26: vhost.conf +Source27: kvm.conf +Source28: 95-kvm-memlock.conf +Source30: kvm-s390x.conf +Source31: kvm-x86.conf +Source32: qemu-pr-helper.service +Source33: qemu-pr-helper.socket +Source34: 81-kvm-rhel.rules +Source35: udev-kvm-check.c +Source36: README.tests + + + +Patch0001: 0001-Initial-redhat-build.patch +Patch0002: 0002-Enable-disable-devices-for-RHEL-7.patch +Patch0003: 0003-Add-RHEL-7-machine-types.patch +Patch0004: 0004-block-vxhs-modularize-VXHS-via-g_module.patch +Patch0005: 0005-Use-kvm-by-default.patch +Patch0006: 0006-vfio-cap-number-of-devices-that-can-be-assigned.patch +Patch0007: 0007-Add-support-statement-to-help-output.patch +Patch0008: 0008-globally-limit-the-maximum-number-of-CPUs.patch +Patch0009: 0009-Add-support-for-simpletrace.patch +Patch0010: 0010-Use-qemu-kvm-in-documentation-instead-of-qemu-system.patch +Patch0011: 0011-usb-xhci-Fix-PCI-capability-order.patch +Patch0012: 0012-virtio-scsi-Reject-scsi-cd-if-data-plane-enabled-RHE.patch +Patch0013: 0013-AArch64-Enable-CONFIG_FW_CFG_DMA-for-aarch64.patch +# For bz#1508137 - [IBM 8.0 FEAT] KVM: Interactive Bootloader (qemu) +Patch0016: 0016-pc-bios-s390-ccw-struct-tpi_info-must-be-declared-as.patch +# For bz#1569675 - Backwards compatibility of pc-*-rhel7.5.0 and older machine-types +Patch0020: 0020-pc-pc-rhel75.5.0-compat-code.patch +# For bz#1571145 - qemu-kvm segfaults on RHEL 8 when run guestfsd under TCG +Patch0022: 0022-tcg-workaround-branch-instruction-overflow-in-tcg_ou.patch +# For bz#1508137 - [IBM 8.0 FEAT] KVM: Interactive Bootloader (qemu) +Patch0025: 0025-s390-ccw-force-diag-308-subcode-to-unsigned-long.patch +Patch0026: 0026-pc-bios-s390-ccw-size_t-should-be-unsigned.patch +Patch0027: 0027-pc-bios-s390-ccw-rename-MAX_TABLE_ENTRIES-to-MAX_BOO.patch +Patch0028: 0028-pc-bios-s390-ccw-fix-loadparm-initialization-and-int.patch +Patch0029: 0029-pc-bios-s390-ccw-fix-non-sequential-boot-entries-eck.patch +Patch0030: 0030-pc-bios-s390-ccw-fix-non-sequential-boot-entries-enu.patch +# For bz#1568600 - pc-i440fx-rhel7.6.0 and pc-q35-rhel7.6.0 machine types (x86) +Patch0031: 0031-pc-rhel7.6.0-machine-types.patch +# For bz#1513558 - Remove RHEL6 machine types +Patch0032: 0032-Remove-rhel6-machine-types.patch +Patch0033: 0033-Remove-rhel6_ctrl_guest_workaround.patch +Patch0034: 0034-Remove-SeaBIOS-shadowing.patch +Patch0035: 0035-Remove-ich9_uhci123_irqpin_override.patch +# For bz#1570029 - [IBM 8.0 FEAT] KVM: 3270 Connectivity - qemu part +Patch0036: 0036-s390x-css-disabled-subchannels-cannot-be-status-pend.patch +Patch0037: 0037-virtio-ccw-common-reset-handler.patch +Patch0038: 0038-s390x-ccw-make-sure-all-ccw-devices-are-properly-res.patch +Patch0039: 0039-s390x-Re-enable-CONFIG_TERMINAL3270.patch +# For bz#1585651 - RHEL 7.6 new pseries machine type (ppc64le) +Patch0041: 0041-redhat-define-pseries-rhel7.6.0-machine-types.patch +# For bz#1592337 - [IBM 8.0 FEAT] KVM: CPU Model z14 ZR1 (qemu-kvm) +Patch0042: 0042-s390x-cpumodels-add-z14-Model-ZR1.patch +# For bz#1590511 - Fails to start guest with Intel vGPU device +Patch0043: kvm-vfio-pci-Default-display-option-to-off.patch +# For bz#1571533 - Convert qemu-kvm python scripts to python3 +Patch0044: kvm-python-futurize-f-libfuturize.fixes.fix_print_with_i.patch +# For bz#1571533 - Convert qemu-kvm python scripts to python3 +Patch0045: kvm-python-futurize-f-lib2to3.fixes.fix_except.patch +# For bz#1599593 - User can't hotplug memory to less memory numa node on rhel8 +Patch50: kvm-spapr-Add-ibm-max-associativity-domains-property.patch +# For bz#1599593 - User can't hotplug memory to less memory numa node on rhel8 +Patch51: kvm-Revert-spapr-Don-t-allow-memory-hotplug-to-memory-le.patch +# For bz#1594969 - simpletrace.py fails when running with Python 3 +Patch52: kvm-simpletrace-Convert-name-from-mapping-record-to-str.patch +# For bz#1602403 - test-crypto-tlssession unit test fails with assertions +Patch53: kvm-tests-fix-TLS-handshake-failure-with-TLS-1.3.patch +# For bz#1601671 - After rebooting guest,all the hot plug memory will be assigned to the 1st numa node. +Patch54: kvm-spapr-Correct-inverted-test-in-spapr_pc_dimm_node.patch +# For bz#1601317 - RHEL8.0 - qemu patch to align memory to allow 2MB THP +Patch55: kvm-osdep-powerpc64-align-memory-to-allow-2MB-radix-THP-.patch +# For bz#1595501 - Create pseries-rhel7.6.0-sxxm machine type +Patch56: kvm-RHEL-8.0-Add-pseries-rhel7.6.0-sxxm-machine-type.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch57: kvm-i386-Helpers-to-encode-cache-information-consistentl.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch58: kvm-i386-Add-cache-information-in-X86CPUDefinition.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch59: kvm-i386-Initialize-cache-information-for-EPYC-family-pr.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch60: kvm-i386-Add-new-property-to-control-cache-info.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch61: kvm-i386-Clean-up-cache-CPUID-code.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch62: kvm-i386-Populate-AMD-Processor-Cache-Information-for-cp.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch63: kvm-i386-Add-support-for-CPUID_8000_001E-for-AMD.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch64: kvm-i386-Fix-up-the-Node-id-for-CPUID_8000_001E.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch65: kvm-i386-Enable-TOPOEXT-feature-on-AMD-EPYC-CPU.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch66: kvm-i386-Remove-generic-SMT-thread-check.patch +# For bz#1597739 - AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm) +Patch67: kvm-i386-Allow-TOPOEXT-to-be-enabled-on-older-kernels.patch +# For bz#1610906 - [IBM 8.0 FEAT] KVM: Huge Pages - libhugetlbfs Enablement - qemu-kvm part +Patch68: kvm-linux-headers-Update-to-include-KVM_CAP_S390_HPAGE_1.patch +# For bz#1610906 - [IBM 8.0 FEAT] KVM: Huge Pages - libhugetlbfs Enablement - qemu-kvm part +Patch69: kvm-s390x-Enable-KVM-huge-page-backing-support.patch +# For bz#1558723 - Create RHEL-7.6 QEMU machine type for AArch64 +Patch70: kvm-AArch64-Add-virt-rhel7.6-machine-type.patch +# For bz#1566153 - IOERROR pause code lost after resuming a VM while I/O error is still present +Patch71: kvm-cpus-Fix-event-order-on-resume-of-stopped-guest.patch +# For bz#1523065 - "qemu-img resize" should fail to decrease the size of logical partition/lvm/iSCSI image with raw format +Patch72: kvm-qemu-img-Check-post-truncation-size.patch +# For bz#1575541 - qemu core dump while installing win10 guest +Patch73: kvm-vga-catch-depth-0.patch +# For bz#1583959 - Incorrect vcpu count limit for 7.4 machine types for windows guests +Patch74: kvm-Fix-x-hv-max-vps-compat-value-for-7.4-machine-type.patch +# For bz#1584984 - Vm starts failed with 'passthrough' smartcard +Patch75: kvm-ccid-card-passthru-fix-regression-in-realize.patch +# For bz#1542080 - Qemu core dump at cirrus_invalidate_region +Patch76: kvm-Use-4-MB-vram-for-cirrus.patch +# For bz#1505664 - "qemu-kvm: System page size 0x1000000 is not enabled in page_size_mask (0x11000). Performance may be slow" show up while using hugepage as guest's memory +Patch77: kvm-spapr_pci-Remove-unhelpful-pagesize-warning.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch79: kvm-qobject-Use-qobject_to-instead-of-type-cast.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch80: kvm-qobject-Ensure-base-is-at-offset-0.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch81: kvm-qobject-use-a-QObjectBase_-struct.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch82: kvm-qobject-Replace-qobject_incref-QINCREF-qobject_decre.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch83: kvm-qobject-Modify-qobject_ref-to-return-obj.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch84: kvm-rbd-Drop-deprecated-drive-parameter-filename.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch85: kvm-iscsi-Drop-deprecated-drive-parameter-filename.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch86: kvm-block-Add-block-specific-QDict-header.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch87: kvm-qobject-Move-block-specific-qdict-code-to-block-qdic.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch88: kvm-block-Fix-blockdev-for-certain-non-string-scalars.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch89: kvm-block-Fix-drive-for-certain-non-string-scalars.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch90: kvm-block-Clean-up-a-misuse-of-qobject_to-in-.bdrv_co_cr.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch91: kvm-block-Factor-out-qobject_input_visitor_new_flat_conf.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch92: kvm-block-Make-remaining-uses-of-qobject-input-visitor-m.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch93: kvm-block-qdict-Simplify-qdict_flatten_qdict.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch94: kvm-block-qdict-Tweak-qdict_flatten_qdict-qdict_flatten_.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch95: kvm-block-qdict-Clean-up-qdict_crumple-a-bit.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch96: kvm-block-qdict-Simplify-qdict_is_list-some.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch97: kvm-check-block-qdict-Rename-qdict_flatten-s-variables-f.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch98: kvm-check-block-qdict-Cover-flattening-of-empty-lists-an.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch99: kvm-block-Fix-blockdev-blockdev-add-for-empty-objects-an.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch100: kvm-rbd-New-parameter-auth-client-required.patch +# For bz#1557995 - QAPI schema for RBD storage misses the 'password-secret' option +Patch101: kvm-rbd-New-parameter-key-secret.patch +# For bz#1572856 - 'block-job-cancel' can not cancel a "drive-mirror" job +Patch102: kvm-block-mirror-honor-ratelimit-again.patch +# For bz#1572856 - 'block-job-cancel' can not cancel a "drive-mirror" job +Patch103: kvm-block-mirror-Make-cancel-always-cancel-pre-READY.patch +# For bz#1572856 - 'block-job-cancel' can not cancel a "drive-mirror" job +Patch104: kvm-iotests-Add-test-for-cancelling-a-mirror-job.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch105: kvm-iotests-Split-214-off-of-122.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch106: kvm-block-Add-COR-filter-driver.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch107: kvm-block-BLK_PERM_WRITE-includes-._UNCHANGED.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch108: kvm-block-Add-BDRV_REQ_WRITE_UNCHANGED-flag.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch109: kvm-block-Set-BDRV_REQ_WRITE_UNCHANGED-for-COR-writes.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch110: kvm-block-quorum-Support-BDRV_REQ_WRITE_UNCHANGED.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch111: kvm-block-Support-BDRV_REQ_WRITE_UNCHANGED-in-filters.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch112: kvm-iotests-Clean-up-wrap-image-in-197.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch113: kvm-iotests-Copy-197-for-COR-filter-driver.patch +# For bz#1518738 - Add 'copy-on-read' filter driver for use with blockdev-add +Patch114: kvm-iotests-Add-test-for-COR-across-nodes.patch +# For bz#1576598 - Segfault in qemu-io and qemu-img with -U --image-opts force-share=off +Patch115: kvm-qemu-io-Use-purely-string-blockdev-options.patch +# For bz#1576598 - Segfault in qemu-io and qemu-img with -U --image-opts force-share=off +Patch116: kvm-qemu-img-Use-only-string-options-in-img_open_opts.patch +# For bz#1576598 - Segfault in qemu-io and qemu-img with -U --image-opts force-share=off +Patch117: kvm-iotests-Add-test-for-U-force-share-conflicts.patch +# For bz#1519617 - The exit code should be non-zero when qemu-io reports an error +Patch118: kvm-qemu-io-Drop-command-functions-return-values.patch +# For bz#1519617 - The exit code should be non-zero when qemu-io reports an error +Patch119: kvm-qemu-io-Let-command-functions-return-error-code.patch +# For bz#1519617 - The exit code should be non-zero when qemu-io reports an error +Patch120: kvm-qemu-io-Exit-with-error-when-a-command-failed.patch +# For bz#1519617 - The exit code should be non-zero when qemu-io reports an error +Patch121: kvm-iotests.py-Add-qemu_io_silent.patch +# For bz#1519617 - The exit code should be non-zero when qemu-io reports an error +Patch122: kvm-iotests-Let-216-make-use-of-qemu-io-s-exit-code.patch +# For bz#1527085 - The copied flag should be updated during '-r leaks' +Patch123: kvm-qcow2-Repair-OFLAG_COPIED-when-fixing-leaks.patch +# For bz#1527085 - The copied flag should be updated during '-r leaks' +Patch124: kvm-iotests-Repairing-error-during-snapshot-deletion.patch +# For bz#1588039 - Possible assertion failure in qemu when a corrupted image is used during an incoming migration +Patch125: kvm-block-Make-bdrv_is_writable-public.patch +# For bz#1588039 - Possible assertion failure in qemu when a corrupted image is used during an incoming migration +Patch126: kvm-qcow2-Do-not-mark-inactive-images-corrupt.patch +# For bz#1588039 - Possible assertion failure in qemu when a corrupted image is used during an incoming migration +Patch127: kvm-iotests-Add-case-for-a-corrupted-inactive-image.patch +# For bz#1168213 - main-loop: WARNING: I/O thread spun for 1000 iterations while doing stream block device. +Patch128: kvm-main-loop-drop-spin_counter.patch +# For bz#1560847 - [Power8][FW b0320a_1812.861][rhel7.5rc2 3.10.0-861.el7.ppc64le][qemu-kvm-{ma,rhev}-2.10.0-21.el7_5.1.ppc64le] KVM guest does not default to ori type flush even with pseries-rhel7.5.0-sxxm +Patch129: kvm-target-ppc-Factor-out-the-parsing-in-kvmppc_get_cpu_.patch +# For bz#1560847 - [Power8][FW b0320a_1812.861][rhel7.5rc2 3.10.0-861.el7.ppc64le][qemu-kvm-{ma,rhev}-2.10.0-21.el7_5.1.ppc64le] KVM guest does not default to ori type flush even with pseries-rhel7.5.0-sxxm +Patch130: kvm-target-ppc-Don-t-require-private-l1d-cache-on-POWER8.patch +# For bz#1560847 - [Power8][FW b0320a_1812.861][rhel7.5rc2 3.10.0-861.el7.ppc64le][qemu-kvm-{ma,rhev}-2.10.0-21.el7_5.1.ppc64le] KVM guest does not default to ori type flush even with pseries-rhel7.5.0-sxxm +Patch131: kvm-ppc-spapr_caps-Don-t-disable-cap_cfpc-on-POWER8-by-d.patch +# For bz#1567733 - qemu abort when migrate during guest reboot +Patch132: kvm-qxl-fix-local-renderer-crash.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch133: kvm-qemu-img-Amendment-support-implies-create_opts.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch134: kvm-block-Add-Error-parameter-to-bdrv_amend_options.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch135: kvm-qemu-option-Pull-out-Supported-options-print.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch136: kvm-qemu-img-Add-print_amend_option_help.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch137: kvm-qemu-img-Recognize-no-creation-support-in-o-help.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch138: kvm-iotests-Test-help-option-for-unsupporting-formats.patch +# For bz#1537956 - RFE: qemu-img amend should list the true supported options +Patch139: kvm-iotests-Rework-113.patch +# For bz#1569835 - qemu-img get wrong backing file path after rebasing image with relative path +Patch140: kvm-qemu-img-Resolve-relative-backing-paths-in-rebase.patch +# For bz#1569835 - qemu-img get wrong backing file path after rebasing image with relative path +Patch141: kvm-iotests-Add-test-for-rebasing-with-relative-paths.patch +# For bz#1527898 - [RFE] qemu-img should leave cluster unallocated if it's read as zero throughout the backing chain +Patch142: kvm-qemu-img-Special-post-backing-convert-handling.patch +# For bz#1527898 - [RFE] qemu-img should leave cluster unallocated if it's read as zero throughout the backing chain +Patch143: kvm-iotests-Test-post-backing-convert-target-behavior.patch +# For bz#1564576 - Pegas 1.1 - Require to backport qemu-kvm patch that fixes expected_downtime calculation during migration +Patch144: kvm-migration-calculate-expected_downtime-with-ram_bytes.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch145: kvm-sheepdog-Fix-sd_co_create_opts-memory-leaks.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch146: kvm-qemu-iotests-reduce-chance-of-races-in-185.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch147: kvm-blockjob-do-not-cancel-timer-in-resume.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch148: kvm-nfs-Fix-error-path-in-nfs_options_qdict_to_qapi.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch149: kvm-nfs-Remove-processed-options-from-QDict.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch150: kvm-blockjob-drop-block_job_pause-resume_all.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch151: kvm-blockjob-expose-error-string-via-query.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch152: kvm-blockjob-Fix-assertion-in-block_job_finalize.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch153: kvm-blockjob-Wrappers-for-progress-counter-access.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch154: kvm-blockjob-Move-RateLimit-to-BlockJob.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch155: kvm-blockjob-Implement-block_job_set_speed-centrally.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch156: kvm-blockjob-Introduce-block_job_ratelimit_get_delay.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch157: kvm-blockjob-Add-block_job_driver.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch158: kvm-blockjob-Update-block-job-pause-resume-documentation.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch159: kvm-blockjob-Improve-BlockJobInfo.offset-len-documentati.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch160: kvm-job-Create-Job-JobDriver-and-job_create.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch161: kvm-job-Rename-BlockJobType-into-JobType.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch162: kvm-job-Add-JobDriver.job_type.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch163: kvm-job-Add-job_delete.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch164: kvm-job-Maintain-a-list-of-all-jobs.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch165: kvm-job-Move-state-transitions-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch166: kvm-job-Add-reference-counting.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch167: kvm-job-Move-cancelled-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch168: kvm-job-Add-Job.aio_context.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch169: kvm-job-Move-defer_to_main_loop-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch170: kvm-job-Move-coroutine-and-related-code-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch171: kvm-job-Add-job_sleep_ns.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch172: kvm-job-Move-pause-resume-functions-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch173: kvm-job-Replace-BlockJob.completed-with-job_is_completed.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch174: kvm-job-Move-BlockJobCreateFlags-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch175: kvm-blockjob-Split-block_job_event_pending.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch176: kvm-job-Add-job_event_.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch177: kvm-job-Move-single-job-finalisation-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch178: kvm-job-Convert-block_job_cancel_async-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch179: kvm-job-Add-job_drain.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch180: kvm-job-Move-.complete-callback-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch181: kvm-job-Move-job_finish_sync-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch182: kvm-job-Switch-transactions-to-JobTxn.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch183: kvm-job-Move-transactions-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch184: kvm-job-Move-completion-and-cancellation-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch185: kvm-block-Cancel-job-in-bdrv_close_all-callers.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch186: kvm-job-Add-job_yield.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch187: kvm-job-Add-job_dismiss.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch188: kvm-job-Add-job_is_ready.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch189: kvm-job-Add-job_transition_to_ready.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch190: kvm-job-Move-progress-fields-to-Job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch191: kvm-job-Introduce-qapi-job.json.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch192: kvm-job-Add-JOB_STATUS_CHANGE-QMP-event.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch193: kvm-job-Add-lifecycle-QMP-commands.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch194: kvm-job-Add-query-jobs-QMP-command.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch195: kvm-blockjob-Remove-BlockJob.driver.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch196: kvm-iotests-Move-qmp_to_opts-to-VM.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch197: kvm-qemu-iotests-Test-job-with-block-jobs.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch198: kvm-vdi-Fix-vdi_co_do_create-return-value.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch199: kvm-vhdx-Fix-vhdx_co_create-return-value.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch200: kvm-job-Add-error-message-for-failing-jobs.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch201: kvm-block-create-Make-x-blockdev-create-a-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch202: kvm-qemu-iotests-Add-VM.get_qmp_events_filtered.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch203: kvm-qemu-iotests-Add-VM.qmp_log.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch204: kvm-qemu-iotests-Add-iotests.img_info_log.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch205: kvm-qemu-iotests-Add-VM.run_job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch206: kvm-qemu-iotests-iotests.py-helper-for-non-file-protocol.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch207: kvm-qemu-iotests-Rewrite-206-for-blockdev-create-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch208: kvm-qemu-iotests-Rewrite-207-for-blockdev-create-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch209: kvm-qemu-iotests-Rewrite-210-for-blockdev-create-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch210: kvm-qemu-iotests-Rewrite-211-for-blockdev-create-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch211: kvm-qemu-iotests-Rewrite-212-for-blockdev-create-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch212: kvm-qemu-iotests-Rewrite-213-for-blockdev-create-job.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch213: kvm-block-create-Mark-blockdev-create-stable.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch214: kvm-jobs-fix-stale-wording.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch215: kvm-jobs-fix-verb-references-in-docs.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch216: kvm-iotests-Fix-219-s-timing.patch +# For bz#1513543 - [RFE] Add block job to create format on a storage device +Patch217: kvm-iotests-improve-pause_job.patch +# For bz#1572851 - Core dumped after migration when with usb-host +Patch220: kvm-usb-host-skip-open-on-pending-postload-bh.patch +# For bz#1574216 - CVE-2018-3639 qemu-kvm-rhev: hw: cpu: speculative store bypass [rhel-7.6] +Patch221: kvm-i386-Define-the-Virt-SSBD-MSR-and-handling-of-it-CVE.patch +# For bz#1574216 - CVE-2018-3639 qemu-kvm-rhev: hw: cpu: speculative store bypass [rhel-7.6] +Patch222: kvm-i386-define-the-AMD-virt-ssbd-CPUID-feature-bit-CVE-.patch +# For bz#1519144 - qemu-img: image locking doesn't cover image creation +Patch223: kvm-block-file-posix-Pass-FD-to-locking-helpers.patch +# For bz#1519144 - qemu-img: image locking doesn't cover image creation +Patch224: kvm-block-file-posix-File-locking-during-creation.patch +# For bz#1519144 - qemu-img: image locking doesn't cover image creation +Patch225: kvm-iotests-Add-creation-test-to-153.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch226: kvm-vhost-user-add-Net-prefix-to-internal-state-structur.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch227: kvm-virtio-support-setting-memory-region-based-host-noti.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch228: kvm-vhost-user-support-receiving-file-descriptors-in-sla.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch229: kvm-osdep-add-wait.h-compat-macros.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch230: kvm-vhost-user-bridge-support-host-notifier.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch231: kvm-vhost-allow-backends-to-filter-memory-sections.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch232: kvm-vhost-user-allow-slave-to-send-fds-via-slave-channel.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch233: kvm-vhost-user-introduce-shared-vhost-user-state.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch234: kvm-vhost-user-support-registering-external-host-notifie.patch +# For bz#1526645 - [Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev +Patch235: kvm-libvhost-user-support-host-notifier.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch236: kvm-block-Introduce-API-for-copy-offloading.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch237: kvm-raw-Check-byte-range-uniformly.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch238: kvm-raw-Implement-copy-offloading.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch239: kvm-qcow2-Implement-copy-offloading.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch240: kvm-file-posix-Implement-bdrv_co_copy_range.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch241: kvm-iscsi-Query-and-save-device-designator-when-opening.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch242: kvm-iscsi-Create-and-use-iscsi_co_wait_for_task.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch243: kvm-iscsi-Implement-copy-offloading.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch244: kvm-block-backend-Add-blk_co_copy_range.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch245: kvm-qemu-img-Convert-with-copy-offloading.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch246: kvm-qcow2-Fix-src_offset-in-copy-offloading.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch247: kvm-iscsi-Don-t-blindly-use-designator-length-in-respons.patch +# For bz#1482537 - [RFE] qemu-img copy-offloading (convert command) +Patch248: kvm-file-posix-Fix-EINTR-handling.patch +# For bz#1595180 - Can't set rerror/werror with usb-storage +Patch249: kvm-usb-storage-Add-rerror-werror-properties.patch +# For bz#1578381 - Error message need update when specify numa distance with node index >=128 +Patch250: kvm-numa-clarify-error-message-when-node-index-is-out-of.patch +# For bz#1528541 - qemu-img check reports tons of leaked clusters after re-start nfs service to resume writing data in guest +Patch251: kvm-qemu-iotests-Update-026.out.nocache-reference-output.patch +# For bz#1528541 - qemu-img check reports tons of leaked clusters after re-start nfs service to resume writing data in guest +Patch252: kvm-qcow2-Free-allocated-clusters-on-write-error.patch +# For bz#1528541 - qemu-img check reports tons of leaked clusters after re-start nfs service to resume writing data in guest +Patch253: kvm-qemu-iotests-Test-qcow2-not-leaking-clusters-on-writ.patch +# For bz#1586313 - -smp option is not easily found in the output of qemu help +Patch254: kvm-qemu-options-Add-missing-newline-to-accel-help-text.patch +# For bz#1594135 - system_reset many times linux guests cause qemu process Aborted +Patch255: kvm-xhci-fix-guest-triggerable-assert.patch +# For bz#1589634 - Migration failed when rebooting guest with multiple virtio videos +Patch256: kvm-virtio-gpu-tweak-scanout-disable.patch +# For bz#1589634 - Migration failed when rebooting guest with multiple virtio videos +Patch257: kvm-virtio-gpu-update-old-resource-too.patch +# For bz#1589634 - Migration failed when rebooting guest with multiple virtio videos +Patch258: kvm-virtio-gpu-disable-scanout-when-backing-resource-is-.patch +# For bz#1549654 - Reject node-names which would be truncated by the block layer commands +Patch259: kvm-block-Don-t-silently-truncate-node-names.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch260: kvm-pr-helper-fix-socket-path-default-in-help.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch261: kvm-pr-helper-fix-assertion-failure-on-failed-multipath-.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch262: kvm-pr-manager-helper-avoid-SIGSEGV-when-writing-to-the-.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch263: kvm-pr-manager-put-stubs-in-.c-file.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch264: kvm-pr-manager-add-query-pr-managers-QMP-command.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch265: kvm-pr-manager-helper-report-event-on-connection-disconn.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch266: kvm-pr-helper-avoid-error-on-PR-IN-command-with-zero-req.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch267: kvm-pr-helper-Rework-socket-path-handling.patch +# For bz#1533158 - QEMU support for libvirtd restarting qemu-pr-helper +Patch268: kvm-pr-manager-helper-fix-memory-leak-on-event.patch +# For bz#1556678 - Hot plug usb-ccid for the 2nd time with the same ID as the 1st time failed +Patch269: kvm-object-fix-OBJ_PROP_LINK_UNREF_ON_RELEASE-ambivalenc.patch +# For bz#1556678 - Hot plug usb-ccid for the 2nd time with the same ID as the 1st time failed +Patch270: kvm-usb-hcd-xhci-test-add-a-test-for-ccid-hotplug.patch +# For bz#1556678 - Hot plug usb-ccid for the 2nd time with the same ID as the 1st time failed +Patch271: kvm-Revert-usb-release-the-created-buses.patch +# For bz#1599335 - Image creation locking is too tight and is not properly released +Patch272: kvm-file-posix-Fix-creation-locking.patch +# For bz#1599335 - Image creation locking is too tight and is not properly released +Patch273: kvm-file-posix-Unlock-FD-after-creation.patch +# For bz#1584914 - SATA emulator lags and hangs +Patch274: kvm-ahci-trim-signatures-on-raise-lower.patch +# For bz#1584914 - SATA emulator lags and hangs +Patch275: kvm-ahci-fix-PxCI-register-race.patch +# For bz#1584914 - SATA emulator lags and hangs +Patch276: kvm-ahci-don-t-schedule-unnecessary-BH.patch +# For bz#1595173 - blockdev-create is blocking +Patch277: kvm-qcow2-Fix-qcow2_truncate-error-return-value.patch +# For bz#1595173 - blockdev-create is blocking +Patch278: kvm-block-Convert-.bdrv_truncate-callback-to-coroutine_f.patch +# For bz#1595173 - blockdev-create is blocking +Patch279: kvm-qcow2-Remove-coroutine-trampoline-for-preallocate_co.patch +# For bz#1595173 - blockdev-create is blocking +Patch280: kvm-block-Move-bdrv_truncate-implementation-to-io.c.patch +# For bz#1595173 - blockdev-create is blocking +Patch281: kvm-block-Use-tracked-request-for-truncate.patch +# For bz#1595173 - blockdev-create is blocking +Patch282: kvm-file-posix-Make-.bdrv_co_truncate-asynchronous.patch +# For bz#1590640 - qemu-kvm: block/io.c:1098: bdrv_co_do_copy_on_readv: Assertion `skip_bytes < pnum' failed. +Patch283: kvm-block-Fix-copy-on-read-crash-with-partial-final-clus.patch +# For bz#1599515 - qemu core-dump with aio_read via hmp (util/qemu-thread-posix.c:64: qemu_mutex_lock_impl: Assertion `mutex->initialized' failed) +Patch284: kvm-block-fix-QEMU-crash-with-scsi-hd-and-drive_del.patch +# For bz#1576743 - virtio-rng hangs when running on recent (2.x) QEMU versions +Patch285: kvm-virtio-rng-process-pending-requests-on-DRIVER_OK.patch +# For bz#1525829 - can not boot up a scsi-block passthrough disk via -blockdev with error "cannot get SG_IO version number: Operation not supported. Is this a SCSI device?" +Patch286: kvm-file-posix-specify-expected-filetypes.patch +# For bz#1525829 - can not boot up a scsi-block passthrough disk via -blockdev with error "cannot get SG_IO version number: Operation not supported. Is this a SCSI device?" +Patch287: kvm-iotests-add-test-226-for-file-driver-types.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch288: kvm-block-dirty-bitmap-add-lock-to-bdrv_enable-disable_d.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch289: kvm-qapi-add-x-block-dirty-bitmap-enable-disable.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch290: kvm-qmp-transaction-support-for-x-block-dirty-bitmap-ena.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch291: kvm-qapi-add-x-block-dirty-bitmap-merge.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch292: kvm-qapi-add-disabled-parameter-to-block-dirty-bitmap-ad.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch293: kvm-block-dirty-bitmap-add-bdrv_enable_dirty_bitmap_lock.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch294: kvm-dirty-bitmap-fix-double-lock-on-bitmap-enabling.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch295: kvm-block-qcow2-bitmap-fix-free_bitmap_clusters.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch296: kvm-qcow2-add-overlap-check-for-bitmap-directory.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch297: kvm-blockdev-enable-non-root-nodes-for-backup-source.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch298: kvm-iotests-add-222-to-test-basic-fleecing.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch299: kvm-qcow2-Remove-dead-check-on-ret.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch300: kvm-block-Move-request-tracking-to-children-in-copy-offl.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch301: kvm-block-Fix-parameter-checking-in-bdrv_co_copy_range_i.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch302: kvm-block-Honour-BDRV_REQ_NO_SERIALISING-in-copy-range.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch303: kvm-backup-Use-copy-offloading.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch304: kvm-block-backup-disable-copy-offloading-for-backup.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch305: kvm-iotests-222-Don-t-run-with-luks.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch306: kvm-block-io-fix-copy_range.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch307: kvm-block-split-flags-in-copy_range.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch308: kvm-block-add-BDRV_REQ_SERIALISING-flag.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch309: kvm-block-backup-fix-fleecing-scheme-use-serialized-writ.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch310: kvm-nbd-server-Reject-0-length-block-status-request.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch311: kvm-nbd-server-fix-trace.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch312: kvm-nbd-server-refactor-NBDExportMetaContexts.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch313: kvm-nbd-server-add-nbd_meta_empty_or_pattern-helper.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch314: kvm-nbd-server-implement-dirty-bitmap-export.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch315: kvm-qapi-new-qmp-command-nbd-server-add-bitmap.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch316: kvm-docs-interop-add-nbd.txt.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch317: kvm-nbd-server-introduce-NBD_CMD_CACHE.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch318: kvm-nbd-server-Silence-gcc-false-positive.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch319: kvm-nbd-server-Fix-dirty-bitmap-logic-regression.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch320: kvm-nbd-server-fix-nbd_co_send_block_status.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch321: kvm-nbd-client-Add-x-dirty-bitmap-to-query-bitmap-from-s.patch +# For bz#1207657 - RFE: QEMU Incremental live backup - push and pull modes +Patch322: kvm-iotests-New-test-223-for-exporting-dirty-bitmap-over.patch +# For bz#1592817 - Retrying on serial_xmit if the pipe is broken may compromise the Guest +Patch323: kvm-hw-char-serial-Only-retry-if-qemu_chr_fe_write-retur.patch +# For bz#1592817 - Retrying on serial_xmit if the pipe is broken may compromise the Guest +Patch324: kvm-hw-char-serial-retry-write-if-EAGAIN.patch +# For bz#1535914 - Disable io throttling for one member disk of a group during io will induce the other one hang with io +Patch325: kvm-throttle-groups-fix-hang-when-group-member-leaves.patch +# For bz#1586357 - Disable new devices in 2.12 +Patch326: kvm-Disable-aarch64-devices-reappeared-after-2.12-rebase.patch +# For bz#1586357 - Disable new devices in 2.12 +Patch327: kvm-Disable-split-irq-device.patch +# For bz#1586357 - Disable new devices in 2.12 +Patch328: kvm-Disable-AT24Cx-i2c-eeprom.patch +# For bz#1586357 - Disable new devices in 2.12 +Patch329: kvm-Disable-CAN-bus-devices.patch +# For bz#1586357 - Disable new devices in 2.12 +Patch330: kvm-Disable-new-superio-devices.patch +# For bz#1586357 - Disable new devices in 2.12 +Patch331: kvm-Disable-new-pvrdma-device.patch +# For bz#1607891 - Hotplug events are sometimes lost with virtio-scsi + iothread +Patch332: kvm-qdev-add-HotplugHandler-post_plug-callback.patch +# For bz#1607891 - Hotplug events are sometimes lost with virtio-scsi + iothread +Patch333: kvm-virtio-scsi-fix-hotplug-reset-vs-event-race.patch +# For bz#1608778 - qemu/migration: migrate failed from RHEL.7.6 to RHEL.7.5 with e1000-82540em +Patch334: kvm-e1000-Fix-tso_props-compat-for-82540em.patch +# For bz#1586255 - CVE-2018-11806 qemu-kvm-rhev: QEMU: slirp: heap buffer overflow while reassembling fragmented datagrams [rhel-7.6] +Patch335: kvm-slirp-correct-size-computation-while-concatenating-m.patch +# For bz#1595740 - RHEL-Alt-7.6 - qemu has error during migration of larger guests +Patch336: kvm-s390x-sclp-fix-maxram-calculation.patch +# For bz#1594384 - 2.12 migration fixes +Patch337: kvm-migration-stop-compressing-page-in-migration-thread.patch +# For bz#1594384 - 2.12 migration fixes +Patch338: kvm-migration-stop-compression-to-allocate-and-free-memo.patch +# For bz#1594384 - 2.12 migration fixes +Patch339: kvm-migration-stop-decompression-to-allocate-and-free-me.patch +# For bz#1594384 - 2.12 migration fixes +Patch340: kvm-migration-detect-compression-and-decompression-error.patch +# For bz#1594384 - 2.12 migration fixes +Patch341: kvm-migration-introduce-control_save_page.patch +# For bz#1594384 - 2.12 migration fixes +Patch342: kvm-migration-move-some-code-to-ram_save_host_page.patch +# For bz#1594384 - 2.12 migration fixes +Patch343: kvm-migration-move-calling-control_save_page-to-the-comm.patch +# For bz#1594384 - 2.12 migration fixes +Patch344: kvm-migration-move-calling-save_zero_page-to-the-common-.patch +# For bz#1594384 - 2.12 migration fixes +Patch345: kvm-migration-introduce-save_normal_page.patch +# For bz#1594384 - 2.12 migration fixes +Patch346: kvm-migration-remove-ram_save_compressed_page.patch +# For bz#1594384 - 2.12 migration fixes +Patch347: kvm-migration-block-dirty-bitmap-fix-memory-leak-in-dirt.patch +# For bz#1594384 - 2.12 migration fixes +Patch348: kvm-migration-fix-saving-normal-page-even-if-it-s-been-c.patch +# For bz#1594384 - 2.12 migration fixes +Patch349: kvm-migration-update-index-field-when-delete-or-qsort-RD.patch +# For bz#1594384 - 2.12 migration fixes +Patch350: kvm-migration-introduce-decompress-error-check.patch +# For bz#1594384 - 2.12 migration fixes +Patch351: kvm-migration-Don-t-activate-block-devices-if-using-S.patch +# For bz#1594384 - 2.12 migration fixes +Patch352: kvm-migration-not-wait-RDMA_CM_EVENT_DISCONNECTED-event-.patch +# For bz#1594384 - 2.12 migration fixes +Patch353: kvm-migration-block-dirty-bitmap-fix-dirty_bitmap_load.patch +# For bz#1595718 - Add ppa15/bpb to the default cpu model for z196 and higher in the 7.6 s390-ccw-virtio machine +Patch354: kvm-s390x-add-RHEL-7.6-machine-type-for-ccw.patch +# For bz#1595718 - Add ppa15/bpb to the default cpu model for z196 and higher in the 7.6 s390-ccw-virtio machine +Patch355: kvm-s390x-cpumodel-default-enable-bpb-and-ppa15-for-z196.patch +# For bz#1612938 - Add etoken support to qemu-kvm for s390x KVM guests +Patch356: kvm-linux-headers-asm-s390-kvm.h-header-sync.patch +# For bz#1612938 - Add etoken support to qemu-kvm for s390x KVM guests +Patch357: kvm-s390x-kvm-add-etoken-facility.patch +# For bz#1594384 - 2.12 migration fixes +Patch358: kvm-Migration-TLS-Fix-crash-due-to-double-cleanup.patch +# For bz#1622656 - qemu-kvm fails to build due to libusb_set_debug being deprecated +Patch359: kvm-Fix-libusb-1.0.22-deprecated-libusb_set_debug-with-l.patch +# For bz#1596024 - The network link can't be detected on guest when the guest uses e1000e model type +Patch360: kvm-e1000e-Do-not-auto-clear-ICR-bits-which-aren-t-set-i.patch +# For bz#1596024 - The network link can't be detected on guest when the guest uses e1000e model type +Patch361: kvm-e1000e-Prevent-MSI-MSI-X-storms.patch +# For bz#1615717 - Memory leaks +Patch362: kvm-target-i386-sev-fix-memory-leaks.patch +# For bz#1615717 - Memory leaks +Patch363: kvm-i386-Fix-arch_query_cpu_model_expansion-leak.patch +# For bz#1619804 - kernel panic in init_amd_cacheinfo +Patch364: kvm-i386-Disable-TOPOEXT-by-default-on-cpu-host.patch +# For bz#1625185 - Re-enable disabled Hyper-V enlightenments +Patch365: kvm-Re-enable-disabled-Hyper-V-enlightenments.patch +# For bz#1624164 - Review annocheck distro flag failures in qemu-kvm +Patch366: kvm-Fix-annocheck-issues.patch +# For bz#1630746 - qemu_ram_mmap: Assertion `is_power_of_2(align)' failed +Patch367: kvm-exec-check-that-alignment-is-a-power-of-two.patch +# For bz#1575925 - "SSL: no alternative certificate subject name matches target host name" error even though sslverify = off +Patch368: kvm-curl-Make-sslverify-off-disable-host-as-well-as-peer.patch +# For bz#1608765 - After postcopy migration, do savevm and loadvm, guest hang and call trace +Patch370: kvm-migration-postcopy-Clear-have_listen_thread.patch +# For bz#1608765 - After postcopy migration, do savevm and loadvm, guest hang and call trace +Patch371: kvm-migration-cleanup-in-error-paths-in-loadvm.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch372: kvm-jobs-change-start-callback-to-run-callback.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch373: kvm-jobs-canonize-Error-object.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch374: kvm-jobs-add-exit-shim.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch375: kvm-block-commit-utilize-job_exit-shim.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch376: kvm-block-mirror-utilize-job_exit-shim.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch377: kvm-jobs-utilize-job_exit-shim.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch378: kvm-block-backup-make-function-variables-consistently-na.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch379: kvm-jobs-remove-ret-argument-to-job_completed-privatize-.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch380: kvm-jobs-remove-job_defer_to_main_loop.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch381: kvm-block-commit-add-block-job-creation-flags.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch382: kvm-block-mirror-add-block-job-creation-flags.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch383: kvm-block-stream-add-block-job-creation-flags.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch384: kvm-block-commit-refactor-commit-to-use-job-callbacks.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch385: kvm-block-mirror-don-t-install-backing-chain-on-abort.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch386: kvm-block-mirror-conservative-mirror_exit-refactor.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch387: kvm-block-stream-refactor-stream-to-use-job-callbacks.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch388: kvm-tests-blockjob-replace-Blockjob-with-Job.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch389: kvm-tests-test-blockjob-remove-exit-callback.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch390: kvm-tests-test-blockjob-txn-move-.exit-to-.clean.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch391: kvm-jobs-remove-.exit-callback.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch392: kvm-qapi-block-commit-expose-new-job-properties.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch393: kvm-qapi-block-mirror-expose-new-job-properties.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch394: kvm-qapi-block-stream-expose-new-job-properties.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch395: kvm-block-backup-qapi-documentation-fixup.patch +# For bz#1632939 - qemu blockjobs other than backup do not support job-finalize or job-dismiss +Patch396: kvm-blockdev-document-transactional-shortcomings.patch +# For bz#1618356 - qemu-kvm: Qemu: seccomp: blacklist is not applied to all threads [rhel-8] +Patch397: kvm-seccomp-allow-sched_setscheduler-with-SCHED_IDLE-pol.patch +# For bz#1618356 - qemu-kvm: Qemu: seccomp: blacklist is not applied to all threads [rhel-8] +Patch398: kvm-seccomp-use-SIGSYS-signal-instead-of-killing-the-thr.patch +# For bz#1618356 - qemu-kvm: Qemu: seccomp: blacklist is not applied to all threads [rhel-8] +Patch399: kvm-seccomp-prefer-SCMP_ACT_KILL_PROCESS-if-available.patch +# For bz#1618356 - qemu-kvm: Qemu: seccomp: blacklist is not applied to all threads [rhel-8] +Patch400: kvm-configure-require-libseccomp-2.2.0.patch +# For bz#1618356 - qemu-kvm: Qemu: seccomp: blacklist is not applied to all threads [rhel-8] +Patch401: kvm-seccomp-set-the-seccomp-filter-to-all-threads.patch +# For bz#1600365 - QEMU core dumped when hotplug memory exceeding host hugepages and with discard-data=yes +Patch402: kvm-memory-cleanup-side-effects-of-memory_region_init_fo.patch +# For bz#1621817 - Disable IVSHMEM in RHEL 8 +Patch403: kvm-check-Only-test-ivshm-when-it-is-compiled-in.patch +# For bz#1621817 - Disable IVSHMEM in RHEL 8 +Patch404: kvm-Disable-ivshmem.patch +# For bz#1637963 - Segfault on 'blockdev-mirror' with same node as source and target +Patch405: kvm-mirror-Fail-gracefully-for-source-target.patch +# For bz#1637970 - allow using node-names with block-commit +Patch406: kvm-commit-Add-top-node-base-node-options.patch +# For bz#1637970 - allow using node-names with block-commit +Patch407: kvm-qemu-iotests-Test-commit-with-top-node-base-node.patch +# For bz#1635583 - Quitting VM causes qemu core dump once the block mirror job paused for no enough target space +Patch408: kvm-block-for-jobs-do-not-clear-user_paused-until-after-.patch +# For bz#1635583 - Quitting VM causes qemu core dump once the block mirror job paused for no enough target space +Patch409: kvm-iotests-Add-failure-matching-to-common.qemu.patch +# For bz#1635583 - Quitting VM causes qemu core dump once the block mirror job paused for no enough target space +Patch410: kvm-block-iotest-to-catch-abort-on-forced-blockjob-cance.patch +# For bz#1609235 - Win2016 guest can't recognize pc-dimm hotplugged to node 0 +Patch411: kvm-Revert-hw-acpi-build-build-SRAT-memory-affinity-stru.patch +# For bz#1623085 - VM doesn't boot from HD +Patch413: kvm-aio-posix-Don-t-count-ctx-notifier-as-progress-when-.patch +# For bz#1623085 - VM doesn't boot from HD +Patch414: kvm-aio-Do-aio_notify_accept-only-during-blocking-aio_po.patch +# For bz#1632622 - ~40% virtio_blk disk performance drop for win2012r2 guest when comparing qemu-kvm-rhev-2.12.0-9 with qemu-kvm-rhev-2.12.0-12 +Patch415: kvm-aio-posix-fix-concurrent-access-to-poll_disable_cnt.patch +# For bz#1632622 - ~40% virtio_blk disk performance drop for win2012r2 guest when comparing qemu-kvm-rhev-2.12.0-9 with qemu-kvm-rhev-2.12.0-12 +Patch416: kvm-aio-posix-compute-timeout-before-polling.patch +# For bz#1632622 - ~40% virtio_blk disk performance drop for win2012r2 guest when comparing qemu-kvm-rhev-2.12.0-9 with qemu-kvm-rhev-2.12.0-12 +Patch417: kvm-aio-posix-do-skip-system-call-if-ctx-notifier-pollin.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch418: kvm-intel-iommu-send-PSI-always-even-if-across-PDEs.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch419: kvm-intel-iommu-remove-IntelIOMMUNotifierNode.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch420: kvm-intel-iommu-add-iommu-lock.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch421: kvm-intel-iommu-only-do-page-walk-for-MAP-notifiers.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch422: kvm-intel-iommu-introduce-vtd_page_walk_info.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch423: kvm-intel-iommu-pass-in-address-space-when-page-walk.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch424: kvm-intel-iommu-trace-domain-id-during-page-walk.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch425: kvm-util-implement-simple-iova-tree.patch +# For bz#1450712 - Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu) +Patch426: kvm-intel-iommu-rework-the-page-walk-logic.patch +# For bz#1633928 - CVE-2018-3639 qemu-kvm: hw: cpu: speculative store bypass [rhel-8.0] +Patch427: kvm-i386-define-the-ssbd-CPUID-feature-bit-CVE-2018-3639.patch +# For bz#1508142 - [IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part +Patch428: kvm-linux-headers-update.patch +# For bz#1508142 - [IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part +Patch429: kvm-s390x-cpumodel-Set-up-CPU-model-for-AP-device-suppor.patch +# For bz#1508142 - [IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part +Patch430: kvm-s390x-kvm-enable-AP-instruction-interpretation-for-g.patch +# For bz#1508142 - [IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part +Patch431: kvm-s390x-ap-base-Adjunct-Processor-AP-object-model.patch +# For bz#1508142 - [IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part +Patch432: kvm-s390x-vfio-ap-Introduce-VFIO-AP-device.patch +# For bz#1508142 - [IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part +Patch433: kvm-s390-doc-detailed-specifications-for-AP-virtualizati.patch +# For bz#1609327 - qemu-kvm[37046]: Could not find keytab file: /etc/qemu/krb5.tab: Unknown error 49408 +Patch434: kvm-vnc-call-sasl_server_init-only-when-required.patch +# For bz#1636142 - qemu NBD_CMD_CACHE flaws impacting non-qemu NBD clients +Patch435: kvm-nbd-server-fix-NBD_CMD_CACHE.patch +# For bz#1636142 - qemu NBD_CMD_CACHE flaws impacting non-qemu NBD clients +Patch436: kvm-nbd-fix-NBD_FLAG_SEND_CACHE-value.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch437: kvm-test-bdrv-drain-bdrv_drain-works-with-cross-AioConte.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch438: kvm-block-Use-bdrv_do_drain_begin-end-in-bdrv_drain_all.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch439: kvm-block-Remove-recursive-parameter-from-bdrv_drain_inv.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch440: kvm-block-Don-t-manually-poll-in-bdrv_drain_all.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch441: kvm-tests-test-bdrv-drain-bdrv_drain_all-works-in-corout.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch442: kvm-block-Avoid-unnecessary-aio_poll-in-AIO_WAIT_WHILE.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch443: kvm-block-Really-pause-block-jobs-on-drain.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch444: kvm-block-Remove-bdrv_drain_recurse.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch445: kvm-test-bdrv-drain-Add-test-for-node-deletion.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch446: kvm-block-Drain-recursively-with-a-single-BDRV_POLL_WHIL.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch447: kvm-test-bdrv-drain-Test-node-deletion-in-subtree-recurs.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch448: kvm-block-Don-t-poll-in-parent-drain-callbacks.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch449: kvm-test-bdrv-drain-Graph-change-through-parent-callback.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch450: kvm-block-Defer-.bdrv_drain_begin-callback-to-polling-ph.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch451: kvm-test-bdrv-drain-Test-that-bdrv_drain_invoke-doesn-t-.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch452: kvm-block-Allow-AIO_WAIT_WHILE-with-NULL-ctx.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch453: kvm-block-Move-bdrv_drain_all_begin-out-of-coroutine-con.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch454: kvm-block-ignore_bds_parents-parameter-for-drain-functio.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch455: kvm-block-Allow-graph-changes-in-bdrv_drain_all_begin-en.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch456: kvm-test-bdrv-drain-Test-graph-changes-in-drain_all-sect.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch457: kvm-block-Poll-after-drain-on-attaching-a-node.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch458: kvm-test-bdrv-drain-Test-bdrv_append-to-drained-node.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch459: kvm-block-linux-aio-acquire-AioContext-before-qemu_laio_.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch460: kvm-util-async-use-qemu_aio_coroutine_enter-in-co_schedu.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch461: kvm-job-Fix-nested-aio_poll-hanging-in-job_txn_apply.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch462: kvm-job-Fix-missing-locking-due-to-mismerge.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch463: kvm-blockjob-Wake-up-BDS-when-job-becomes-idle.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch464: kvm-aio-wait-Increase-num_waiters-even-in-home-thread.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch465: kvm-test-bdrv-drain-Drain-with-block-jobs-in-an-I-O-thre.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch466: kvm-test-blockjob-Acquire-AioContext-around-job_cancel_s.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch467: kvm-job-Use-AIO_WAIT_WHILE-in-job_finish_sync.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch468: kvm-test-bdrv-drain-Test-AIO_WAIT_WHILE-in-completion-ca.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch469: kvm-block-Add-missing-locking-in-bdrv_co_drain_bh_cb.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch470: kvm-block-backend-Add-.drained_poll-callback.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch471: kvm-block-backend-Fix-potential-double-blk_delete.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch472: kvm-block-backend-Decrease-in_flight-only-after-callback.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch473: kvm-blockjob-Lie-better-in-child_job_drained_poll.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch474: kvm-block-Remove-aio_poll-in-bdrv_drain_poll-variants.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch475: kvm-test-bdrv-drain-Test-nested-poll-in-bdrv_drain_poll_.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch476: kvm-job-Avoid-deadlocks-in-job_completed_txn_abort.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch477: kvm-test-bdrv-drain-AIO_WAIT_WHILE-in-job-.commit-.abort.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch478: kvm-test-bdrv-drain-Fix-outdated-comments.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch479: kvm-block-Use-a-single-global-AioWait.patch +# For bz#1637976 - Crashes and hangs with iothreads vs. block jobs +Patch480: kvm-test-bdrv-drain-Test-draining-job-source-child-and-p.patch +# For bz#1639374 - qemu-img map 'Aborted (core dumped)' when specifying a plain file +Patch481: kvm-qemu-img-Fix-assert-when-mapping-unaligned-raw-file.patch +# For bz#1639374 - qemu-img map 'Aborted (core dumped)' when specifying a plain file +Patch482: kvm-iotests-Add-test-221-to-catch-qemu-img-map-regressio.patch +# For bz#1635585 - rbd json format of 7.6 is incompatible with 7.5 +Patch483: kvm-block-rbd-pull-out-qemu_rbd_convert_options.patch +# For bz#1635585 - rbd json format of 7.6 is incompatible with 7.5 +Patch484: kvm-block-rbd-Attempt-to-parse-legacy-filenames.patch +# For bz#1635585 - rbd json format of 7.6 is incompatible with 7.5 +Patch485: kvm-block-rbd-add-deprecation-documentation-for-filename.patch +# For bz#1635585 - rbd json format of 7.6 is incompatible with 7.5 +Patch486: kvm-block-rbd-add-iotest-for-rbd-legacy-keyvalue-filenam.patch +# For bz#1629701 - "share-rw=on" does not work for luks format image - Fast Train +Patch487: kvm-luks-Allow-share-rw-on.patch +# For bz#1636185 - [RFE] Report disk device name and serial number (qemu-guest-agent on Linux) +Patch488: kvm-configure-add-test-for-libudev.patch +# For bz#1636185 - [RFE] Report disk device name and serial number (qemu-guest-agent on Linux) +Patch489: kvm-qga-linux-report-disk-serial-number.patch +# For bz#1636185 - [RFE] Report disk device name and serial number (qemu-guest-agent on Linux) +Patch490: kvm-qga-linux-return-disk-device-in-guest-get-fsinfo.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +Patch491: kvm-qemu-error-introduce-error-warn-_report_once.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +Patch492: kvm-intel-iommu-start-to-use-error_report_once.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +Patch493: kvm-intel-iommu-replace-more-vtd_err_-traces.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +Patch494: kvm-intel_iommu-introduce-vtd_reset_caches.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +Patch495: kvm-intel_iommu-better-handling-of-dmar-state-switch.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +# For bz#1629616 - boot guest with q35+vIOMMU+ device assignment, qemu terminal shows "qemu-kvm: VFIO_UNMAP_DMA: -22" when return assigned network devices from vfio driver to ixgbe in guest +Patch496: kvm-intel_iommu-move-ce-fetching-out-when-sync-shadow.patch +# For bz#1625173 - [NVMe Device Assignment] Guest could not boot up with q35+iommu +# For bz#1629616 - boot guest with q35+vIOMMU+ device assignment, qemu terminal shows "qemu-kvm: VFIO_UNMAP_DMA: -22" when return assigned network devices from vfio driver to ixgbe in guest +Patch497: kvm-intel_iommu-handle-invalid-ce-for-shadow-sync.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch498: kvm-block-remove-bdrv_dirty_bitmap_make_anon.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch499: kvm-block-simplify-code-around-releasing-bitmaps.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch500: kvm-hbitmap-Add-advance-param-to-hbitmap_iter_next.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch501: kvm-test-hbitmap-Add-non-advancing-iter_next-tests.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch502: kvm-block-dirty-bitmap-Add-bdrv_dirty_iter_next_area.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch503: kvm-blockdev-backup-add-bitmap-argument.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch504: kvm-dirty-bitmap-switch-assert-fails-to-errors-in-bdrv_m.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch505: kvm-dirty-bitmap-rename-bdrv_undo_clear_dirty_bitmap.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch506: kvm-dirty-bitmap-make-it-possible-to-restore-bitmap-afte.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch507: kvm-blockdev-rename-block-dirty-bitmap-clear-transaction.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch508: kvm-qapi-add-transaction-support-for-x-block-dirty-bitma.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch509: kvm-block-dirty-bitmaps-add-user_locked-status-checker.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch510: kvm-block-dirty-bitmaps-fix-merge-permissions.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch511: kvm-block-dirty-bitmaps-allow-clear-on-disabled-bitmaps.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch512: kvm-block-dirty-bitmaps-prohibit-enable-disable-on-locke.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch513: kvm-block-backup-prohibit-backup-from-using-in-use-bitma.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch514: kvm-nbd-forbid-use-of-frozen-bitmaps.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch515: kvm-bitmap-Update-count-after-a-merge.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch516: kvm-iotests-169-drop-deprecated-autoload-parameter.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch517: kvm-block-qcow2-improve-error-message-in-qcow2_inactivat.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch518: kvm-bloc-qcow2-drop-dirty_bitmaps_loaded-state-variable.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch519: kvm-dirty-bitmaps-clean-up-bitmaps-loading-and-migration.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch520: kvm-iotests-improve-169.patch +# For bz#1518989 - RFE: QEMU Incremental live backup +Patch521: kvm-iotests-169-add-cases-for-source-vm-resuming.patch +# For bz#1630116 - pc_dimm_get_free_addr: assertion failed: (QEMU_ALIGN_UP(address_space_start, align) == address_space_start) +Patch522: kvm-pc-dimm-turn-alignment-assert-into-check.patch +# For bz#1598842 - Compile out unused block drivers +Patch523: kvm-block-Make-more-block-drivers-compile-time-configura.patch +# For bz#1639069 - [IBM 8.0 FEAT] POWER9 - Nested virtualization in RHEL8.0 KVM for ppc64le - qemu-kvm side +Patch524: kvm-target-ppc-add-basic-support-for-PTCR-on-POWER9.patch +# For bz#1639069 - [IBM 8.0 FEAT] POWER9 - Nested virtualization in RHEL8.0 KVM for ppc64le - qemu-kvm side +Patch525: kvm-linux-headers-Update-for-nested-KVM-HV-downstream-on.patch +# For bz#1639069 - [IBM 8.0 FEAT] POWER9 - Nested virtualization in RHEL8.0 KVM for ppc64le - qemu-kvm side +Patch526: kvm-target-ppc-Add-one-reg-id-for-ptcr.patch +# For bz#1639069 - [IBM 8.0 FEAT] POWER9 - Nested virtualization in RHEL8.0 KVM for ppc64le - qemu-kvm side +Patch527: kvm-ppc-spapr_caps-Add-SPAPR_CAP_NESTED_KVM_HV.patch +# For bz#1651195 - Re-enable hyperv-testdev device +Patch528: kvm-Re-enable-CONFIG_HYPERV_TESTDEV.patch +# For bz#1610163 - guest shows border blurred screen with some resolutions when qemu boot with -device qxl-vga ,and guest on rhel7.6 has no such question +Patch529: kvm-qxl-use-guest_monitor_config-for-local-renderer.patch +# For bz#1651994 - Declare the "Cirrus VGA" device emulation of QEMU as deprecated in RHEL8 +Patch530: kvm-Declare-cirrus-vga-as-deprecated.patch +# For bz#1654651 - Qemu: hw: bt: keep bt/* objects from building [rhel-8.0] +Patch531: kvm-Do-not-build-bluetooth-support.patch +# For bz#1645840 - Qemu core dump when hotplug nvme:// drive via -blockdev +Patch532: kvm-vfio-helpers-Fix-qemu_vfio_open_pci-crash.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch533: kvm-balloon-Allow-multiple-inhibit-users.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch534: kvm-Use-inhibit-to-prevent-ballooning-without-synchr.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch535: kvm-vfio-Inhibit-ballooning-based-on-group-attachment-to.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch536: kvm-vfio-ccw-pci-Allow-devices-to-opt-in-for-ballooning.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch537: kvm-vfio-pci-Handle-subsystem-realpath-returning-NULL.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch538: kvm-vfio-pci-Fix-failure-to-close-file-descriptor-on-err.patch +# For bz#1650272 - Ballooning is incompatible with vfio assigned devices, but not prevented +Patch539: kvm-postcopy-Synchronize-usage-of-the-balloon-inhibitor.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch540: kvm-qcow2-Give-the-refcount-cache-the-minimum-possible-s.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch541: kvm-docs-Document-the-new-default-sizes-of-the-qcow2-cac.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch542: kvm-qcow2-Fix-Coverity-warning-when-calculating-the-refc.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch543: kvm-include-Add-IEC-binary-prefixes-in-qemu-units.h.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch544: kvm-qcow2-Options-documentation-fixes.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch545: kvm-include-Add-a-lookup-table-of-sizes.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch546: kvm-qcow2-Make-sizes-more-humanly-readable.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch547: kvm-qcow2-Avoid-duplication-in-setting-the-refcount-cach.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch548: kvm-qcow2-Assign-the-L2-cache-relatively-to-the-image-si.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch549: kvm-qcow2-Increase-the-default-upper-limit-on-the-L2-cac.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch550: kvm-qcow2-Resize-the-cache-upon-image-resizing.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch551: kvm-qcow2-Set-the-default-cache-clean-interval-to-10-min.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch552: kvm-qcow2-Explicit-number-replaced-by-a-constant.patch +# For bz#1657637 - Wrong werror default for -device drive= +Patch553: kvm-block-backend-Set-werror-rerror-defaults-in-blk_new.patch +# For bz#1656507 - [RHEL.8] qcow2 cache is too small +Patch554: kvm-qcow2-Fix-cache-clean-interval-documentation.patch +# For bz#1640044 - Disable CONFIG_I2C and CONFIG_IPMI in default-configs/ppc64-softmmu.mak +Patch555: kvm-Disable-CONFIG_IPMI-and-CONFIG_I2C-for-ppc64.patch +# For bz#1640042 - Disable CONFIG_CAN_BUS and CONFIG_CAN_SJA1000 config switches +Patch556: kvm-Disable-CONFIG_CAN_BUS-and-CONFIG_CAN_SJA1000.patch +# For bz#1639446 - Cross migration from RHEL7.5 to RHEL8 shouldn't fail with cpu flag stibp [qemu-kvm] +Patch562: kvm-i386-Add-stibp-flag-name.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch563: kvm-Add-functional-acceptance-tests-infrastructure.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch564: kvm-scripts-qemu.py-allow-adding-to-the-list-of-extra-ar.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch565: kvm-Acceptance-tests-add-quick-VNC-tests.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch566: kvm-scripts-qemu.py-introduce-set_console-method.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch567: kvm-Acceptance-tests-add-Linux-kernel-boot-and-console-c.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch568: kvm-Bootstrap-Python-venv-for-tests.patch +# For bz#1655807 - Backport avocado-qemu tests for QEMU 2.12 +Patch569: kvm-Acceptance-tests-add-make-rule-for-running-them.patch +# For bz#1654486 - [RFE] enable TPM passthrough at compile time (qemu-kvm) +Patch570: kvm-redhat-enable-tpmdev-passthrough.patch +# For bz#1598284 - [Intel 8.0 Alpha] physical bits should < 48 when host with 5level paging &EPT5 and qemu command with "-cpu qemu64" parameters. +Patch571: kvm-x86-host-phys-bits-limit-option.patch +# For bz#1598284 - [Intel 8.0 Alpha] physical bits should < 48 when host with 5level paging &EPT5 and qemu command with "-cpu qemu64" parameters. +Patch572: kvm-rhel-Set-host-phys-bits-limit-48-on-rhel-machine-typ.patch +# For bz#1659565 - machine type: required compat flag x-migrate-smi-count=off +Patch573: kvm-i386-do-not-migrate-MSR_SMI_COUNT-on-machine-types-2.patch +# For bz#1659565 - machine type: required compat flag x-migrate-smi-count=off +Patch574: kvm-pc-x-migrate-smi-count-to-PC_RHEL_COMPAT.patch +# For bz#1656829 - 8->7 migration failed: qemu-kvm: error: failed to set MSR 0x4b564d02 to 0x27fc13285 +Patch575: kvm-slow-train-kvm-clear-out-KVM_ASYNC_PF_DELIVERY_AS_PF.patch +# For bz#1652871 - QEMU doesn't expose rendernode option for egl-headless display type +Patch576: kvm-ui-add-qapi-parser-for-display.patch +# For bz#1652871 - QEMU doesn't expose rendernode option for egl-headless display type +Patch577: kvm-ui-switch-trivial-displays-to-qapi-parser.patch +# For bz#1652871 - QEMU doesn't expose rendernode option for egl-headless display type +Patch578: kvm-qapi-Add-rendernode-display-option-for-egl-headless.patch +# For bz#1652871 - QEMU doesn't expose rendernode option for egl-headless display type +Patch579: kvm-ui-Allow-specifying-rendernode-display-option-for-eg.patch +# For bz#1652871 - QEMU doesn't expose rendernode option for egl-headless display type +Patch580: kvm-qapi-add-query-display-options-command.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch581: kvm-s390x-tcg-avoid-overflows-in-time2tod-tod2time.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch582: kvm-s390x-kvm-pass-values-instead-of-pointers-to-kvm_s39.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch583: kvm-s390x-tod-factor-out-TOD-into-separate-device.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch584: kvm-s390x-tcg-drop-tod_basetime.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch585: kvm-s390x-tcg-properly-implement-the-TOD.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch586: kvm-s390x-tcg-SET-CLOCK-COMPARATOR-can-clear-CKC-interru.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch587: kvm-s390x-tcg-implement-SET-CLOCK.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch588: kvm-s390x-tcg-rearm-the-CKC-timer-during-migration.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch589: kvm-s390x-tcg-fix-locking-problem-with-tcg_s390_tod_upda.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch590: kvm-hw-s390x-Include-the-tod-qemu-also-for-builds-with-d.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch591: kvm-s390x-tod-Properly-stop-the-KVM-TOD-while-the-guest-.patch +# For bz#1653569 - Stress guest and stop it, then do live migration, guest hit call trace on destination end +Patch592: kvm-hw-s390x-Fix-bad-mask-in-time2tod.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch593: kvm-migration-discard-non-migratable-RAMBlocks.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch594: kvm-vfio-pci-do-not-set-the-PCIDevice-has_rom-attribute.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch595: kvm-memory-exec-Expose-all-memory-block-related-flags.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch596: kvm-memory-exec-switch-file-ram-allocation-functions-to-.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch597: kvm-configure-add-libpmem-support.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch598: kvm-hostmem-file-add-the-pmem-option.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch599: kvm-mem-nvdimm-ensure-write-persistence-to-PMEM-in-label.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch600: kvm-migration-ram-Add-check-and-info-message-to-nvdimm-p.patch +# For bz#1539285 - [Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM +Patch601: kvm-migration-ram-ensure-write-persistence-on-loading-al.patch +# For bz#1659395 - src qemu core dump when do migration ( block device node-name changed after change cdrom) - Slow Train +Patch602: kvm-block-Don-t-inactivate-children-before-parents.patch +# For bz#1659395 - src qemu core dump when do migration ( block device node-name changed after change cdrom) - Slow Train +Patch603: kvm-iotests-Test-migration-with-blockdev.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch604: kvm-block-Update-flags-in-bdrv_set_read_only.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch605: kvm-block-Add-auto-read-only-option.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch606: kvm-rbd-Close-image-in-qemu_rbd_open-error-path.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch607: kvm-block-Require-auto-read-only-for-existing-fallbacks.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch608: kvm-nbd-Support-auto-read-only-option.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch609: kvm-file-posix-Support-auto-read-only-option.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch610: kvm-curl-Support-auto-read-only-option.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch611: kvm-gluster-Support-auto-read-only-option.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch612: kvm-iscsi-Support-auto-read-only-option.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch613: kvm-block-Make-auto-read-only-on-default-for-drive.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch614: kvm-qemu-iotests-Test-auto-read-only-with-drive-and-bloc.patch +# For bz#1644996 - block-commit can't be used with -blockdev +Patch615: kvm-block-Fix-update-of-BDRV_O_AUTO_RDONLY-in-update_fla.patch +# For bz#1623082 - [rhel.8.0]Target files for 'qemu-img convert' do not support thin_provisoning with iscsi/nfs backend +Patch616: kvm-qemu-img-Add-C-option-for-convert-with-copy-offloadi.patch +# For bz#1623082 - [rhel.8.0]Target files for 'qemu-img convert' do not support thin_provisoning with iscsi/nfs backend +Patch617: kvm-iotests-Add-test-for-qemu-img-convert-C-compatibilit.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch618: kvm-hw-scsi-cleanups-before-VPD-BL-emulation.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch619: kvm-hw-scsi-centralize-SG_IO-calls-into-single-function.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch620: kvm-hw-scsi-add-VPD-Block-Limits-emulation.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch621: kvm-scsi-disk-Block-Device-Characteristics-emulation-fix.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch622: kvm-scsi-generic-keep-VPD-page-list-sorted.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch623: kvm-scsi-generic-avoid-out-of-bounds-access-to-VPD-page-.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch624: kvm-scsi-generic-avoid-invalid-access-to-struct-when-emu.patch +# For bz#1639957 - [RHEL.8] scsi host device passthrough limits IO writes - slow train +Patch625: kvm-scsi-generic-do-not-do-VPD-emulation-for-sense-other.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch626: kvm-ne2000-fix-possible-out-of-bound-access-in-ne2000_re.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch627: kvm-rtl8139-fix-possible-out-of-bound-access.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch628: kvm-pcnet-fix-possible-buffer-overflow.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch629: kvm-net-ignore-packet-size-greater-than-INT_MAX.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch630: kvm-net-drop-too-large-packet-early.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch631: kvm-net-hub-suppress-warnings-of-no-host-network-for-qte.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch632: kvm-virtio-net-test-accept-variable-length-argument-in-p.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch633: kvm-virtio-net-test-remove-unused-macro.patch +# For bz#1636784 - CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8] +Patch634: kvm-virtio-net-test-add-large-tx-buffer-test.patch +# For bz#1668261 - [RHEL8] Backport diag308 stable exception fix (qemu-kvm) +Patch635: kvm-s390x-Return-specification-exception-for-unimplement.patch +# For bz#1665844 - Guest quit with error when hotunplug cpu +Patch636: kvm-cpus-ignore-ESRCH-in-qemu_cpu_kick_thread.patch +# For bz#1666952 - qemu-guest-agent does not parse PCI bridge links in "build_guest_fsinfo_for_real_device" (q35) +Patch637: kvm-qemu-ga-make-get-fsinfo-work-over-pci-bridges.patch +# For bz#1666952 - qemu-guest-agent does not parse PCI bridge links in "build_guest_fsinfo_for_real_device" (q35) +Patch638: kvm-qga-fix-driver-leak-in-guest-get-fsinfo.patch +# For bz#1669069 - CVE-2019-6778 qemu-kvm: QEMU: slirp: heap buffer overflow in tcp_emu() [rhel-8.0] +Patch639: kvm-slirp-check-data-length-while-emulating-ident-functi.patch +# For bz#1668162 - CVE-2019-6501 qemu-kvm: QEMU: scsi-generic: possible OOB access while handling inquiry request [rhel-8] +Patch640: kvm-scsi-generic-avoid-possible-out-of-bounds-access-to-.patch + +BuildRequires: zlib-devel +BuildRequires: glib2-devel +BuildRequires: which +BuildRequires: gnutls-devel +BuildRequires: cyrus-sasl-devel +BuildRequires: libtool +BuildRequires: libaio-devel +BuildRequires: rsync +BuildRequires: python3-devel +BuildRequires: pciutils-devel +BuildRequires: libiscsi-devel +BuildRequires: ncurses-devel +BuildRequires: libattr-devel +BuildRequires: libusbx-devel >= 1.0.22 +%if %{have_usbredir} +BuildRequires: usbredir-devel >= 0.7.1 +%endif +BuildRequires: texinfo +%if %{have_spice} +BuildRequires: spice-protocol >= 0.12.12 +BuildRequires: spice-server-devel >= 0.12.8 +BuildRequires: libcacard-devel +# For smartcard NSS support +BuildRequires: nss-devel +%endif +BuildRequires: libseccomp-devel >= 2.3.0 +# For network block driver +BuildRequires: libcurl-devel +BuildRequires: libssh2-devel +BuildRequires: librados-devel +BuildRequires: librbd-devel +%if %{have_gluster} +# For gluster block driver +BuildRequires: glusterfs-api-devel >= 3.6.0 +BuildRequires: glusterfs-devel +%endif +# We need both because the 'stap' binary is probed for by configure +BuildRequires: systemtap +BuildRequires: systemtap-sdt-devel +# For VNC PNG support +BuildRequires: libpng-devel +# For uuid generation +BuildRequires: libuuid-devel +# For BlueZ device support +BuildRequires: bluez-libs-devel +# For Braille device support +BuildRequires: brlapi-devel +# For test suite +BuildRequires: check-devel +# For virtfs +BuildRequires: libcap-devel +# Hard requirement for version >= 1.3 +BuildRequires: pixman-devel +# Documentation requirement +BuildRequires: perl-podlators +BuildRequires: texinfo +# For rdma +%if 0%{?have_librdma} +BuildRequires: rdma-core-devel +%endif +%if %{have_fdt} +BuildRequires: libfdt-devel >= 1.4.3 +%endif +# iasl and cpp for acpi generation (not a hard requirement as we can use +# pre-compiled files, but it's better to use this) +%ifarch %{ix86} x86_64 +BuildRequires: iasl +BuildRequires: cpp +%endif +# For compressed guest memory dumps +BuildRequires: lzo-devel snappy-devel +# For NUMA memory binding +%ifnarch s390x +BuildRequires: numactl-devel +%endif +BuildRequires: libgcrypt-devel +# qemu-pr-helper multipath support (requires libudev too) +BuildRequires: device-mapper-multipath-devel +BuildRequires: systemd-devel +# used by qemu-bridge-helper and qemu-pr-helper +BuildRequires: libcap-ng-devel + +BuildRequires: diffutils +%ifarch x86_64 +BuildRequires: libpmem-devel +Requires: libpmem +%endif + +# qemu-keymap +BuildRequires: pkgconfig(xkbcommon) + +# For s390-pgste flag +%ifarch s390x +BuildRequires: binutils >= 2.27-16 +%endif + +%if %{have_opengl} +BuildRequires: pkgconfig(epoxy) +BuildRequires: pkgconfig(libdrm) +BuildRequires: pkgconfig(gbm) +Requires: mesa-libGL +Requires: mesa-libEGL +Requires: mesa-dri-drivers +%endif + +Requires: qemu-kvm-core = %{epoch}:%{version}-%{release} +%rhev_ma_conflicts qemu-kvm + +%{requires_all_modules} + +%define qemudocdir %{_docdir}/%{name} + +%description +qemu-kvm is an open source virtualizer that provides hardware +emulation for the KVM hypervisor. qemu-kvm acts as a virtual +machine monitor together with the KVM kernel modules, and emulates the +hardware for a full system such as a PC and its associated peripherals. + + +%package -n qemu-kvm-core +Summary: qemu-kvm core components +Requires: qemu-img = %{epoch}:%{version}-%{release} +%ifarch %{ix86} x86_64 +Requires: seabios-bin >= 1.10.2-1 +Requires: sgabios-bin +Requires: edk2-ovmf +%endif +%ifarch aarch64 +Requires: edk2-aarch64 +%endif + +%ifnarch aarch64 s390x +Requires: seavgabios-bin >= 1.10.2-1 +Requires: ipxe-roms-qemu >= 20170123-1 +%endif +%ifarch %{power64} +Requires: SLOF >= %{SLOF_gittagdate}-1.git%{SLOF_gittagcommit} +%endif +Requires: %{name}-common = %{epoch}:%{version}-%{release} +Requires: libseccomp >= 2.3.0 +# For compressed guest memory dumps +Requires: lzo snappy +%if %{have_gluster} +Requires: glusterfs-api >= 3.6.0 +%endif +%if %{have_kvm_setup} +Requires(post): systemd-units + %ifarch %{power64} +Requires: powerpc-utils + %endif +%endif +Requires: libusbx >= 1.0.19 +%if %{have_usbredir} +Requires: usbredir >= 0.7.1 +%endif + +%rhev_ma_conflicts qemu-kvm + +%description -n qemu-kvm-core +qemu-kvm is an open source virtualizer that provides hardware +emulation for the KVM hypervisor. qemu-kvm acts as a virtual +machine monitor together with the KVM kernel modules, and emulates the +hardware for a full system such as a PC and its associated peripherals. + + +%package -n qemu-img +Summary: QEMU command line tool for manipulating disk images +Group: Development/Tools + +%rhev_ma_conflicts qemu-img + +%description -n qemu-img +This package provides a command line tool for manipulating disk images. + +%package -n qemu-kvm-common +Summary: QEMU common files needed by all QEMU targets +Group: Development/Tools +Requires(post): /usr/bin/getent +Requires(post): /usr/sbin/groupadd +Requires(post): /usr/sbin/useradd +Requires(post): systemd-units +Requires(preun): systemd-units +Requires(postun): systemd-units + +%rhev_ma_conflicts qemu-kvm-common + +%description -n qemu-kvm-common +qemu-kvm is an open source virtualizer that provides hardware emulation for +the KVM hypervisor. + +This package provides documentation and auxiliary programs used with qemu-kvm. + + +%package -n qemu-guest-agent +Summary: QEMU guest agent +Requires(post): systemd-units +Requires(preun): systemd-units +Requires(postun): systemd-units + +%description -n qemu-guest-agent +qemu-kvm is an open source virtualizer that provides hardware emulation for +the KVM hypervisor. + +This package provides an agent to run inside guests, which communicates +with the host over a virtio-serial channel named "org.qemu.guest_agent.0" + +This package does not need to be installed on the host OS. + +%package tests +Summary: tests for the qemu-kvm package +Requires: %{name} = %{epoch}:%{version}-%{release} + +%define testsdir %{_libdir}/%{name}/tests-src + +%description tests +The qemu-kvm-tests rpm contains tests that can be used to verify +the functionality of the installed qemu-kvm package + +Install this package if you want access to the avocado_qemu +tests, or qemu-iotests. + +%package block-curl +Summary: QEMU CURL block driver +Requires: %{name}-common%{?_isa} = %{epoch}:%{version}-%{release} + +%description block-curl +This package provides the additional CURL block driver for QEMU. + +Install this package if you want to access remote disks over +http, https, ftp and other transports provided by the CURL library. + + +%if %{have_gluster} +%package block-gluster +Summary: QEMU Gluster block driver +Requires: %{name}-common%{?_isa} = %{epoch}:%{version}-%{release} +%description block-gluster +This package provides the additional Gluster block driver for QEMU. + +Install this package if you want to access remote Gluster storage. +%endif + + +%package block-iscsi +Summary: QEMU iSCSI block driver +Requires: %{name}-common%{?_isa} = %{epoch}:%{version}-%{release} + +%description block-iscsi +This package provides the additional iSCSI block driver for QEMU. + +Install this package if you want to access iSCSI volumes. + + +%package block-rbd +Summary: QEMU Ceph/RBD block driver +Requires: %{name}-common%{?_isa} = %{epoch}:%{version}-%{release} + +%description block-rbd +This package provides the additional Ceph/RBD block driver for QEMU. + +Install this package if you want to access remote Ceph volumes +using the rbd protocol. + + +%package block-ssh +Summary: QEMU SSH block driver +Requires: %{name}-common%{?_isa} = %{epoch}:%{version}-%{release} + +%description block-ssh +This package provides the additional SSH block driver for QEMU. + +Install this package if you want to access remote disks using +the Secure Shell (SSH) protocol. + + +%prep +%setup -q -n qemu-%{version} +%autopatch -p1 + +%build +%global buildarch %{kvm_target}-softmmu + +# --build-id option is used for giving info to the debug packages. +buildldflags="VL_LDFLAGS=-Wl,--build-id" + +%global block_drivers_list qcow2,raw,file,host_device,nbd,iscsi,rbd,blkdebug,luks,null-co,nvme,copy-on-read,throttle + +%if 0%{have_vxhs} + %global block_drivers_list %{block_drivers_list},vxhs +%endif +%if 0%{have_gluster} + %global block_drivers_list %{block_drivers_list},gluster +%endif + +./configure \ + --prefix="%{_prefix}" \ + --libdir="%{_libdir}" \ + --sysconfdir="%{_sysconfdir}" \ + --interp-prefix=%{_prefix}/qemu-%M \ + --localstatedir="%{_localstatedir}" \ + --docdir="%{qemudocdir}" \ + --libexecdir="%{_libexecdir}" \ + --extra-ldflags="-Wl,--build-id -Wl,-z,relro -Wl,-z,now" \ + --extra-cflags="%{optflags}" \ + --with-pkgversion="%{name}-%{version}-%{release}" \ + --with-confsuffix=/"%{name}" \ + --firmwarepath=%{_prefix}/share/qemu-firmware \ +%if 0%{have_fdt} + --enable-fdt \ +%else + --disable-fdt \ + %endif +%if 0%{have_gluster} + --enable-glusterfs \ +%else + --disable-glusterfs \ +%endif + --enable-guest-agent \ +%ifnarch s390x + --enable-numa \ +%else + --disable-numa \ +%endif + --enable-rbd \ +%if 0%{have_librdma} + --enable-rdma \ +%else + --disable-rdma \ +%endif + --enable-seccomp \ +%if 0%{have_spice} + --enable-spice \ + --enable-smartcard \ +%else + --disable-spice \ + --disable-smartcard \ +%endif +%if 0%{have_opengl} + --enable-opengl \ +%else + --disable-opengl \ +%endif +%if 0%{have_usbredir} + --enable-usb-redir \ +%else + --disable-usb-redir \ +%endif + --disable-tcmalloc \ +%if 0%{have_vxhs} + --enable-vxhs \ +%else + --disable-vxhs \ +%endif +%ifarch x86_64 + --enable-libpmem \ +%else + --disable-libpmem \ +%endif + --enable-vhost-user \ + --python=%{__python3} \ + --target-list="%{buildarch}" \ + --block-drv-rw-whitelist=%{block_drivers_list} \ + --audio-drv-list= \ + --block-drv-ro-whitelist=vmdk,vhdx,vpc,https,ssh \ + --with-coroutine=ucontext \ + --tls-priority=NORMAL \ + --disable-bluez \ + --disable-brlapi \ + --disable-cap-ng \ + --enable-coroutine-pool \ + --enable-curl \ + --disable-curses \ + --disable-debug-tcg \ + --enable-docs \ + --disable-gtk \ + --enable-kvm \ + --enable-libiscsi \ + --disable-libnfs \ + --enable-libssh2 \ + --enable-libusb \ + --disable-bzip2 \ + --enable-linux-aio \ + --disable-live-block-migration \ + --enable-lzo \ + --enable-pie \ + --disable-qom-cast-debug \ + --disable-sdl \ + --enable-snappy \ + --disable-sparse \ + --disable-strip \ + --enable-tpm \ + --enable-trace-backend=dtrace \ + --disable-vde \ + --disable-vhost-scsi \ + --disable-virtfs \ + --disable-vnc-jpeg \ + --disable-vte \ + --enable-vnc-png \ + --enable-vnc-sasl \ + --enable-werror \ + --disable-xen \ + --disable-xfsctl \ + --enable-gnutls \ + --enable-gcrypt \ + --disable-nettle \ + --enable-attr \ + --disable-bsd-user \ + --disable-cocoa \ + --enable-debug-info \ + --disable-guest-agent-msi \ + --disable-hax \ + --disable-jemalloc \ + --disable-linux-user \ + --enable-modules \ + --disable-netmap \ + --disable-replication \ + --enable-system \ + --enable-tools \ + --disable-user \ + --enable-vhost-net \ + --enable-vhost-vsock \ + --enable-vnc \ + --enable-mpath \ + --disable-virglrenderer \ + --disable-xen-pci-passthrough \ + --enable-tcg \ + --with-git=git \ + --disable-sanitizers \ + --disable-hvf \ + --disable-whpx \ + --enable-malloc-trim \ + --disable-membarrier \ + --disable-vhost-crypto \ + --disable-libxml2 \ + --enable-capstone \ + --disable-git-update \ + --disable-crypto-afalg \ + --disable-bochs \ + --disable-cloop \ + --disable-dmg \ + --disable-qcow1 \ + --disable-vdi \ + --disable-vvfat \ + --disable-qed \ + --disable-parallels \ + --disable-sheepdog + + +echo "config-host.mak contents:" +echo "===" +cat config-host.mak +echo "===" + +make V=1 %{?_smp_mflags} $buildldflags + +# Setup back compat qemu-kvm binary +%{__python3} scripts/tracetool.py --backend dtrace --format stap --group=all \ + --binary %{_libexecdir}/qemu-kvm --target-name %{kvm_target} \ + --target-type system --probe-prefix \ + qemu.kvm trace-events-all > qemu-kvm.stp + +%{__python3} scripts/tracetool.py --backend dtrace --format simpletrace-stap \ + --group=all --binary %{_libexecdir}/qemu-kvm --target-name %{kvm_target} \ + --target-type system --probe-prefix \ + qemu.kvm trace-events-all > qemu-kvm-simpletrace.stp + +cp -a %{kvm_target}-softmmu/qemu-system-%{kvm_target} qemu-kvm + +gcc %{SOURCE6} $RPM_OPT_FLAGS $RPM_LD_FLAGS -o ksmctl +gcc %{SOURCE35} $RPM_OPT_FLAGS $RPM_LD_FLAGS -o udev-kvm-check + +%install +%define _udevdir %(pkg-config --variable=udevdir udev) +%define _udevrulesdir %{_udevdir}/rules.d + +install -D -p -m 0644 %{SOURCE4} $RPM_BUILD_ROOT%{_unitdir}/ksm.service +install -D -p -m 0644 %{SOURCE5} $RPM_BUILD_ROOT%{_sysconfdir}/sysconfig/ksm +install -D -p -m 0755 ksmctl $RPM_BUILD_ROOT%{_libexecdir}/ksmctl + +install -D -p -m 0644 %{SOURCE7} $RPM_BUILD_ROOT%{_unitdir}/ksmtuned.service +install -D -p -m 0755 %{SOURCE8} $RPM_BUILD_ROOT%{_sbindir}/ksmtuned +install -D -p -m 0644 %{SOURCE9} $RPM_BUILD_ROOT%{_sysconfdir}/ksmtuned.conf +install -D -p -m 0644 %{SOURCE26} $RPM_BUILD_ROOT%{_sysconfdir}/modprobe.d/vhost.conf +%ifarch s390x + install -D -p -m 0644 %{SOURCE30} $RPM_BUILD_ROOT%{_sysconfdir}/modprobe.d/kvm.conf +%else +%ifarch %{ix86} x86_64 + install -D -p -m 0644 %{SOURCE31} $RPM_BUILD_ROOT%{_sysconfdir}/modprobe.d/kvm.conf +%else + install -D -p -m 0644 %{SOURCE27} $RPM_BUILD_ROOT%{_sysconfdir}/modprobe.d/kvm.conf +%endif +%endif + +mkdir -p $RPM_BUILD_ROOT%{_bindir}/ +mkdir -p $RPM_BUILD_ROOT%{_udevrulesdir}/ +mkdir -p $RPM_BUILD_ROOT%{_datadir}/%{name} + +# Create new directories and put them all under tests-src +mkdir -p $RPM_BUILD_ROOT%{testsdir}/tests/ +mkdir -p $RPM_BUILD_ROOT%{testsdir}/tests/acceptance +mkdir -p $RPM_BUILD_ROOT%{testsdir}/tests/qemu-iotests +mkdir -p $RPM_BUILD_ROOT%{testsdir}/scripts +mkdir -p $RPM_BUILD_ROOT%{testsdir}/scripts/qmp + +install -p -m 0755 udev-kvm-check $RPM_BUILD_ROOT%{_udevdir} +install -p -m 0644 %{SOURCE34} $RPM_BUILD_ROOT%{_udevrulesdir} + +install -m 0644 scripts/dump-guest-memory.py \ + $RPM_BUILD_ROOT%{_datadir}/%{name} + +# Install avocado_qemu tests +cp -R tests/acceptance/* $RPM_BUILD_ROOT%{testsdir}/tests/acceptance/ + +# Install qemu.py and qmp/ scripts required to run avocado_qemu tests +install -p -m 0644 scripts/qemu.py $RPM_BUILD_ROOT%{testsdir}/scripts/ +cp -R scripts/qmp/* $RPM_BUILD_ROOT%{testsdir}/scripts/qmp +install -p -m 0755 tests/Makefile.include $RPM_BUILD_ROOT%{testsdir}/tests/ + +# Install qemu-iotests +cp -R tests/qemu-iotests/* $RPM_BUILD_ROOT%{testsdir}/tests/qemu-iotests/ +# Avoid ambiguous 'python' interpreter name +find $RPM_BUILD_ROOT%{testsdir}/tests/qemu-iotests/* -maxdepth 1 -type f -exec sed -i -e '1 s/python/python3/' {} \; +find $RPM_BUILD_ROOT%{testsdir}/scripts/qmp/* -maxdepth 1 -type f -exec sed -i -e '1 s/python/python3/' {} \; + +install -p -m 0644 %{SOURCE36} $RPM_BUILD_ROOT%{testsdir}/README + +make DESTDIR=$RPM_BUILD_ROOT \ + sharedir="%{_datadir}/%{name}" \ + datadir="%{_datadir}/%{name}" \ + install + +mkdir -p $RPM_BUILD_ROOT%{_datadir}/systemtap/tapset + +# Install qemu-guest-agent service and udev rules +install -m 0644 %{_sourcedir}/qemu-guest-agent.service %{buildroot}%{_unitdir} +install -m 0644 %{_sourcedir}/qemu-ga.sysconfig %{buildroot}%{_sysconfdir}/sysconfig/qemu-ga +install -m 0644 %{_sourcedir}/99-qemu-guest-agent.rules %{buildroot}%{_udevrulesdir} + +# - the fsfreeze hook script: +install -D --preserve-timestamps \ + scripts/qemu-guest-agent/fsfreeze-hook \ + $RPM_BUILD_ROOT%{_sysconfdir}/qemu-ga/fsfreeze-hook + +# - the directory for user scripts: +mkdir $RPM_BUILD_ROOT%{_sysconfdir}/qemu-ga/fsfreeze-hook.d + +# - and the fsfreeze script samples: +mkdir --parents $RPM_BUILD_ROOT%{_datadir}/%{name}/qemu-ga/fsfreeze-hook.d/ +install --preserve-timestamps --mode=0644 \ + scripts/qemu-guest-agent/fsfreeze-hook.d/*.sample \ + $RPM_BUILD_ROOT%{_datadir}/%{name}/qemu-ga/fsfreeze-hook.d/ + +# - Install dedicated log directory: +mkdir -p -v $RPM_BUILD_ROOT%{_localstatedir}/log/qemu-ga/ + +mkdir -p $RPM_BUILD_ROOT%{_bindir} +install -c -m 0755 qemu-ga ${RPM_BUILD_ROOT}%{_bindir}/qemu-ga + +mkdir -p $RPM_BUILD_ROOT%{_mandir}/man8 +install -m 0644 qemu-ga.8 ${RPM_BUILD_ROOT}%{_mandir}/man8/ + + +install -m 0755 qemu-kvm $RPM_BUILD_ROOT%{_libexecdir}/ +install -m 0644 qemu-kvm.stp $RPM_BUILD_ROOT%{_datadir}/systemtap/tapset/ +install -m 0644 qemu-kvm-simpletrace.stp $RPM_BUILD_ROOT%{_datadir}/systemtap/tapset/ + +rm $RPM_BUILD_ROOT%{_bindir}/qemu-system-%{kvm_target} +rm $RPM_BUILD_ROOT%{_datadir}/systemtap/tapset/qemu-system-%{kvm_target}.stp +rm $RPM_BUILD_ROOT%{_datadir}/systemtap/tapset/qemu-system-%{kvm_target}-simpletrace.stp + +# Install simpletrace +install -m 0755 scripts/simpletrace.py $RPM_BUILD_ROOT%{_datadir}/%{name}/simpletrace.py +# Avoid ambiguous 'python' interpreter name +sed -i -e '1 s/python/python3/' $RPM_BUILD_ROOT%{_datadir}/%{name}/simpletrace.py +mkdir -p $RPM_BUILD_ROOT%{_datadir}/%{name}/tracetool +install -m 0644 -t $RPM_BUILD_ROOT%{_datadir}/%{name}/tracetool scripts/tracetool/*.py +mkdir -p $RPM_BUILD_ROOT%{_datadir}/%{name}/tracetool/backend +install -m 0644 -t $RPM_BUILD_ROOT%{_datadir}/%{name}/tracetool/backend scripts/tracetool/backend/*.py +mkdir -p $RPM_BUILD_ROOT%{_datadir}/%{name}/tracetool/format +install -m 0644 -t $RPM_BUILD_ROOT%{_datadir}/%{name}/tracetool/format scripts/tracetool/format/*.py + +mkdir -p $RPM_BUILD_ROOT%{qemudocdir} +install -p -m 0644 -t ${RPM_BUILD_ROOT}%{qemudocdir} Changelog README README.systemtap COPYING COPYING.LIB LICENSE docs/interop/qmp-spec.txt +chmod -x ${RPM_BUILD_ROOT}%{_mandir}/man1/* +chmod -x ${RPM_BUILD_ROOT}%{_mandir}/man8/* + +install -D -p -m 0644 qemu.sasl $RPM_BUILD_ROOT%{_sysconfdir}/sasl2/%{name}.conf + +# Provided by package openbios +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/openbios-ppc +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/openbios-sparc32 +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/openbios-sparc64 +# Provided by package SLOF +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/slof.bin + +# Remove unpackaged files. +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/palcode-clipper +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/petalogix*.dtb +rm -f ${RPM_BUILD_ROOT}%{_datadir}/%{name}/bamboo.dtb +rm -f ${RPM_BUILD_ROOT}%{_datadir}/%{name}/ppc_rom.bin +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/s390-zipl.rom +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/u-boot.e500 +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/qemu_vga.ndrv +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/skiboot.lid + +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/s390-ccw.img +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/hppa-firmware.img +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/canyonlands.dtb +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/u-boot-sam460-20100605.bin + +%ifarch s390x + # Use the s390-ccw.img that we've just built, not the pre-built one + install -m 0644 pc-bios/s390-ccw/s390-ccw.img $RPM_BUILD_ROOT%{_datadir}/%{name}/ +%else + rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/s390-netboot.img +%endif + +%ifnarch %{power64} + rm -f ${RPM_BUILD_ROOT}%{_datadir}/%{name}/spapr-rtas.bin +%endif + +%ifnarch x86_64 + rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/kvmvapic.bin + rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/linuxboot.bin + rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/multiboot.bin +%endif + +# Remove sparc files +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/QEMU,tcx.bin +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/QEMU,cgthree.bin + +# Remove ivshmem example programs +rm -rf ${RPM_BUILD_ROOT}%{_bindir}/ivshmem-client +rm -rf ${RPM_BUILD_ROOT}%{_bindir}/ivshmem-server + +# Remove efi roms +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/efi*.rom + +# Provided by package ipxe +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/pxe*rom +# Provided by package vgabios +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/vgabios*bin +# Provided by package seabios +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/bios*.bin +# Provided by package sgabios +rm -rf ${RPM_BUILD_ROOT}%{_datadir}/%{name}/sgabios.bin + +# the pxe gpxe images will be symlinks to the images on +# /usr/share/ipxe, as QEMU doesn't know how to look +# for other paths, yet. +pxe_link() { + ln -s ../ipxe.efi/$2.rom %{buildroot}%{_datadir}/%{name}/efi-$1.rom +} + +%ifnarch aarch64 s390x +pxe_link e1000 8086100e +pxe_link ne2k_pci 10ec8029 +pxe_link pcnet 10222000 +pxe_link rtl8139 10ec8139 +pxe_link virtio 1af41000 +pxe_link e1000e 808610d3 +%endif + +rom_link() { + ln -s $1 %{buildroot}%{_datadir}/%{name}/$2 +} + +%ifnarch aarch64 s390x + rom_link ../seavgabios/vgabios-isavga.bin vgabios.bin + rom_link ../seavgabios/vgabios-cirrus.bin vgabios-cirrus.bin + rom_link ../seavgabios/vgabios-qxl.bin vgabios-qxl.bin + rom_link ../seavgabios/vgabios-stdvga.bin vgabios-stdvga.bin + rom_link ../seavgabios/vgabios-vmware.bin vgabios-vmware.bin + rom_link ../seavgabios/vgabios-virtio.bin vgabios-virtio.bin +%endif +%ifarch x86_64 + rom_link ../seabios/bios.bin bios.bin + rom_link ../seabios/bios-256k.bin bios-256k.bin + rom_link ../sgabios/sgabios.bin sgabios.bin +%endif + +%if 0%{have_kvm_setup} + install -D -p -m 755 %{SOURCE21} $RPM_BUILD_ROOT%{_prefix}/lib/systemd/kvm-setup + install -D -p -m 644 %{SOURCE22} $RPM_BUILD_ROOT%{_unitdir}/kvm-setup.service + install -D -p -m 644 %{SOURCE23} $RPM_BUILD_ROOT%{_presetdir}/85-kvm.preset +%endif + +%if 0%{have_memlock_limits} + install -D -p -m 644 %{SOURCE28} $RPM_BUILD_ROOT%{_sysconfdir}/security/limits.d/95-kvm-memlock.conf +%endif + +# Install rules to use the bridge helper with libvirt's virbr0 +install -D -m 0644 %{SOURCE12} $RPM_BUILD_ROOT%{_sysconfdir}/%{name}/bridge.conf + +# Install qemu-pr-helper service +install -m 0644 %{_sourcedir}/qemu-pr-helper.service %{buildroot}%{_unitdir} +install -m 0644 %{_sourcedir}/qemu-pr-helper.socket %{buildroot}%{_unitdir} + +find $RPM_BUILD_ROOT -name '*.la' -or -name '*.a' | xargs rm -f + +# We need to make the block device modules executable else +# RPM won't pick up their dependencies. +chmod +x $RPM_BUILD_ROOT%{_libdir}/qemu-kvm/block-*.so + +%check +export DIFF=diff; make check V=1 +pushd tests/qemu-iotests +./check -v -raw 001 002 004 005 008 009 010 011 012 021 025 032 033 048 052 063 077 086 101 106 120 140 143 145 150 159 160 162 170 171 175 184 221 226 ||: +./check -v -qcow2 001 002 004 005 008 009 010 011 012 017 018 019 020 021 024 025 027 028 029 032 033 034 035 037 038 042 046 047 048 050 052 053 058 062 063 066 068 069 072 073 074 086 087 089 090 095 098 102 103 105 107 108 110 111 120 127 133 134 138 140 141 143 144 145 150 154 156 158 159 162 170 177 179 182 184 188 190 195 204 209 217 226 ||: +popd + +%post -n qemu-kvm-core +# load kvm modules now, so we can make sure no reboot is needed. +# If there's already a kvm module installed, we don't mess with it +%udev_rules_update +sh %{_sysconfdir}/sysconfig/modules/kvm.modules &> /dev/null || : + udevadm trigger --subsystem-match=misc --sysname-match=kvm --action=add || : +%if %{have_kvm_setup} + systemctl daemon-reload # Make sure it sees the new presets and unitfile + %systemd_post kvm-setup.service + if systemctl is-enabled kvm-setup.service > /dev/null; then + systemctl start kvm-setup.service + fi +%endif + +%post -n qemu-kvm-common +%systemd_post ksm.service +%systemd_post ksmtuned.service + +getent group kvm >/dev/null || groupadd -g 36 -r kvm +getent group qemu >/dev/null || groupadd -g 107 -r qemu +getent passwd qemu >/dev/null || \ +useradd -r -u 107 -g qemu -G kvm -d / -s /sbin/nologin \ + -c "qemu user" qemu + +%preun -n qemu-kvm-common +%systemd_preun ksm.service +%systemd_preun ksmtuned.service + +%postun -n qemu-kvm-common +%systemd_postun_with_restart ksm.service +%systemd_postun_with_restart ksmtuned.service + +%global qemu_kvm_files \ +%{_libexecdir}/qemu-kvm \ +%{_datadir}/systemtap/tapset/qemu-kvm.stp \ +%{_datadir}/%{name}/trace-events-all \ +%{_datadir}/systemtap/tapset/qemu-kvm-simpletrace.stp \ +%{_datadir}/%{name}/systemtap/script.d/qemu_kvm.stp \ +%{_datadir}/%{name}/systemtap/conf.d/qemu_kvm.conf + +%files +# Deliberately empty + + +%files -n qemu-kvm-common +%defattr(-,root,root) +%dir %{qemudocdir} +%doc %{qemudocdir}/Changelog +%doc %{qemudocdir}/README +%doc %{qemudocdir}/qemu-doc.html +%doc %{qemudocdir}/COPYING +%doc %{qemudocdir}/COPYING.LIB +%doc %{qemudocdir}/LICENSE +%doc %{qemudocdir}/README.systemtap +%doc %{qemudocdir}/qmp-spec.txt +%doc %{qemudocdir}/qemu-doc.txt +%doc %{qemudocdir}/qemu-ga-ref.html +%doc %{qemudocdir}/qemu-ga-ref.txt +%doc %{qemudocdir}/qemu-qmp-ref.html +%doc %{qemudocdir}/qemu-qmp-ref.txt +%{_mandir}/man7/qemu-qmp-ref.7* +%{_bindir}/qemu-keymap +%{_bindir}/qemu-pr-helper +%{_unitdir}/qemu-pr-helper.service +%{_unitdir}/qemu-pr-helper.socket +%{_mandir}/man7/qemu-ga-ref.7* + +%dir %{_datadir}/%{name}/ +%{_datadir}/%{name}/keymaps/ +%{_mandir}/man1/%{name}.1* +%{_mandir}/man7/qemu-block-drivers.7* +%attr(4755, -, -) %{_libexecdir}/qemu-bridge-helper +%config(noreplace) %{_sysconfdir}/sasl2/%{name}.conf +%{_unitdir}/ksm.service +%{_libexecdir}/ksmctl +%config(noreplace) %{_sysconfdir}/sysconfig/ksm +%{_unitdir}/ksmtuned.service +%{_sbindir}/ksmtuned +%{_udevdir}/udev-kvm-check +%{_udevrulesdir}/81-kvm-rhel.rules +%ghost %{_sysconfdir}/kvm +%config(noreplace) %{_sysconfdir}/ksmtuned.conf +%dir %{_sysconfdir}/%{name} +%config(noreplace) %{_sysconfdir}/%{name}/bridge.conf +%config(noreplace) %{_sysconfdir}/modprobe.d/vhost.conf +%config(noreplace) %{_sysconfdir}/modprobe.d/kvm.conf +%{_datadir}/%{name}/simpletrace.py* +%{_datadir}/%{name}/tracetool/*.py* +%{_datadir}/%{name}/tracetool/backend/*.py* +%{_datadir}/%{name}/tracetool/format/*.py* + +%files -n qemu-kvm-core +%defattr(-,root,root) +%ifarch x86_64 + %{_datadir}/%{name}/bios.bin + %{_datadir}/%{name}/bios-256k.bin + %{_datadir}/%{name}/linuxboot.bin + %{_datadir}/%{name}/multiboot.bin + %{_datadir}/%{name}/kvmvapic.bin + %{_datadir}/%{name}/sgabios.bin +%endif +%ifarch s390x + %{_datadir}/%{name}/s390-ccw.img + %{_datadir}/%{name}/s390-netboot.img +%endif +%ifnarch aarch64 s390x + %{_datadir}/%{name}/vgabios.bin + %{_datadir}/%{name}/vgabios-cirrus.bin + %{_datadir}/%{name}/vgabios-qxl.bin + %{_datadir}/%{name}/vgabios-stdvga.bin + %{_datadir}/%{name}/vgabios-vmware.bin + %{_datadir}/%{name}/vgabios-virtio.bin + %{_datadir}/%{name}/efi-e1000.rom + %{_datadir}/%{name}/efi-e1000e.rom + %{_datadir}/%{name}/efi-virtio.rom + %{_datadir}/%{name}/efi-pcnet.rom + %{_datadir}/%{name}/efi-rtl8139.rom + %{_datadir}/%{name}/efi-ne2k_pci.rom +%endif +%{_datadir}/%{name}/qemu-icon.bmp +%{_datadir}/%{name}/qemu_logo_no_text.svg +%{_datadir}/%{name}/linuxboot_dma.bin +%{_datadir}/%{name}/dump-guest-memory.py* +%ifarch %{power64} + %{_datadir}/%{name}/spapr-rtas.bin +%endif +%{?qemu_kvm_files:} +%if 0%{have_kvm_setup} + %{_prefix}/lib/systemd/kvm-setup + %{_unitdir}/kvm-setup.service + %{_presetdir}/85-kvm.preset +%endif +%if 0%{have_memlock_limits} + %{_sysconfdir}/security/limits.d/95-kvm-memlock.conf +%endif + +%files -n qemu-img +%defattr(-,root,root) +%{_bindir}/qemu-img +%{_bindir}/qemu-io +%{_bindir}/qemu-nbd +%{_mandir}/man1/qemu-img.1* +%{_mandir}/man8/qemu-nbd.8* + +%files -n qemu-guest-agent +%defattr(-,root,root,-) +%doc COPYING README +%{_bindir}/qemu-ga +%{_mandir}/man8/qemu-ga.8* +%{_unitdir}/qemu-guest-agent.service +%{_udevrulesdir}/99-qemu-guest-agent.rules +%config(noreplace) %{_sysconfdir}/sysconfig/qemu-ga +%{_sysconfdir}/qemu-ga +%{_datadir}/%{name}/qemu-ga +%dir %{_localstatedir}/log/qemu-ga + +%files tests +%{testsdir} + +%files block-curl +%{_libdir}/qemu-kvm/block-curl.so + +%if %{have_gluster} +%files block-gluster +%{_libdir}/qemu-kvm/block-gluster.so +%endif + +%files block-iscsi +%{_libdir}/qemu-kvm/block-iscsi.so + +%files block-rbd +%{_libdir}/qemu-kvm/block-rbd.so + +%files block-ssh +%{_libdir}/qemu-kvm/block-ssh.so + + +%changelog +* Tue Feb 26 2019 Danilo Cesar Lemes de Paula - 2.12.0-63.el8 +- kvm-scsi-generic-avoid-possible-out-of-bounds-access-to-.patch [bz#1668162] +- Resolves: bz#1668162 + (CVE-2019-6501 qemu-kvm: QEMU: scsi-generic: possible OOB access while handling inquiry request [rhel-8]) + +* Mon Feb 25 2019 Danilo Cesar Lemes de Paula - 2.12.0-62.el8 +- kvm-slirp-check-data-length-while-emulating-ident-functi.patch [bz#1669069] +- Resolves: bz#1669069 + (CVE-2019-6778 qemu-kvm: QEMU: slirp: heap buffer overflow in tcp_emu() [rhel-8.0]) + +* Mon Feb 11 2019 Danilo Cesar Lemes de Paula - 2.12.0-61.el8 +- kvm-qemu-ga-make-get-fsinfo-work-over-pci-bridges.patch [bz#1666952] +- kvm-qga-fix-driver-leak-in-guest-get-fsinfo.patch [bz#1666952] +- Resolves: bz#1666952 + (qemu-guest-agent does not parse PCI bridge links in "build_guest_fsinfo_for_real_device" (q35)) + +* Mon Jan 28 2019 Danilo Cesar Lemes de Paula - 2.12.0-60.el8 +- kvm-ne2000-fix-possible-out-of-bound-access-in-ne2000_re.patch [bz#1636784] +- kvm-rtl8139-fix-possible-out-of-bound-access.patch [bz#1636784] +- kvm-pcnet-fix-possible-buffer-overflow.patch [bz#1636784] +- kvm-net-ignore-packet-size-greater-than-INT_MAX.patch [bz#1636784] +- kvm-net-drop-too-large-packet-early.patch [bz#1636784] +- kvm-net-hub-suppress-warnings-of-no-host-network-for-qte.patch [bz#1636784] +- kvm-virtio-net-test-accept-variable-length-argument-in-p.patch [bz#1636784] +- kvm-virtio-net-test-remove-unused-macro.patch [bz#1636784] +- kvm-virtio-net-test-add-large-tx-buffer-test.patch [bz#1636784] +- kvm-s390x-Return-specification-exception-for-unimplement.patch [bz#1668261] +- kvm-cpus-ignore-ESRCH-in-qemu_cpu_kick_thread.patch [bz#1665844] +- Resolves: bz#1636784 + (CVE-2018-17963 qemu-kvm: Qemu: net: ignore packets with large size [rhel-8]) +- Resolves: bz#1665844 + (Guest quit with error when hotunplug cpu) +- Resolves: bz#1668261 + ([RHEL8] Backport diag308 stable exception fix (qemu-kvm)) + +* Thu Jan 24 2019 Danilo Cesar Lemes de Paula - 2.12.0-59.el8 +- kvm-hw-scsi-cleanups-before-VPD-BL-emulation.patch [bz#1639957] +- kvm-hw-scsi-centralize-SG_IO-calls-into-single-function.patch [bz#1639957] +- kvm-hw-scsi-add-VPD-Block-Limits-emulation.patch [bz#1639957] +- kvm-scsi-disk-Block-Device-Characteristics-emulation-fix.patch [bz#1639957] +- kvm-scsi-generic-keep-VPD-page-list-sorted.patch [bz#1639957] +- kvm-scsi-generic-avoid-out-of-bounds-access-to-VPD-page-.patch [bz#1639957] +- kvm-scsi-generic-avoid-invalid-access-to-struct-when-emu.patch [bz#1639957] +- kvm-scsi-generic-do-not-do-VPD-emulation-for-sense-other.patch [bz#1639957] +- Resolves: bz#1639957 + ([RHEL.8] scsi host device passthrough limits IO writes - slow train) + +* Mon Jan 21 2019 Danilo Cesar Lemes de Paula - 2.12.0-58.el8 +- kvm-block-Update-flags-in-bdrv_set_read_only.patch [bz#1644996] +- kvm-block-Add-auto-read-only-option.patch [bz#1644996] +- kvm-rbd-Close-image-in-qemu_rbd_open-error-path.patch [bz#1644996] +- kvm-block-Require-auto-read-only-for-existing-fallbacks.patch [bz#1644996] +- kvm-nbd-Support-auto-read-only-option.patch [bz#1644996] +- kvm-file-posix-Support-auto-read-only-option.patch [bz#1644996] +- kvm-curl-Support-auto-read-only-option.patch [bz#1644996] +- kvm-gluster-Support-auto-read-only-option.patch [bz#1644996] +- kvm-iscsi-Support-auto-read-only-option.patch [bz#1644996] +- kvm-block-Make-auto-read-only-on-default-for-drive.patch [bz#1644996] +- kvm-qemu-iotests-Test-auto-read-only-with-drive-and-bloc.patch [bz#1644996] +- kvm-block-Fix-update-of-BDRV_O_AUTO_RDONLY-in-update_fla.patch [bz#1644996] +- kvm-qemu-img-Add-C-option-for-convert-with-copy-offloadi.patch [bz#1623082] +- kvm-iotests-Add-test-for-qemu-img-convert-C-compatibilit.patch [bz#1623082] +- Resolves: bz#1623082 + ([rhel.8.0]Target files for 'qemu-img convert' do not support thin_provisoning with iscsi/nfs backend) +- Resolves: bz#1644996 + (block-commit can't be used with -blockdev) + +* Fri Jan 11 2019 Danilo Cesar Lemes de Paula - 2.12.0-57.el8 +- kvm-qemu-kvm.spec.template-Update-files-for-tests-rpm-to.patch [bz#1601107] + +* Fri Jan 11 2019 Danilo Cesar Lemes de Paula - 2.12.0-56.el8 +- kvm-Run-iotests-as-part-of-the-build-process.patch [bz#1661026] +- kvm-Introduce-the-qemu-kvm-tests-rpm.patch [bz#1601107] +- Resolves: bz#1601107 + (qemu-kvm packaging: make running qemu-iotests more robust) +- Resolves: bz#1661026 + (Run iotests as part of build process) + +* Tue Jan 08 2019 Danilo Cesar Lemes de Paula - 2.12.0-55.el8 +- kvm-block-Don-t-inactivate-children-before-parents.patch [bz#1659395] +- kvm-iotests-Test-migration-with-blockdev.patch [bz#1659395] +- Resolves: bz#1659395 + (src qemu core dump when do migration ( block device node-name changed after change cdrom) - Slow Train) + +* Tue Jan 08 2019 Danilo Cesar Lemes de Paula - 2.12.0-54.el8 +- kvm-s390x-tcg-avoid-overflows-in-time2tod-tod2time.patch [bz#1653569] +- kvm-s390x-kvm-pass-values-instead-of-pointers-to-kvm_s39.patch [bz#1653569] +- kvm-s390x-tod-factor-out-TOD-into-separate-device.patch [bz#1653569] +- kvm-s390x-tcg-drop-tod_basetime.patch [bz#1653569] +- kvm-s390x-tcg-properly-implement-the-TOD.patch [bz#1653569] +- kvm-s390x-tcg-SET-CLOCK-COMPARATOR-can-clear-CKC-interru.patch [bz#1653569] +- kvm-s390x-tcg-implement-SET-CLOCK.patch [bz#1653569] +- kvm-s390x-tcg-rearm-the-CKC-timer-during-migration.patch [bz#1653569] +- kvm-s390x-tcg-fix-locking-problem-with-tcg_s390_tod_upda.patch [bz#1653569] +- kvm-hw-s390x-Include-the-tod-qemu-also-for-builds-with-d.patch [bz#1653569] +- kvm-s390x-tod-Properly-stop-the-KVM-TOD-while-the-guest-.patch [bz#1653569] +- kvm-hw-s390x-Fix-bad-mask-in-time2tod.patch [bz#1653569] +- kvm-migration-discard-non-migratable-RAMBlocks.patch [bz#1539285] +- kvm-vfio-pci-do-not-set-the-PCIDevice-has_rom-attribute.patch [bz#1539285] +- kvm-memory-exec-Expose-all-memory-block-related-flags.patch [bz#1539285] +- kvm-memory-exec-switch-file-ram-allocation-functions-to-.patch [bz#1539285] +- kvm-configure-add-libpmem-support.patch [bz#1539285] +- kvm-hostmem-file-add-the-pmem-option.patch [bz#1539285] +- kvm-mem-nvdimm-ensure-write-persistence-to-PMEM-in-label.patch [bz#1539285] +- kvm-migration-ram-Add-check-and-info-message-to-nvdimm-p.patch [bz#1539285] +- kvm-migration-ram-ensure-write-persistence-on-loading-al.patch [bz#1539285] +- Resolves: bz#1539285 + ([Intel 8.0 Bug] [KVM][Crystal Ridge] Lack of data persistence guarantee of QEMU writes to host PMEM) +- Resolves: bz#1653569 + (Stress guest and stop it, then do live migration, guest hit call trace on destination end) + +* Tue Jan 08 2019 Danilo Cesar Lemes de Paula - 2.12.0-53.el8 +- kvm-ui-add-qapi-parser-for-display.patch [bz#1652871] +- kvm-ui-switch-trivial-displays-to-qapi-parser.patch [bz#1652871] +- kvm-qapi-Add-rendernode-display-option-for-egl-headless.patch [bz#1652871] +- kvm-ui-Allow-specifying-rendernode-display-option-for-eg.patch [bz#1652871] +- kvm-qapi-add-query-display-options-command.patch [bz#1652871] +- Resolves: bz#1652871 + (QEMU doesn't expose rendernode option for egl-headless display type) + +* Fri Jan 04 2019 Danilo Cesar Lemes de Paula - 2.12.0-52.el8 +- kvm-Add-edk2-Requires-to-qemu-kvm.patch [bz#1654276] +- Resolves: bz#1654276 + (qemu-kvm: Should depend on the architecture-appropriate guest firmware) + +* Mon Dec 24 2018 Danilo Cesar Lemes de Paula - 2.12.0-51.el8 +- kvm-x86-host-phys-bits-limit-option.patch [bz#1598284] +- kvm-rhel-Set-host-phys-bits-limit-48-on-rhel-machine-typ.patch [bz#1598284] +- kvm-i386-do-not-migrate-MSR_SMI_COUNT-on-machine-types-2.patch [bz#1659565] +- kvm-pc-x-migrate-smi-count-to-PC_RHEL_COMPAT.patch [bz#1659565] +- kvm-slow-train-kvm-clear-out-KVM_ASYNC_PF_DELIVERY_AS_PF.patch [bz#1656829] +- Resolves: bz#1598284 + ([Intel 8.0 Alpha] physical bits should < 48 when host with 5level paging &EPT5 and qemu command with "-cpu qemu64" parameters.) +- Resolves: bz#1656829 + (8->7 migration failed: qemu-kvm: error: failed to set MSR 0x4b564d02 to 0x27fc13285) +- Resolves: bz#1659565 + (machine type: required compat flag x-migrate-smi-count=off) + +* Tue Dec 18 2018 Danilo Cesar Lemes de Paula - 2.12.0-51 +- kvm-Add-edk2-Requires-to-qemu-kvm.patch [bz#1654276] +- Resolves: bz#1654276 + (qemu-kvm: Should depend on the architecture-appropriate guest firmware) + +* Mon Dec 17 2018 Danilo Cesar Lemes de Paula - +- kvm-redhat-enable-tpmdev-passthrough.patch [bz#1654486] +- Resolves: bz#1654486 + ([RFE] enable TPM passthrough at compile time (qemu-kvm)) + +* Fri Dec 14 2018 Danilo Cesar Lemes de Paula - qemu-kvm-2.12.0-48 +- kvm-redhat-use-autopatch-instead-of-PATCHAPPLY.patch [bz#1613128] +- kvm-redhat-Removing-some-unused-build-flags-in-the-spec-.patch [bz#1613128] +- kvm-redhat-Fixing-rhev-ma-conflicts.patch [bz#1613126] +- kvm-redhat-Remove-_smp_mflags-cleanup-workaround-for-s39.patch [bz#1613128] +- kvm-redhat-Removing-dead-code-from-the-spec-file.patch [bz#1613128] +- kvm-i386-Add-stibp-flag-name.patch [bz#1639446] +- kvm-Add-functional-acceptance-tests-infrastructure.patch [bz#1655807] +- kvm-scripts-qemu.py-allow-adding-to-the-list-of-extra-ar.patch [bz#1655807] +- kvm-Acceptance-tests-add-quick-VNC-tests.patch [bz#1655807] +- kvm-scripts-qemu.py-introduce-set_console-method.patch [bz#1655807] +- kvm-Acceptance-tests-add-Linux-kernel-boot-and-console-c.patch [bz#1655807] +- kvm-Bootstrap-Python-venv-for-tests.patch [bz#1655807] +- kvm-Acceptance-tests-add-make-rule-for-running-them.patch [bz#1655807] +- Resolves: bz#1613126 + (Check and fix qemu-kvm-rhev and qemu-kvm-ma conflicts in qemu-kvm for rhel-8) +- Resolves: bz#1613128 + (Spec file clean up) +- Resolves: bz#1639446 + (Cross migration from RHEL7.5 to RHEL8 shouldn't fail with cpu flag stibp [qemu-kvm]) +- Resolves: bz#1655807 + (Backport avocado-qemu tests for QEMU 2.12) + +* Tue Dec 11 2018 Danilo Cesar Lemes de Paula - qemu-kvm-2.12.0-47 +- kvm-Disable-CONFIG_IPMI-and-CONFIG_I2C-for-ppc64.patch [bz#1640044] +- kvm-Disable-CONFIG_CAN_BUS-and-CONFIG_CAN_SJA1000.patch [bz#1640042] +- Resolves: bz#1640042 + (Disable CONFIG_CAN_BUS and CONFIG_CAN_SJA1000 config switches) +- Resolves: bz#1640044 + (Disable CONFIG_I2C and CONFIG_IPMI in default-configs/ppc64-softmmu.mak) + +* Tue Dec 11 2018 Danilo Cesar Lemes de Paula - qemu-kvm-2.12.0-46 +- kvm-qcow2-Give-the-refcount-cache-the-minimum-possible-s.patch [bz#1656507] +- kvm-docs-Document-the-new-default-sizes-of-the-qcow2-cac.patch [bz#1656507] +- kvm-qcow2-Fix-Coverity-warning-when-calculating-the-refc.patch [bz#1656507] +- kvm-include-Add-IEC-binary-prefixes-in-qemu-units.h.patch [bz#1656507] +- kvm-qcow2-Options-documentation-fixes.patch [bz#1656507] +- kvm-include-Add-a-lookup-table-of-sizes.patch [bz#1656507] +- kvm-qcow2-Make-sizes-more-humanly-readable.patch [bz#1656507] +- kvm-qcow2-Avoid-duplication-in-setting-the-refcount-cach.patch [bz#1656507] +- kvm-qcow2-Assign-the-L2-cache-relatively-to-the-image-si.patch [bz#1656507] +- kvm-qcow2-Increase-the-default-upper-limit-on-the-L2-cac.patch [bz#1656507] +- kvm-qcow2-Resize-the-cache-upon-image-resizing.patch [bz#1656507] +- kvm-qcow2-Set-the-default-cache-clean-interval-to-10-min.patch [bz#1656507] +- kvm-qcow2-Explicit-number-replaced-by-a-constant.patch [bz#1656507] +- kvm-block-backend-Set-werror-rerror-defaults-in-blk_new.patch [bz#1657637] +- kvm-qcow2-Fix-cache-clean-interval-documentation.patch [bz#1656507] +- Resolves: bz#1656507 + ([RHEL.8] qcow2 cache is too small) +- Resolves: bz#1657637 + (Wrong werror default for -device drive=) + +* Thu Dec 06 2018 Danilo Cesar Lemes de Paula - qemu-kvm-2.12.0-45 +- kvm-target-ppc-add-basic-support-for-PTCR-on-POWER9.patch [bz#1639069] +- kvm-linux-headers-Update-for-nested-KVM-HV-downstream-on.patch [bz#1639069] +- kvm-target-ppc-Add-one-reg-id-for-ptcr.patch [bz#1639069] +- kvm-ppc-spapr_caps-Add-SPAPR_CAP_NESTED_KVM_HV.patch [bz#1639069] +- kvm-Re-enable-CONFIG_HYPERV_TESTDEV.patch [bz#1651195] +- kvm-qxl-use-guest_monitor_config-for-local-renderer.patch [bz#1610163] +- kvm-Declare-cirrus-vga-as-deprecated.patch [bz#1651994] +- kvm-Do-not-build-bluetooth-support.patch [bz#1654651] +- kvm-vfio-helpers-Fix-qemu_vfio_open_pci-crash.patch [bz#1645840] +- kvm-balloon-Allow-multiple-inhibit-users.patch [bz#1650272] +- kvm-Use-inhibit-to-prevent-ballooning-without-synchr.patch [bz#1650272] +- kvm-vfio-Inhibit-ballooning-based-on-group-attachment-to.patch [bz#1650272] +- kvm-vfio-ccw-pci-Allow-devices-to-opt-in-for-ballooning.patch [bz#1650272] +- kvm-vfio-pci-Handle-subsystem-realpath-returning-NULL.patch [bz#1650272] +- kvm-vfio-pci-Fix-failure-to-close-file-descriptor-on-err.patch [bz#1650272] +- kvm-postcopy-Synchronize-usage-of-the-balloon-inhibitor.patch [bz#1650272] +- Resolves: bz#1610163 + (guest shows border blurred screen with some resolutions when qemu boot with -device qxl-vga ,and guest on rhel7.6 has no such question) +- Resolves: bz#1639069 + ([IBM 8.0 FEAT] POWER9 - Nested virtualization in RHEL8.0 KVM for ppc64le - qemu-kvm side) +- Resolves: bz#1645840 + (Qemu core dump when hotplug nvme:// drive via -blockdev) +- Resolves: bz#1650272 + (Ballooning is incompatible with vfio assigned devices, but not prevented) +- Resolves: bz#1651195 + (Re-enable hyperv-testdev device) +- Resolves: bz#1651994 + (Declare the "Cirrus VGA" device emulation of QEMU as deprecated in RHEL8) +- Resolves: bz#1654651 + (Qemu: hw: bt: keep bt/* objects from building [rhel-8.0]) + +* Tue Nov 27 2018 Danilo Cesar Lemes de Paula - qemu-kvm-2.12.0-44 +- kvm-block-Make-more-block-drivers-compile-time-configura.patch [bz#1598842 bz#1598842] +- kvm-RHEL8-Add-disable-configure-options-to-qemu-spec-fil.patch [bz#1598842] +- Resolves: bz#1598842 + (Compile out unused block drivers) + +* Mon Nov 26 2018 Danilo Cesar Lemes de Paula - qemu-kvm-2.12.0-43 +- kvm-configure-add-test-for-libudev.patch [bz#1636185] +- kvm-qga-linux-report-disk-serial-number.patch [bz#1636185] +- kvm-qga-linux-return-disk-device-in-guest-get-fsinfo.patch [bz#1636185] +- kvm-qemu-error-introduce-error-warn-_report_once.patch [bz#1625173] +- kvm-intel-iommu-start-to-use-error_report_once.patch [bz#1625173] +- kvm-intel-iommu-replace-more-vtd_err_-traces.patch [bz#1625173] +- kvm-intel_iommu-introduce-vtd_reset_caches.patch [bz#1625173] +- kvm-intel_iommu-better-handling-of-dmar-state-switch.patch [bz#1625173] +- kvm-intel_iommu-move-ce-fetching-out-when-sync-shadow.patch [bz#1625173 bz#1629616] +- kvm-intel_iommu-handle-invalid-ce-for-shadow-sync.patch [bz#1625173 bz#1629616] +- kvm-block-remove-bdrv_dirty_bitmap_make_anon.patch [bz#1518989] +- kvm-block-simplify-code-around-releasing-bitmaps.patch [bz#1518989] +- kvm-hbitmap-Add-advance-param-to-hbitmap_iter_next.patch [bz#1518989] +- kvm-test-hbitmap-Add-non-advancing-iter_next-tests.patch [bz#1518989] +- kvm-block-dirty-bitmap-Add-bdrv_dirty_iter_next_area.patch [bz#1518989] +- kvm-blockdev-backup-add-bitmap-argument.patch [bz#1518989] +- kvm-dirty-bitmap-switch-assert-fails-to-errors-in-bdrv_m.patch [bz#1518989] +- kvm-dirty-bitmap-rename-bdrv_undo_clear_dirty_bitmap.patch [bz#1518989] +- kvm-dirty-bitmap-make-it-possible-to-restore-bitmap-afte.patch [bz#1518989] +- kvm-blockdev-rename-block-dirty-bitmap-clear-transaction.patch [bz#1518989] +- kvm-qapi-add-transaction-support-for-x-block-dirty-bitma.patch [bz#1518989] +- kvm-block-dirty-bitmaps-add-user_locked-status-checker.patch [bz#1518989] +- kvm-block-dirty-bitmaps-fix-merge-permissions.patch [bz#1518989] +- kvm-block-dirty-bitmaps-allow-clear-on-disabled-bitmaps.patch [bz#1518989] +- kvm-block-dirty-bitmaps-prohibit-enable-disable-on-locke.patch [bz#1518989] +- kvm-block-backup-prohibit-backup-from-using-in-use-bitma.patch [bz#1518989] +- kvm-nbd-forbid-use-of-frozen-bitmaps.patch [bz#1518989] +- kvm-bitmap-Update-count-after-a-merge.patch [bz#1518989] +- kvm-iotests-169-drop-deprecated-autoload-parameter.patch [bz#1518989] +- kvm-block-qcow2-improve-error-message-in-qcow2_inactivat.patch [bz#1518989] +- kvm-bloc-qcow2-drop-dirty_bitmaps_loaded-state-variable.patch [bz#1518989] +- kvm-dirty-bitmaps-clean-up-bitmaps-loading-and-migration.patch [bz#1518989] +- kvm-iotests-improve-169.patch [bz#1518989] +- kvm-iotests-169-add-cases-for-source-vm-resuming.patch [bz#1518989] +- kvm-pc-dimm-turn-alignment-assert-into-check.patch [bz#1630116] +- Resolves: bz#1518989 + (RFE: QEMU Incremental live backup) +- Resolves: bz#1625173 + ([NVMe Device Assignment] Guest could not boot up with q35+iommu) +- Resolves: bz#1629616 + (boot guest with q35+vIOMMU+ device assignment, qemu terminal shows "qemu-kvm: VFIO_UNMAP_DMA: -22" when return assigned network devices from vfio driver to ixgbe in guest) +- Resolves: bz#1630116 + (pc_dimm_get_free_addr: assertion failed: (QEMU_ALIGN_UP(address_space_start, align) == address_space_start)) +- Resolves: bz#1636185 + ([RFE] Report disk device name and serial number (qemu-guest-agent on Linux)) + +* Mon Nov 05 2018 Danilo Cesar Lemes de Paula - 2.12.0-42.el8 +- kvm-luks-Allow-share-rw-on.patch [bz#1629701] +- kvm-redhat-reenable-gluster-support.patch [bz#1599340] +- kvm-redhat-bump-libusb-requirement.patch [bz#1627970] +- Resolves: bz#1599340 + (Reenable glusterfs in qemu-kvm once BZ#1567292 gets fixed) +- Resolves: bz#1627970 + (symbol lookup error: /usr/libexec/qemu-kvm: undefined symbol: libusb_set_option) +- Resolves: bz#1629701 + ("share-rw=on" does not work for luks format image - Fast Train) + +* Tue Oct 16 2018 Danilo Cesar Lemes de Paula - 2.12.0-41.el8 +- kvm-block-rbd-pull-out-qemu_rbd_convert_options.patch [bz#1635585] +- kvm-block-rbd-Attempt-to-parse-legacy-filenames.patch [bz#1635585] +- kvm-block-rbd-add-deprecation-documentation-for-filename.patch [bz#1635585] +- kvm-block-rbd-add-iotest-for-rbd-legacy-keyvalue-filenam.patch [bz#1635585] +- Resolves: bz#1635585 + (rbd json format of 7.6 is incompatible with 7.5) + +* Tue Oct 16 2018 Danilo Cesar Lemes de Paula - 2.12.0-40.el8 +- kvm-vnc-call-sasl_server_init-only-when-required.patch [bz#1609327] +- kvm-nbd-server-fix-NBD_CMD_CACHE.patch [bz#1636142] +- kvm-nbd-fix-NBD_FLAG_SEND_CACHE-value.patch [bz#1636142] +- kvm-test-bdrv-drain-bdrv_drain-works-with-cross-AioConte.patch [bz#1637976] +- kvm-block-Use-bdrv_do_drain_begin-end-in-bdrv_drain_all.patch [bz#1637976] +- kvm-block-Remove-recursive-parameter-from-bdrv_drain_inv.patch [bz#1637976] +- kvm-block-Don-t-manually-poll-in-bdrv_drain_all.patch [bz#1637976] +- kvm-tests-test-bdrv-drain-bdrv_drain_all-works-in-corout.patch [bz#1637976] +- kvm-block-Avoid-unnecessary-aio_poll-in-AIO_WAIT_WHILE.patch [bz#1637976] +- kvm-block-Really-pause-block-jobs-on-drain.patch [bz#1637976] +- kvm-block-Remove-bdrv_drain_recurse.patch [bz#1637976] +- kvm-test-bdrv-drain-Add-test-for-node-deletion.patch [bz#1637976] +- kvm-block-Drain-recursively-with-a-single-BDRV_POLL_WHIL.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-node-deletion-in-subtree-recurs.patch [bz#1637976] +- kvm-block-Don-t-poll-in-parent-drain-callbacks.patch [bz#1637976] +- kvm-test-bdrv-drain-Graph-change-through-parent-callback.patch [bz#1637976] +- kvm-block-Defer-.bdrv_drain_begin-callback-to-polling-ph.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-that-bdrv_drain_invoke-doesn-t-.patch [bz#1637976] +- kvm-block-Allow-AIO_WAIT_WHILE-with-NULL-ctx.patch [bz#1637976] +- kvm-block-Move-bdrv_drain_all_begin-out-of-coroutine-con.patch [bz#1637976] +- kvm-block-ignore_bds_parents-parameter-for-drain-functio.patch [bz#1637976] +- kvm-block-Allow-graph-changes-in-bdrv_drain_all_begin-en.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-graph-changes-in-drain_all-sect.patch [bz#1637976] +- kvm-block-Poll-after-drain-on-attaching-a-node.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-bdrv_append-to-drained-node.patch [bz#1637976] +- kvm-block-linux-aio-acquire-AioContext-before-qemu_laio_.patch [bz#1637976] +- kvm-util-async-use-qemu_aio_coroutine_enter-in-co_schedu.patch [bz#1637976] +- kvm-job-Fix-nested-aio_poll-hanging-in-job_txn_apply.patch [bz#1637976] +- kvm-job-Fix-missing-locking-due-to-mismerge.patch [bz#1637976] +- kvm-blockjob-Wake-up-BDS-when-job-becomes-idle.patch [bz#1637976] +- kvm-aio-wait-Increase-num_waiters-even-in-home-thread.patch [bz#1637976] +- kvm-test-bdrv-drain-Drain-with-block-jobs-in-an-I-O-thre.patch [bz#1637976] +- kvm-test-blockjob-Acquire-AioContext-around-job_cancel_s.patch [bz#1637976] +- kvm-job-Use-AIO_WAIT_WHILE-in-job_finish_sync.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-AIO_WAIT_WHILE-in-completion-ca.patch [bz#1637976] +- kvm-block-Add-missing-locking-in-bdrv_co_drain_bh_cb.patch [bz#1637976] +- kvm-block-backend-Add-.drained_poll-callback.patch [bz#1637976] +- kvm-block-backend-Fix-potential-double-blk_delete.patch [bz#1637976] +- kvm-block-backend-Decrease-in_flight-only-after-callback.patch [bz#1637976] +- kvm-blockjob-Lie-better-in-child_job_drained_poll.patch [bz#1637976] +- kvm-block-Remove-aio_poll-in-bdrv_drain_poll-variants.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-nested-poll-in-bdrv_drain_poll_.patch [bz#1637976] +- kvm-job-Avoid-deadlocks-in-job_completed_txn_abort.patch [bz#1637976] +- kvm-test-bdrv-drain-AIO_WAIT_WHILE-in-job-.commit-.abort.patch [bz#1637976] +- kvm-test-bdrv-drain-Fix-outdated-comments.patch [bz#1637976] +- kvm-block-Use-a-single-global-AioWait.patch [bz#1637976] +- kvm-test-bdrv-drain-Test-draining-job-source-child-and-p.patch [bz#1637976] +- kvm-qemu-img-Fix-assert-when-mapping-unaligned-raw-file.patch [bz#1639374] +- kvm-iotests-Add-test-221-to-catch-qemu-img-map-regressio.patch [bz#1639374] +- Resolves: bz#1609327 + (qemu-kvm[37046]: Could not find keytab file: /etc/qemu/krb5.tab: Unknown error 49408) +- Resolves: bz#1636142 + (qemu NBD_CMD_CACHE flaws impacting non-qemu NBD clients) +- Resolves: bz#1637976 + (Crashes and hangs with iothreads vs. block jobs) +- Resolves: bz#1639374 + (qemu-img map 'Aborted (core dumped)' when specifying a plain file) + +* Tue Oct 16 2018 Danilo Cesar Lemes de Paula - 2.12.0-39.el8 +- kvm-linux-headers-update.patch [bz#1508142] +- kvm-s390x-cpumodel-Set-up-CPU-model-for-AP-device-suppor.patch [bz#1508142] +- kvm-s390x-kvm-enable-AP-instruction-interpretation-for-g.patch [bz#1508142] +- kvm-s390x-ap-base-Adjunct-Processor-AP-object-model.patch [bz#1508142] +- kvm-s390x-vfio-ap-Introduce-VFIO-AP-device.patch [bz#1508142] +- kvm-s390-doc-detailed-specifications-for-AP-virtualizati.patch [bz#1508142] +- Resolves: bz#1508142 + ([IBM 8.0 FEAT] KVM: Guest-dedicated Crypto Adapters - qemu part) + +* Mon Oct 15 2018 Danilo Cesar Lemes de Paula - 2.12.0-38.el8 +- kvm-Revert-hw-acpi-build-build-SRAT-memory-affinity-stru.patch [bz#1609235] +- kvm-add-udev-kvm-check.patch [bz#1552663] +- kvm-aio-posix-Don-t-count-ctx-notifier-as-progress-when-.patch [bz#1623085] +- kvm-aio-Do-aio_notify_accept-only-during-blocking-aio_po.patch [bz#1623085] +- kvm-aio-posix-fix-concurrent-access-to-poll_disable_cnt.patch [bz#1632622] +- kvm-aio-posix-compute-timeout-before-polling.patch [bz#1632622] +- kvm-aio-posix-do-skip-system-call-if-ctx-notifier-pollin.patch [bz#1632622] +- kvm-intel-iommu-send-PSI-always-even-if-across-PDEs.patch [bz#1450712] +- kvm-intel-iommu-remove-IntelIOMMUNotifierNode.patch [bz#1450712] +- kvm-intel-iommu-add-iommu-lock.patch [bz#1450712] +- kvm-intel-iommu-only-do-page-walk-for-MAP-notifiers.patch [bz#1450712] +- kvm-intel-iommu-introduce-vtd_page_walk_info.patch [bz#1450712] +- kvm-intel-iommu-pass-in-address-space-when-page-walk.patch [bz#1450712] +- kvm-intel-iommu-trace-domain-id-during-page-walk.patch [bz#1450712] +- kvm-util-implement-simple-iova-tree.patch [bz#1450712] +- kvm-intel-iommu-rework-the-page-walk-logic.patch [bz#1450712] +- kvm-i386-define-the-ssbd-CPUID-feature-bit-CVE-2018-3639.patch [bz#1633928] +- Resolves: bz#1450712 + (Booting nested guest with vIOMMU, the assigned network devices can not receive packets (qemu)) +- Resolves: bz#1552663 + (81-kvm-rhel.rules is no longer part of initscripts) +- Resolves: bz#1609235 + (Win2016 guest can't recognize pc-dimm hotplugged to node 0) +- Resolves: bz#1623085 + (VM doesn't boot from HD) +- Resolves: bz#1632622 + (~40% virtio_blk disk performance drop for win2012r2 guest when comparing qemu-kvm-rhev-2.12.0-9 with qemu-kvm-rhev-2.12.0-12) +- Resolves: bz#1633928 + (CVE-2018-3639 qemu-kvm: hw: cpu: speculative store bypass [rhel-8.0]) + +* Fri Oct 12 2018 Danilo Cesar Lemes de Paula - 2.12.0-37.el8 +- kvm-block-for-jobs-do-not-clear-user_paused-until-after-.patch [bz#1635583] +- kvm-iotests-Add-failure-matching-to-common.qemu.patch [bz#1635583] +- kvm-block-iotest-to-catch-abort-on-forced-blockjob-cance.patch [bz#1635583] +- Resolves: bz#1635583 + (Quitting VM causes qemu core dump once the block mirror job paused for no enough target space) + +* Fri Oct 12 2018 Danilo Cesar Lemes de Paula - 2.12.0-36.el8 +- kvm-check-Only-test-ivshm-when-it-is-compiled-in.patch [bz#1621817] +- kvm-Disable-ivshmem.patch [bz#1621817] +- kvm-mirror-Fail-gracefully-for-source-target.patch [bz#1637963] +- kvm-commit-Add-top-node-base-node-options.patch [bz#1637970] +- kvm-qemu-iotests-Test-commit-with-top-node-base-node.patch [bz#1637970] +- Resolves: bz#1621817 + (Disable IVSHMEM in RHEL 8) +- Resolves: bz#1637963 + (Segfault on 'blockdev-mirror' with same node as source and target) +- Resolves: bz#1637970 + (allow using node-names with block-commit) + +* Thu Oct 11 2018 Danilo Cesar Lemes de Paula - 2.12.0-35.el8 +- kvm-redhat-make-the-plugins-executable.patch [bz#1638304] +- Resolves: bz#1638304 + (the driver packages lack all the library Requires) + +* Thu Oct 11 2018 Danilo Cesar Lemes de Paula - 2.12.0-34.el8 +- kvm-seccomp-allow-sched_setscheduler-with-SCHED_IDLE-pol.patch [bz#1618356] +- kvm-seccomp-use-SIGSYS-signal-instead-of-killing-the-thr.patch [bz#1618356] +- kvm-seccomp-prefer-SCMP_ACT_KILL_PROCESS-if-available.patch [bz#1618356] +- kvm-configure-require-libseccomp-2.2.0.patch [bz#1618356] +- kvm-seccomp-set-the-seccomp-filter-to-all-threads.patch [bz#1618356] +- kvm-memory-cleanup-side-effects-of-memory_region_init_fo.patch [bz#1600365] +- Resolves: bz#1600365 + (QEMU core dumped when hotplug memory exceeding host hugepages and with discard-data=yes) +- Resolves: bz#1618356 + (qemu-kvm: Qemu: seccomp: blacklist is not applied to all threads [rhel-8]) + +* Fri Oct 05 2018 Danilo Cesar Lemes de Paula - 2.12.0-33.el8 +- kvm-migration-postcopy-Clear-have_listen_thread.patch [bz#1608765] +- kvm-migration-cleanup-in-error-paths-in-loadvm.patch [bz#1608765] +- kvm-jobs-change-start-callback-to-run-callback.patch [bz#1632939] +- kvm-jobs-canonize-Error-object.patch [bz#1632939] +- kvm-jobs-add-exit-shim.patch [bz#1632939] +- kvm-block-commit-utilize-job_exit-shim.patch [bz#1632939] +- kvm-block-mirror-utilize-job_exit-shim.patch [bz#1632939] +- kvm-jobs-utilize-job_exit-shim.patch [bz#1632939] +- kvm-block-backup-make-function-variables-consistently-na.patch [bz#1632939] +- kvm-jobs-remove-ret-argument-to-job_completed-privatize-.patch [bz#1632939] +- kvm-jobs-remove-job_defer_to_main_loop.patch [bz#1632939] +- kvm-block-commit-add-block-job-creation-flags.patch [bz#1632939] +- kvm-block-mirror-add-block-job-creation-flags.patch [bz#1632939] +- kvm-block-stream-add-block-job-creation-flags.patch [bz#1632939] +- kvm-block-commit-refactor-commit-to-use-job-callbacks.patch [bz#1632939] +- kvm-block-mirror-don-t-install-backing-chain-on-abort.patch [bz#1632939] +- kvm-block-mirror-conservative-mirror_exit-refactor.patch [bz#1632939] +- kvm-block-stream-refactor-stream-to-use-job-callbacks.patch [bz#1632939] +- kvm-tests-blockjob-replace-Blockjob-with-Job.patch [bz#1632939] +- kvm-tests-test-blockjob-remove-exit-callback.patch [bz#1632939] +- kvm-tests-test-blockjob-txn-move-.exit-to-.clean.patch [bz#1632939] +- kvm-jobs-remove-.exit-callback.patch [bz#1632939] +- kvm-qapi-block-commit-expose-new-job-properties.patch [bz#1632939] +- kvm-qapi-block-mirror-expose-new-job-properties.patch [bz#1632939] +- kvm-qapi-block-stream-expose-new-job-properties.patch [bz#1632939] +- kvm-block-backup-qapi-documentation-fixup.patch [bz#1632939] +- kvm-blockdev-document-transactional-shortcomings.patch [bz#1632939] +- Resolves: bz#1608765 + (After postcopy migration, do savevm and loadvm, guest hang and call trace) +- Resolves: bz#1632939 + (qemu blockjobs other than backup do not support job-finalize or job-dismiss) + +* Fri Sep 28 2018 Danilo Cesar Lemes de Paula - 2.12.0-32.el8 +- kvm-Re-enable-disabled-Hyper-V-enlightenments.patch [bz#1625185] +- kvm-Fix-annocheck-issues.patch [bz#1624164] +- kvm-exec-check-that-alignment-is-a-power-of-two.patch [bz#1630746] +- kvm-curl-Make-sslverify-off-disable-host-as-well-as-peer.patch [bz#1575925] +- Resolves: bz#1575925 + ("SSL: no alternative certificate subject name matches target host name" error even though sslverify = off) +- Resolves: bz#1624164 + (Review annocheck distro flag failures in qemu-kvm) +- Resolves: bz#1625185 + (Re-enable disabled Hyper-V enlightenments) +- Resolves: bz#1630746 + (qemu_ram_mmap: Assertion `is_power_of_2(align)' failed) + +* Tue Sep 11 2018 Danilo Cesar Lemes de Paula - 2.12.0-31.el8 +- kvm-i386-Disable-TOPOEXT-by-default-on-cpu-host.patch [bz#1619804] +- kvm-redhat-enable-opengl-add-build-and-runtime-deps.patch [bz#1618412] +- Resolves: bz#1618412 + (Enable opengl (for intel vgpu display)) +- Resolves: bz#1619804 + (kernel panic in init_amd_cacheinfo) + +* Wed Sep 05 2018 Danilo Cesar Lemes de Paula - 2.12.0-30.el8 +- kvm-redhat-Disable-vhost-crypto.patch [bz#1625668] +- Resolves: bz#1625668 + (Decide if we should disable 'vhost-crypto' or not) + +* Wed Sep 05 2018 Danilo Cesar Lemes de Paula - 2.12.0-29.el8 +- kvm-target-i386-sev-fix-memory-leaks.patch [bz#1615717] +- kvm-i386-Fix-arch_query_cpu_model_expansion-leak.patch [bz#1615717] +- kvm-redhat-Update-build-configuration.patch [bz#1573156] +- Resolves: bz#1573156 + (Update build configure for QEMU 2.12.0) +- Resolves: bz#1615717 + (Memory leaks) + +* Tue Sep 04 2018 Danilo Cesar Lemes de Paula - 2.12.0-28.el8 +- kvm-e1000e-Do-not-auto-clear-ICR-bits-which-aren-t-set-i.patch [bz#1596024] +- kvm-e1000e-Prevent-MSI-MSI-X-storms.patch [bz#1596024] +- kvm-Drop-build_configure.sh-and-Makefile.local-files.patch [] +- kvm-Fix-subject-line-in-.gitpublish.patch [] +- Resolves: bz#1596024 + (The network link can't be detected on guest when the guest uses e1000e model type) + +* Wed Aug 29 2018 Danilo Cesar Lemes de Paula - 2.12.0-27.el8 +- kvm-Fix-libusb-1.0.22-deprecated-libusb_set_debug-with-l.patch [bz#1622656] +- Resolves: bz#1622656 + (qemu-kvm fails to build due to libusb_set_debug being deprecated) + +* Fri Aug 17 2018 Danilo Cesar Lemes de Paula - 2.12.0-26.el8 +- kvm-redhat-remove-extra-in-rhel_rhev_conflicts-macro.patch [bz#1618752] +- Resolves: bz#1618752 + (qemu-kvm can't be installed in RHEL-8 as it Conflicts with itself.) + +* Thu Aug 16 2018 Danilo Cesar Lemes de Paula - 2.12.0-25.el8 +- kvm-Migration-TLS-Fix-crash-due-to-double-cleanup.patch [bz#1594384] +- Resolves: bz#1594384 + (2.12 migration fixes) + +* Tue Aug 14 2018 Danilo Cesar Lemes de Paula - 2.12.0-24.el8 +- kvm-Add-qemu-keymap-to-qemu-kvm-common.patch [bz#1593117] +- Resolves: bz#1593117 + (add qemu-keymap utility) + +* Fri Aug 10 2018 Danilo Cesar Lemes de Paula - 2.12.0-23.el8 +- Fixing an issue with some old command in the spec file + +* Fri Aug 10 2018 Danilo Cesar Lemes de Paula - 2.12.0-22.el8 +- Fix an issue with the build_configure script. +- Resolves: bz#1425820 + (Improve QEMU packaging layout with modularization of the block layer) + + +* Fri Aug 10 2018 Danilo Cesar Lemes de Paula - 2.12.0-20.el8 +- kvm-migration-stop-compressing-page-in-migration-thread.patch [bz#1594384] +- kvm-migration-stop-compression-to-allocate-and-free-memo.patch [bz#1594384] +- kvm-migration-stop-decompression-to-allocate-and-free-me.patch [bz#1594384] +- kvm-migration-detect-compression-and-decompression-error.patch [bz#1594384] +- kvm-migration-introduce-control_save_page.patch [bz#1594384] +- kvm-migration-move-some-code-to-ram_save_host_page.patch [bz#1594384] +- kvm-migration-move-calling-control_save_page-to-the-comm.patch [bz#1594384] +- kvm-migration-move-calling-save_zero_page-to-the-common-.patch [bz#1594384] +- kvm-migration-introduce-save_normal_page.patch [bz#1594384] +- kvm-migration-remove-ram_save_compressed_page.patch [bz#1594384] +- kvm-migration-block-dirty-bitmap-fix-memory-leak-in-dirt.patch [bz#1594384] +- kvm-migration-fix-saving-normal-page-even-if-it-s-been-c.patch [bz#1594384] +- kvm-migration-update-index-field-when-delete-or-qsort-RD.patch [bz#1594384] +- kvm-migration-introduce-decompress-error-check.patch [bz#1594384] +- kvm-migration-Don-t-activate-block-devices-if-using-S.patch [bz#1594384] +- kvm-migration-not-wait-RDMA_CM_EVENT_DISCONNECTED-event-.patch [bz#1594384] +- kvm-migration-block-dirty-bitmap-fix-dirty_bitmap_load.patch [bz#1594384] +- kvm-s390x-add-RHEL-7.6-machine-type-for-ccw.patch [bz#1595718] +- kvm-s390x-cpumodel-default-enable-bpb-and-ppa15-for-z196.patch [bz#1595718] +- kvm-linux-headers-asm-s390-kvm.h-header-sync.patch [bz#1612938] +- kvm-s390x-kvm-add-etoken-facility.patch [bz#1612938] +- Resolves: bz#1594384 + (2.12 migration fixes) +- Resolves: bz#1595718 + (Add ppa15/bpb to the default cpu model for z196 and higher in the 7.6 s390-ccw-virtio machine) +- Resolves: bz#1612938 + (Add etoken support to qemu-kvm for s390x KVM guests) + +* Fri Aug 10 2018 Danilo Cesar Lemes de Paula - 2.12.0-18.el8 + Mass import from RHEL 7.6 qemu-kvm-rhev, including fixes to the following BZs: + +- kvm-AArch64-Add-virt-rhel7.6-machine-type.patch [bz#1558723] +- kvm-cpus-Fix-event-order-on-resume-of-stopped-guest.patch [bz#1566153] +- kvm-qemu-img-Check-post-truncation-size.patch [bz#1523065] +- kvm-vga-catch-depth-0.patch [bz#1575541] +- kvm-Fix-x-hv-max-vps-compat-value-for-7.4-machine-type.patch [bz#1583959] +- kvm-ccid-card-passthru-fix-regression-in-realize.patch [bz#1584984] +- kvm-Use-4-MB-vram-for-cirrus.patch [bz#1542080] +- kvm-spapr_pci-Remove-unhelpful-pagesize-warning.patch [bz#1505664] +- kvm-rpm-Add-nvme-VFIO-driver-to-rw-whitelist.patch [bz#1416180] +- kvm-qobject-Use-qobject_to-instead-of-type-cast.patch [bz#1557995] +- kvm-qobject-Ensure-base-is-at-offset-0.patch [bz#1557995] +- kvm-qobject-use-a-QObjectBase_-struct.patch [bz#1557995] +- kvm-qobject-Replace-qobject_incref-QINCREF-qobject_decre.patch [bz#1557995] +- kvm-qobject-Modify-qobject_ref-to-return-obj.patch [bz#1557995] +- kvm-rbd-Drop-deprecated-drive-parameter-filename.patch [bz#1557995] +- kvm-iscsi-Drop-deprecated-drive-parameter-filename.patch [bz#1557995] +- kvm-block-Add-block-specific-QDict-header.patch [bz#1557995] +- kvm-qobject-Move-block-specific-qdict-code-to-block-qdic.patch [bz#1557995] +- kvm-block-Fix-blockdev-for-certain-non-string-scalars.patch [bz#1557995] +- kvm-block-Fix-drive-for-certain-non-string-scalars.patch [bz#1557995] +- kvm-block-Clean-up-a-misuse-of-qobject_to-in-.bdrv_co_cr.patch [bz#1557995] +- kvm-block-Factor-out-qobject_input_visitor_new_flat_conf.patch [bz#1557995] +- kvm-block-Make-remaining-uses-of-qobject-input-visitor-m.patch [bz#1557995] +- kvm-block-qdict-Simplify-qdict_flatten_qdict.patch [bz#1557995] +- kvm-block-qdict-Tweak-qdict_flatten_qdict-qdict_flatten_.patch [bz#1557995] +- kvm-block-qdict-Clean-up-qdict_crumple-a-bit.patch [bz#1557995] +- kvm-block-qdict-Simplify-qdict_is_list-some.patch [bz#1557995] +- kvm-check-block-qdict-Rename-qdict_flatten-s-variables-f.patch [bz#1557995] +- kvm-check-block-qdict-Cover-flattening-of-empty-lists-an.patch [bz#1557995] +- kvm-block-Fix-blockdev-blockdev-add-for-empty-objects-an.patch [bz#1557995] +- kvm-rbd-New-parameter-auth-client-required.patch [bz#1557995] +- kvm-rbd-New-parameter-key-secret.patch [bz#1557995] +- kvm-block-mirror-honor-ratelimit-again.patch [bz#1572856] +- kvm-block-mirror-Make-cancel-always-cancel-pre-READY.patch [bz#1572856] +- kvm-iotests-Add-test-for-cancelling-a-mirror-job.patch [bz#1572856] +- kvm-iotests-Split-214-off-of-122.patch [bz#1518738] +- kvm-block-Add-COR-filter-driver.patch [bz#1518738] +- kvm-block-BLK_PERM_WRITE-includes-._UNCHANGED.patch [bz#1518738] +- kvm-block-Add-BDRV_REQ_WRITE_UNCHANGED-flag.patch [bz#1518738] +- kvm-block-Set-BDRV_REQ_WRITE_UNCHANGED-for-COR-writes.patch [bz#1518738] +- kvm-block-quorum-Support-BDRV_REQ_WRITE_UNCHANGED.patch [bz#1518738] +- kvm-block-Support-BDRV_REQ_WRITE_UNCHANGED-in-filters.patch [bz#1518738] +- kvm-iotests-Clean-up-wrap-image-in-197.patch [bz#1518738] +- kvm-iotests-Copy-197-for-COR-filter-driver.patch [bz#1518738] +- kvm-iotests-Add-test-for-COR-across-nodes.patch [bz#1518738] +- kvm-qemu-io-Use-purely-string-blockdev-options.patch [bz#1576598] +- kvm-qemu-img-Use-only-string-options-in-img_open_opts.patch [bz#1576598] +- kvm-iotests-Add-test-for-U-force-share-conflicts.patch [bz#1576598] +- kvm-qemu-io-Drop-command-functions-return-values.patch [bz#1519617] +- kvm-qemu-io-Let-command-functions-return-error-code.patch [bz#1519617] +- kvm-qemu-io-Exit-with-error-when-a-command-failed.patch [bz#1519617] +- kvm-iotests.py-Add-qemu_io_silent.patch [bz#1519617] +- kvm-iotests-Let-216-make-use-of-qemu-io-s-exit-code.patch [bz#1519617] +- kvm-qcow2-Repair-OFLAG_COPIED-when-fixing-leaks.patch [bz#1527085] +- kvm-iotests-Repairing-error-during-snapshot-deletion.patch [bz#1527085] +- kvm-block-Make-bdrv_is_writable-public.patch [bz#1588039] +- kvm-qcow2-Do-not-mark-inactive-images-corrupt.patch [bz#1588039] +- kvm-iotests-Add-case-for-a-corrupted-inactive-image.patch [bz#1588039] +- kvm-main-loop-drop-spin_counter.patch [bz#1168213] +- kvm-target-ppc-Factor-out-the-parsing-in-kvmppc_get_cpu_.patch [bz#1560847] +- kvm-target-ppc-Don-t-require-private-l1d-cache-on-POWER8.patch [bz#1560847] +- kvm-ppc-spapr_caps-Don-t-disable-cap_cfpc-on-POWER8-by-d.patch [bz#1560847] +- kvm-qxl-fix-local-renderer-crash.patch [bz#1567733] +- kvm-qemu-img-Amendment-support-implies-create_opts.patch [bz#1537956] +- kvm-block-Add-Error-parameter-to-bdrv_amend_options.patch [bz#1537956] +- kvm-qemu-option-Pull-out-Supported-options-print.patch [bz#1537956] +- kvm-qemu-img-Add-print_amend_option_help.patch [bz#1537956] +- kvm-qemu-img-Recognize-no-creation-support-in-o-help.patch [bz#1537956] +- kvm-iotests-Test-help-option-for-unsupporting-formats.patch [bz#1537956] +- kvm-iotests-Rework-113.patch [bz#1537956] +- kvm-qemu-img-Resolve-relative-backing-paths-in-rebase.patch [bz#1569835] +- kvm-iotests-Add-test-for-rebasing-with-relative-paths.patch [bz#1569835] +- kvm-qemu-img-Special-post-backing-convert-handling.patch [bz#1527898] +- kvm-iotests-Test-post-backing-convert-target-behavior.patch [bz#1527898] +- kvm-migration-calculate-expected_downtime-with-ram_bytes.patch [bz#1564576] +- kvm-sheepdog-Fix-sd_co_create_opts-memory-leaks.patch [bz#1513543] +- kvm-qemu-iotests-reduce-chance-of-races-in-185.patch [bz#1513543] +- kvm-blockjob-do-not-cancel-timer-in-resume.patch [bz#1513543] +- kvm-nfs-Fix-error-path-in-nfs_options_qdict_to_qapi.patch [bz#1513543] +- kvm-nfs-Remove-processed-options-from-QDict.patch [bz#1513543] +- kvm-blockjob-drop-block_job_pause-resume_all.patch [bz#1513543] +- kvm-blockjob-expose-error-string-via-query.patch [bz#1513543] +- kvm-blockjob-Fix-assertion-in-block_job_finalize.patch [bz#1513543] +- kvm-blockjob-Wrappers-for-progress-counter-access.patch [bz#1513543] +- kvm-blockjob-Move-RateLimit-to-BlockJob.patch [bz#1513543] +- kvm-blockjob-Implement-block_job_set_speed-centrally.patch [bz#1513543] +- kvm-blockjob-Introduce-block_job_ratelimit_get_delay.patch [bz#1513543] +- kvm-blockjob-Add-block_job_driver.patch [bz#1513543] +- kvm-blockjob-Update-block-job-pause-resume-documentation.patch [bz#1513543] +- kvm-blockjob-Improve-BlockJobInfo.offset-len-documentati.patch [bz#1513543] +- kvm-job-Create-Job-JobDriver-and-job_create.patch [bz#1513543] +- kvm-job-Rename-BlockJobType-into-JobType.patch [bz#1513543] +- kvm-job-Add-JobDriver.job_type.patch [bz#1513543] +- kvm-job-Add-job_delete.patch [bz#1513543] +- kvm-job-Maintain-a-list-of-all-jobs.patch [bz#1513543] +- kvm-job-Move-state-transitions-to-Job.patch [bz#1513543] +- kvm-job-Add-reference-counting.patch [bz#1513543] +- kvm-job-Move-cancelled-to-Job.patch [bz#1513543] +- kvm-job-Add-Job.aio_context.patch [bz#1513543] +- kvm-job-Move-defer_to_main_loop-to-Job.patch [bz#1513543] +- kvm-job-Move-coroutine-and-related-code-to-Job.patch [bz#1513543] +- kvm-job-Add-job_sleep_ns.patch [bz#1513543] +- kvm-job-Move-pause-resume-functions-to-Job.patch [bz#1513543] +- kvm-job-Replace-BlockJob.completed-with-job_is_completed.patch [bz#1513543] +- kvm-job-Move-BlockJobCreateFlags-to-Job.patch [bz#1513543] +- kvm-blockjob-Split-block_job_event_pending.patch [bz#1513543] +- kvm-job-Add-job_event_.patch [bz#1513543] +- kvm-job-Move-single-job-finalisation-to-Job.patch [bz#1513543] +- kvm-job-Convert-block_job_cancel_async-to-Job.patch [bz#1513543] +- kvm-job-Add-job_drain.patch [bz#1513543] +- kvm-job-Move-.complete-callback-to-Job.patch [bz#1513543] +- kvm-job-Move-job_finish_sync-to-Job.patch [bz#1513543] +- kvm-job-Switch-transactions-to-JobTxn.patch [bz#1513543] +- kvm-job-Move-transactions-to-Job.patch [bz#1513543] +- kvm-job-Move-completion-and-cancellation-to-Job.patch [bz#1513543] +- kvm-block-Cancel-job-in-bdrv_close_all-callers.patch [bz#1513543] +- kvm-job-Add-job_yield.patch [bz#1513543] +- kvm-job-Add-job_dismiss.patch [bz#1513543] +- kvm-job-Add-job_is_ready.patch [bz#1513543] +- kvm-job-Add-job_transition_to_ready.patch [bz#1513543] +- kvm-job-Move-progress-fields-to-Job.patch [bz#1513543] +- kvm-job-Introduce-qapi-job.json.patch [bz#1513543] +- kvm-job-Add-JOB_STATUS_CHANGE-QMP-event.patch [bz#1513543] +- kvm-job-Add-lifecycle-QMP-commands.patch [bz#1513543] +- kvm-job-Add-query-jobs-QMP-command.patch [bz#1513543] +- kvm-blockjob-Remove-BlockJob.driver.patch [bz#1513543] +- kvm-iotests-Move-qmp_to_opts-to-VM.patch [bz#1513543] +- kvm-qemu-iotests-Test-job-with-block-jobs.patch [bz#1513543] +- kvm-vdi-Fix-vdi_co_do_create-return-value.patch [bz#1513543] +- kvm-vhdx-Fix-vhdx_co_create-return-value.patch [bz#1513543] +- kvm-job-Add-error-message-for-failing-jobs.patch [bz#1513543] +- kvm-block-create-Make-x-blockdev-create-a-job.patch [bz#1513543] +- kvm-qemu-iotests-Add-VM.get_qmp_events_filtered.patch [bz#1513543] +- kvm-qemu-iotests-Add-VM.qmp_log.patch [bz#1513543] +- kvm-qemu-iotests-Add-iotests.img_info_log.patch [bz#1513543] +- kvm-qemu-iotests-Add-VM.run_job.patch [bz#1513543] +- kvm-qemu-iotests-iotests.py-helper-for-non-file-protocol.patch [bz#1513543] +- kvm-qemu-iotests-Rewrite-206-for-blockdev-create-job.patch [bz#1513543] +- kvm-qemu-iotests-Rewrite-207-for-blockdev-create-job.patch [bz#1513543] +- kvm-qemu-iotests-Rewrite-210-for-blockdev-create-job.patch [bz#1513543] +- kvm-qemu-iotests-Rewrite-211-for-blockdev-create-job.patch [bz#1513543] +- kvm-qemu-iotests-Rewrite-212-for-blockdev-create-job.patch [bz#1513543] +- kvm-qemu-iotests-Rewrite-213-for-blockdev-create-job.patch [bz#1513543] +- kvm-block-create-Mark-blockdev-create-stable.patch [bz#1513543] +- kvm-jobs-fix-stale-wording.patch [bz#1513543] +- kvm-jobs-fix-verb-references-in-docs.patch [bz#1513543] +- kvm-iotests-Fix-219-s-timing.patch [bz#1513543] +- kvm-iotests-improve-pause_job.patch [bz#1513543] +- kvm-rpm-Whitelist-copy-on-read-block-driver.patch [bz#1518738] +- kvm-rpm-add-throttle-driver-to-rw-whitelist.patch [bz#1591076] +- kvm-usb-host-skip-open-on-pending-postload-bh.patch [bz#1572851] +- kvm-i386-Define-the-Virt-SSBD-MSR-and-handling-of-it-CVE.patch [bz#1574216] +- kvm-i386-define-the-AMD-virt-ssbd-CPUID-feature-bit-CVE-.patch [bz#1574216] +- kvm-block-file-posix-Pass-FD-to-locking-helpers.patch [bz#1519144] +- kvm-block-file-posix-File-locking-during-creation.patch [bz#1519144] +- kvm-iotests-Add-creation-test-to-153.patch [bz#1519144] +- kvm-vhost-user-add-Net-prefix-to-internal-state-structur.patch [bz#1526645] +- kvm-virtio-support-setting-memory-region-based-host-noti.patch [bz#1526645] +- kvm-vhost-user-support-receiving-file-descriptors-in-sla.patch [bz#1526645] +- kvm-osdep-add-wait.h-compat-macros.patch [bz#1526645] +- kvm-vhost-user-bridge-support-host-notifier.patch [bz#1526645] +- kvm-vhost-allow-backends-to-filter-memory-sections.patch [bz#1526645] +- kvm-vhost-user-allow-slave-to-send-fds-via-slave-channel.patch [bz#1526645] +- kvm-vhost-user-introduce-shared-vhost-user-state.patch [bz#1526645] +- kvm-vhost-user-support-registering-external-host-notifie.patch [bz#1526645] +- kvm-libvhost-user-support-host-notifier.patch [bz#1526645] +- kvm-block-Introduce-API-for-copy-offloading.patch [bz#1482537] +- kvm-raw-Check-byte-range-uniformly.patch [bz#1482537] +- kvm-raw-Implement-copy-offloading.patch [bz#1482537] +- kvm-qcow2-Implement-copy-offloading.patch [bz#1482537] +- kvm-file-posix-Implement-bdrv_co_copy_range.patch [bz#1482537] +- kvm-iscsi-Query-and-save-device-designator-when-opening.patch [bz#1482537] +- kvm-iscsi-Create-and-use-iscsi_co_wait_for_task.patch [bz#1482537] +- kvm-iscsi-Implement-copy-offloading.patch [bz#1482537] +- kvm-block-backend-Add-blk_co_copy_range.patch [bz#1482537] +- kvm-qemu-img-Convert-with-copy-offloading.patch [bz#1482537] +- kvm-qcow2-Fix-src_offset-in-copy-offloading.patch [bz#1482537] +- kvm-iscsi-Don-t-blindly-use-designator-length-in-respons.patch [bz#1482537] +- kvm-file-posix-Fix-EINTR-handling.patch [bz#1482537] +- kvm-usb-storage-Add-rerror-werror-properties.patch [bz#1595180] +- kvm-numa-clarify-error-message-when-node-index-is-out-of.patch [bz#1578381] +- kvm-qemu-iotests-Update-026.out.nocache-reference-output.patch [bz#1528541] +- kvm-qcow2-Free-allocated-clusters-on-write-error.patch [bz#1528541] +- kvm-qemu-iotests-Test-qcow2-not-leaking-clusters-on-writ.patch [bz#1528541] +- kvm-qemu-options-Add-missing-newline-to-accel-help-text.patch [bz#1586313] +- kvm-xhci-fix-guest-triggerable-assert.patch [bz#1594135] +- kvm-virtio-gpu-tweak-scanout-disable.patch [bz#1589634] +- kvm-virtio-gpu-update-old-resource-too.patch [bz#1589634] +- kvm-virtio-gpu-disable-scanout-when-backing-resource-is-.patch [bz#1589634] +- kvm-block-Don-t-silently-truncate-node-names.patch [bz#1549654] +- kvm-pr-helper-fix-socket-path-default-in-help.patch [bz#1533158] +- kvm-pr-helper-fix-assertion-failure-on-failed-multipath-.patch [bz#1533158] +- kvm-pr-manager-helper-avoid-SIGSEGV-when-writing-to-the-.patch [bz#1533158] +- kvm-pr-manager-put-stubs-in-.c-file.patch [bz#1533158] +- kvm-pr-manager-add-query-pr-managers-QMP-command.patch [bz#1533158] +- kvm-pr-manager-helper-report-event-on-connection-disconn.patch [bz#1533158] +- kvm-pr-helper-avoid-error-on-PR-IN-command-with-zero-req.patch [bz#1533158] +- kvm-pr-helper-Rework-socket-path-handling.patch [bz#1533158] +- kvm-pr-manager-helper-fix-memory-leak-on-event.patch [bz#1533158] +- kvm-object-fix-OBJ_PROP_LINK_UNREF_ON_RELEASE-ambivalenc.patch [bz#1556678] +- kvm-usb-hcd-xhci-test-add-a-test-for-ccid-hotplug.patch [bz#1556678] +- kvm-Revert-usb-release-the-created-buses.patch [bz#1556678] +- kvm-file-posix-Fix-creation-locking.patch [bz#1599335] +- kvm-file-posix-Unlock-FD-after-creation.patch [bz#1599335] +- kvm-ahci-trim-signatures-on-raise-lower.patch [bz#1584914] +- kvm-ahci-fix-PxCI-register-race.patch [bz#1584914] +- kvm-ahci-don-t-schedule-unnecessary-BH.patch [bz#1584914] +- kvm-qcow2-Fix-qcow2_truncate-error-return-value.patch [bz#1595173] +- kvm-block-Convert-.bdrv_truncate-callback-to-coroutine_f.patch [bz#1595173] +- kvm-qcow2-Remove-coroutine-trampoline-for-preallocate_co.patch [bz#1595173] +- kvm-block-Move-bdrv_truncate-implementation-to-io.c.patch [bz#1595173] +- kvm-block-Use-tracked-request-for-truncate.patch [bz#1595173] +- kvm-file-posix-Make-.bdrv_co_truncate-asynchronous.patch [bz#1595173] +- kvm-block-Fix-copy-on-read-crash-with-partial-final-clus.patch [bz#1590640] +- kvm-block-fix-QEMU-crash-with-scsi-hd-and-drive_del.patch [bz#1599515] +- kvm-virtio-rng-process-pending-requests-on-DRIVER_OK.patch [bz#1576743] +- kvm-file-posix-specify-expected-filetypes.patch [bz#1525829] +- kvm-iotests-add-test-226-for-file-driver-types.patch [bz#1525829] +- kvm-block-dirty-bitmap-add-lock-to-bdrv_enable-disable_d.patch [bz#1207657] +- kvm-qapi-add-x-block-dirty-bitmap-enable-disable.patch [bz#1207657] +- kvm-qmp-transaction-support-for-x-block-dirty-bitmap-ena.patch [bz#1207657] +- kvm-qapi-add-x-block-dirty-bitmap-merge.patch [bz#1207657] +- kvm-qapi-add-disabled-parameter-to-block-dirty-bitmap-ad.patch [bz#1207657] +- kvm-block-dirty-bitmap-add-bdrv_enable_dirty_bitmap_lock.patch [bz#1207657] +- kvm-dirty-bitmap-fix-double-lock-on-bitmap-enabling.patch [bz#1207657] +- kvm-block-qcow2-bitmap-fix-free_bitmap_clusters.patch [bz#1207657] +- kvm-qcow2-add-overlap-check-for-bitmap-directory.patch [bz#1207657] +- kvm-blockdev-enable-non-root-nodes-for-backup-source.patch [bz#1207657] +- kvm-iotests-add-222-to-test-basic-fleecing.patch [bz#1207657] +- kvm-qcow2-Remove-dead-check-on-ret.patch [bz#1207657] +- kvm-block-Move-request-tracking-to-children-in-copy-offl.patch [bz#1207657] +- kvm-block-Fix-parameter-checking-in-bdrv_co_copy_range_i.patch [bz#1207657] +- kvm-block-Honour-BDRV_REQ_NO_SERIALISING-in-copy-range.patch [bz#1207657] +- kvm-backup-Use-copy-offloading.patch [bz#1207657] +- kvm-block-backup-disable-copy-offloading-for-backup.patch [bz#1207657] +- kvm-iotests-222-Don-t-run-with-luks.patch [bz#1207657] +- kvm-block-io-fix-copy_range.patch [bz#1207657] +- kvm-block-split-flags-in-copy_range.patch [bz#1207657] +- kvm-block-add-BDRV_REQ_SERIALISING-flag.patch [bz#1207657] +- kvm-block-backup-fix-fleecing-scheme-use-serialized-writ.patch [bz#1207657] +- kvm-nbd-server-Reject-0-length-block-status-request.patch [bz#1207657] +- kvm-nbd-server-fix-trace.patch [bz#1207657] +- kvm-nbd-server-refactor-NBDExportMetaContexts.patch [bz#1207657] +- kvm-nbd-server-add-nbd_meta_empty_or_pattern-helper.patch [bz#1207657] +- kvm-nbd-server-implement-dirty-bitmap-export.patch [bz#1207657] +- kvm-qapi-new-qmp-command-nbd-server-add-bitmap.patch [bz#1207657] +- kvm-docs-interop-add-nbd.txt.patch [bz#1207657] +- kvm-nbd-server-introduce-NBD_CMD_CACHE.patch [bz#1207657] +- kvm-nbd-server-Silence-gcc-false-positive.patch [bz#1207657] +- kvm-nbd-server-Fix-dirty-bitmap-logic-regression.patch [bz#1207657] +- kvm-nbd-server-fix-nbd_co_send_block_status.patch [bz#1207657] +- kvm-nbd-client-Add-x-dirty-bitmap-to-query-bitmap-from-s.patch [bz#1207657] +- kvm-iotests-New-test-223-for-exporting-dirty-bitmap-over.patch [bz#1207657] +- kvm-hw-char-serial-Only-retry-if-qemu_chr_fe_write-retur.patch [bz#1592817] +- kvm-hw-char-serial-retry-write-if-EAGAIN.patch [bz#1592817] +- kvm-throttle-groups-fix-hang-when-group-member-leaves.patch [bz#1535914] +- kvm-Disable-aarch64-devices-reappeared-after-2.12-rebase.patch [bz#1586357] +- kvm-Disable-split-irq-device.patch [bz#1586357] +- kvm-Disable-AT24Cx-i2c-eeprom.patch [bz#1586357] +- kvm-Disable-CAN-bus-devices.patch [bz#1586357] +- kvm-Disable-new-superio-devices.patch [bz#1586357] +- kvm-Disable-new-pvrdma-device.patch [bz#1586357] +- kvm-qdev-add-HotplugHandler-post_plug-callback.patch [bz#1607891] +- kvm-virtio-scsi-fix-hotplug-reset-vs-event-race.patch [bz#1607891] +- kvm-e1000-Fix-tso_props-compat-for-82540em.patch [bz#1608778] +- kvm-slirp-correct-size-computation-while-concatenating-m.patch [bz#1586255] +- kvm-s390x-sclp-fix-maxram-calculation.patch [bz#1595740] +- kvm-redhat-Make-gitpublish-profile-the-default-one.patch [bz#1425820] +- Resolves: bz#1168213 + (main-loop: WARNING: I/O thread spun for 1000 iterations while doing stream block device.) +- Resolves: bz#1207657 + (RFE: QEMU Incremental live backup - push and pull modes) +- Resolves: bz#1416180 + (QEMU VFIO based block driver for NVMe devices) +- Resolves: bz#1425820 + (Improve QEMU packaging layout with modularization of the block layer) +- Resolves: bz#1482537 + ([RFE] qemu-img copy-offloading (convert command)) +- Resolves: bz#1505664 + ("qemu-kvm: System page size 0x1000000 is not enabled in page_size_mask (0x11000). Performance may be slow" show up while using hugepage as guest's memory) +- Resolves: bz#1513543 + ([RFE] Add block job to create format on a storage device) +- Resolves: bz#1518738 + (Add 'copy-on-read' filter driver for use with blockdev-add) +- Resolves: bz#1519144 + (qemu-img: image locking doesn't cover image creation) +- Resolves: bz#1519617 + (The exit code should be non-zero when qemu-io reports an error) +- Resolves: bz#1523065 + ("qemu-img resize" should fail to decrease the size of logical partition/lvm/iSCSI image with raw format) +- Resolves: bz#1525829 + (can not boot up a scsi-block passthrough disk via -blockdev with error "cannot get SG_IO version number: Operation not supported. Is this a SCSI device?") +- Resolves: bz#1526645 + ([Intel 7.6 FEAT] vHost Data Plane Acceleration (vDPA) - vhost user client - qemu-kvm-rhev) +- Resolves: bz#1527085 + (The copied flag should be updated during '-r leaks') +- Resolves: bz#1527898 + ([RFE] qemu-img should leave cluster unallocated if it's read as zero throughout the backing chain) +- Resolves: bz#1528541 + (qemu-img check reports tons of leaked clusters after re-start nfs service to resume writing data in guest) +- Resolves: bz#1533158 + (QEMU support for libvirtd restarting qemu-pr-helper) +- Resolves: bz#1535914 + (Disable io throttling for one member disk of a group during io will induce the other one hang with io) +- Resolves: bz#1537956 + (RFE: qemu-img amend should list the true supported options) +- Resolves: bz#1542080 + (Qemu core dump at cirrus_invalidate_region) +- Resolves: bz#1549654 + (Reject node-names which would be truncated by the block layer commands) +- Resolves: bz#1556678 + (Hot plug usb-ccid for the 2nd time with the same ID as the 1st time failed) +- Resolves: bz#1557995 + (QAPI schema for RBD storage misses the 'password-secret' option) +- Resolves: bz#1558723 + (Create RHEL-7.6 QEMU machine type for AArch64) +- Resolves: bz#1560847 + ([Power8][FW b0320a_1812.861][rhel7.5rc2 3.10.0-861.el7.ppc64le][qemu-kvm-{ma,rhev}-2.10.0-21.el7_5.1.ppc64le] KVM guest does not default to ori type flush even with pseries-rhel7.5.0-sxxm) +- Resolves: bz#1564576 + (Pegas 1.1 - Require to backport qemu-kvm patch that fixes expected_downtime calculation during migration) +- Resolves: bz#1566153 + (IOERROR pause code lost after resuming a VM while I/O error is still present) +- Resolves: bz#1567733 + (qemu abort when migrate during guest reboot) +- Resolves: bz#1569835 + (qemu-img get wrong backing file path after rebasing image with relative path) +- Resolves: bz#1572851 + (Core dumped after migration when with usb-host) +- Resolves: bz#1572856 + ('block-job-cancel' can not cancel a "drive-mirror" job) +- Resolves: bz#1574216 + (CVE-2018-3639 qemu-kvm-rhev: hw: cpu: speculative store bypass [rhel-7.6]) +- Resolves: bz#1575541 + (qemu core dump while installing win10 guest) +- Resolves: bz#1576598 + (Segfault in qemu-io and qemu-img with -U --image-opts force-share=off) +- Resolves: bz#1576743 + (virtio-rng hangs when running on recent (2.x) QEMU versions) +- Resolves: bz#1578381 + (Error message need update when specify numa distance with node index >=128) +- Resolves: bz#1583959 + (Incorrect vcpu count limit for 7.4 machine types for windows guests) +- Resolves: bz#1584914 + (SATA emulator lags and hangs) +- Resolves: bz#1584984 + (Vm starts failed with 'passthrough' smartcard) +- Resolves: bz#1586255 + (CVE-2018-11806 qemu-kvm-rhev: QEMU: slirp: heap buffer overflow while reassembling fragmented datagrams [rhel-7.6]) +- Resolves: bz#1586313 + (-smp option is not easily found in the output of qemu help) +- Resolves: bz#1586357 + (Disable new devices in 2.12) +- Resolves: bz#1588039 + (Possible assertion failure in qemu when a corrupted image is used during an incoming migration) +- Resolves: bz#1589634 + (Migration failed when rebooting guest with multiple virtio videos) +- Resolves: bz#1590640 + (qemu-kvm: block/io.c:1098: bdrv_co_do_copy_on_readv: Assertion `skip_bytes < pnum' failed.) +- Resolves: bz#1591076 + (The driver of 'throttle' is not whitelisted) +- Resolves: bz#1592817 + (Retrying on serial_xmit if the pipe is broken may compromise the Guest) +- Resolves: bz#1594135 + (system_reset many times linux guests cause qemu process Aborted) +- Resolves: bz#1595173 + (blockdev-create is blocking) +- Resolves: bz#1595180 + (Can't set rerror/werror with usb-storage) +- Resolves: bz#1595740 + (RHEL-Alt-7.6 - qemu has error during migration of larger guests) +- Resolves: bz#1599335 + (Image creation locking is too tight and is not properly released) +- Resolves: bz#1599515 + (qemu core-dump with aio_read via hmp (util/qemu-thread-posix.c:64: qemu_mutex_lock_impl: Assertion `mutex->initialized' failed)) +- Resolves: bz#1607891 + (Hotplug events are sometimes lost with virtio-scsi + iothread) +- Resolves: bz#1608778 + (qemu/migration: migrate failed from RHEL.7.6 to RHEL.7.5 with e1000-82540em) + +* Mon Aug 06 2018 Danilo Cesar Lemes de Paula - 2.12.0-17.el8 +- kvm-linux-headers-Update-to-include-KVM_CAP_S390_HPAGE_1.patch [bz#1610906] +- kvm-s390x-Enable-KVM-huge-page-backing-support.patch [bz#1610906] +- kvm-redhat-s390x-add-hpage-1-to-kvm.conf.patch [bz#1610906] +- Resolves: bz#1610906 + ([IBM 8.0 FEAT] KVM: Huge Pages - libhugetlbfs Enablement - qemu-kvm part) + +* Tue Jul 31 2018 Danilo Cesar Lemes de Paula - 2.12.0-16.el8 +- kvm-spapr-Correct-inverted-test-in-spapr_pc_dimm_node.patch [bz#1601671] +- kvm-osdep-powerpc64-align-memory-to-allow-2MB-radix-THP-.patch [bz#1601317] +- kvm-RHEL-8.0-Add-pseries-rhel7.6.0-sxxm-machine-type.patch [bz#1595501] +- kvm-i386-Helpers-to-encode-cache-information-consistentl.patch [bz#1597739] +- kvm-i386-Add-cache-information-in-X86CPUDefinition.patch [bz#1597739] +- kvm-i386-Initialize-cache-information-for-EPYC-family-pr.patch [bz#1597739] +- kvm-i386-Add-new-property-to-control-cache-info.patch [bz#1597739] +- kvm-i386-Clean-up-cache-CPUID-code.patch [bz#1597739] +- kvm-i386-Populate-AMD-Processor-Cache-Information-for-cp.patch [bz#1597739] +- kvm-i386-Add-support-for-CPUID_8000_001E-for-AMD.patch [bz#1597739] +- kvm-i386-Fix-up-the-Node-id-for-CPUID_8000_001E.patch [bz#1597739] +- kvm-i386-Enable-TOPOEXT-feature-on-AMD-EPYC-CPU.patch [bz#1597739] +- kvm-i386-Remove-generic-SMT-thread-check.patch [bz#1597739] +- kvm-i386-Allow-TOPOEXT-to-be-enabled-on-older-kernels.patch [bz#1597739] +- Resolves: bz#1595501 + (Create pseries-rhel7.6.0-sxxm machine type) +- Resolves: bz#1597739 + (AMD EPYC/Zen SMT support for KVM / QEMU guest (qemu-kvm)) +- Resolves: bz#1601317 + (RHEL8.0 - qemu patch to align memory to allow 2MB THP) +- Resolves: bz#1601671 + (After rebooting guest,all the hot plug memory will be assigned to the 1st numa node.) + +* Tue Jul 24 2018 Danilo Cesar Lemes de Paula - 2.12.0-15.el8 +- kvm-spapr-Add-ibm-max-associativity-domains-property.patch [bz#1599593] +- kvm-Revert-spapr-Don-t-allow-memory-hotplug-to-memory-le.patch [bz#1599593] +- kvm-simpletrace-Convert-name-from-mapping-record-to-str.patch [bz#1594969] +- kvm-tests-fix-TLS-handshake-failure-with-TLS-1.3.patch [bz#1602403] +- Resolves: bz#1594969 + (simpletrace.py fails when running with Python 3) +- Resolves: bz#1599593 + (User can't hotplug memory to less memory numa node on rhel8) +- Resolves: bz#1602403 + (test-crypto-tlssession unit test fails with assertions) + +* Mon Jul 09 2018 Danilo Cesar Lemes de Paula - 2.12.0-14.el8 +- kvm-vfio-pci-Default-display-option-to-off.patch [bz#1590511] +- kvm-python-futurize-f-libfuturize.fixes.fix_print_with_i.patch [bz#1571533] +- kvm-python-futurize-f-lib2to3.fixes.fix_except.patch [bz#1571533] +- kvm-Revert-Defining-a-shebang-for-python-scripts.patch [bz#1571533] +- kvm-spec-Fix-ambiguous-python-interpreter-name.patch [bz#1571533] +- kvm-qemu-ga-blacklisting-guest-exec-and-guest-exec-statu.patch [bz#1518132] +- kvm-redhat-rewrap-build_configure.sh-cmdline-for-the-rh-.patch +- kvm-redhat-remove-the-VTD-LIVE_BLOCK_OPS-and-RHV-options.patch +- kvm-redhat-fix-the-rh-env-prep-target-s-dependency-on-th.patch +- kvm-redhat-remove-dead-code-related-to-s390-not-s390x.patch +- kvm-redhat-sync-compiler-flags-from-the-spec-file-to-rh-.patch +- kvm-redhat-sync-guest-agent-enablement-and-tcmalloc-usag.patch +- kvm-redhat-fix-up-Python-3-dependency-for-building-QEMU.patch +- kvm-redhat-fix-up-Python-dependency-for-SRPM-generation.patch +- kvm-redhat-disable-glusterfs-dependency-support-temporar.patch +- Resolves: bz#1518132 + (Ensure file access RPCs are disabled by default) +- Resolves: bz#1571533 + (Convert qemu-kvm python scripts to python3) +- Resolves: bz#1590511 + (Fails to start guest with Intel vGPU device) + +* Thu Jun 21 2018 Danilo C. L. de Paula - 2.12.0-13.el8 +- Resolves: bz#1508137 + ([IBM 8.0 FEAT] KVM: Interactive Bootloader (qemu)) +- Resolves: bz#1513558 + (Remove RHEL6 machine types) +- Resolves: bz#1568600 + (pc-i440fx-rhel7.6.0 and pc-q35-rhel7.6.0 machine types (x86)) +- Resolves: bz#1570029 + ([IBM 8.0 FEAT] KVM: 3270 Connectivity - qemu part) +- Resolves: bz#1578855 + (Enable Native Ceph support on non x86_64 CPUs) +- Resolves: bz#1585651 + (RHEL 7.6 new pseries machine type (ppc64le)) +- Resolves: bz#1592337 + ([IBM 8.0 FEAT] KVM: CPU Model z14 ZR1 (qemu-kvm)) + +* Tue May 15 2018 Danilo C. L. de Paula - 2.12.0-11.el8.1 +- Resolves: bz#1576468 + (Enable vhost_user in qemu-kvm 2.12) + +* Wed May 09 2018 Danilo de Paula - 2.12.0-11.el8 +- Resolves: bz#1574406 + ([RHEL 8][qemu-kvm] Failed to find romfile "efi-virtio.rom") +- Resolves: bz#1569675 + (Backwards compatibility of pc-*-rhel7.5.0 and older machine-types) +- Resolves: bz#1576045 + (Fix build issue by using python3) +- Resolves: bz#1571145 + (qemu-kvm segfaults on RHEL 8 when run guestfsd under TCG) + +* Fri Apr 20 2018 Danilo de Paula - 2.12.0-10.el +- Fixing some issues with packaging. +- Rebasing to 2.12.0-rc4 + +* Fri Apr 13 2018 Danilo de Paula - 2.11.0-7.el8 +- Bumping epoch for RHEL8 and dropping self-obsoleting + +* Thu Apr 12 2018 Danilo de Paula - 2.11.0-6.el8 +- Rebuilding + +* Mon Mar 05 2018 Danilo de Paula - 2.11.0-5.el8 +- Prepare building on RHEL-8.0