* refs/heads/tmp-6f994bf:
Revert "ANDROID: sched: Disable find_best_target() by default"
ANDROID: cpufreq: times: don't copy invalid freqs from freq table
UPSTREAM: filemap: add a comment about FAULT_FLAG_RETRY_NOWAIT behavior
BACKPORT: filemap: drop the mmap_sem for all blocking operations
BACKPORT: filemap: kill page_cache_read usage in filemap_fault
UPSTREAM: filemap: pass vm_fault to the mmap ra helpers
ANDROID: binder: remove extra declaration left after backport
FROMGIT: binder: fix BUG_ON found by selinux-testsuite
ANDROID: sched: Disable find_best_target() by default
ANDROID: sched/fair: Make the EAS wake-up prefer-idle aware
Linux 4.19.32
power: supply: charger-manager: Fix incorrect return value
ALSA: hda - Enforces runtime_resume after S3 and S4 for each codec
ALSA: hda - Record the current power state before suspend/resume calls
locking/lockdep: Add debug_locks check in __lock_downgrade()
x86/unwind: Add hardcoded ORC entry for NULL
x86/unwind: Handle NULL pointer calls better in frame unwinder
loop: access lo_backing_file only when the loop device is Lo_bound
netfilter: ebtables: remove BUGPRINT messages
f2fs: fix to avoid deadlock of atomic file operations
RDMA/cma: Rollback source IP address if failing to acquire device
drm: Reorder set_property_atomic to avoid returning with an active ww_ctx
Bluetooth: hci_ldisc: Postpone HCI_UART_PROTO_READY bit set in hci_uart_set_proto()
Bluetooth: hci_ldisc: Initialize hci_dev before open()
Bluetooth: Fix decrementing reference count twice in releasing socket
Bluetooth: hci_uart: Check if socket buffer is ERR_PTR in h4_recv_buf()
media: v4l2-ctrls.c/uvc: zero v4l2_event
ext4: brelse all indirect buffer in ext4_ind_remove_space()
ext4: fix data corruption caused by unaligned direct AIO
ext4: fix NULL pointer dereference while journal is aborted
ALSA: ac97: Fix of-node refcount unbalance
ALSA: hda/ca0132 - make pci_iounmap() call conditional
ALSA: x86: Fix runtime PM for hdmi-lpe-audio
SMB3: Fix SMB3.1.1 guest mounts to Samba
irqchip/gic-v3-its: Fix comparison logic in lpi_range_cmp
objtool: Move objtool_file struct off the stack
perf probe: Fix getting the kernel map
cifs: allow guest mounts to work for smb3.11
futex: Ensure that futex address is aligned in handle_futex_death()
scsi: ibmvscsi: Fix empty event pool access during host removal
scsi: ibmvscsi: Protect ibmvscsi_head from concurrent modificaiton
powerpc/vdso64: Fix CLOCK_MONOTONIC inconsistencies across Y2038
MIPS: Fix kernel crash for R6 in jump label branch function
MIPS: Ensure ELF appended dtb is relocated
mips: loongson64: lemote-2f: Add IRQF_NO_SUSPEND to "cascade" irqaction.
udf: Fix crash on IO error during truncate
libceph: wait for latest osdmap in ceph_monc_blacklist_add()
iommu/amd: fix sg->dma_address for sg->offset bigger than PAGE_SIZE
drm/vmwgfx: Return 0 when gmrid::get_node runs out of ID's
drm/vmwgfx: Don't double-free the mode stored in par->set_mode
mmc: renesas_sdhi: limit block count to 16 bit for old revisions
mmc: mxcmmc: "Revert mmc: mxcmmc: handle highmem pages"
mmc: pxamci: fix enum type confusion
ALSA: firewire-motu: use 'version' field of unit directory to identify model
ALSA: hda - add Lenovo IdeaCentre B550 to the power_save_blacklist
ANDROID: dm-bow: Fix 32 bit compile errors
UPSTREAM: sched/pelt: Skip updating util_est when utilization is higher than CPU's capacity
UPSTREAM: sched/fair: Update scale invariance of PELT
UPSTREAM: sched/fair: Move the rq_of() helper function
UPSTREAM: sched/fair: Remove setting task's se->runnable_weight during PELT update
ANDROID: Add dm-bow to cuttlefish configuration
UPSTREAM: binder: fix handling of misaligned binder object
UPSTREAM: binder: fix sparse issue in binder_alloc_selftest.c
BACKPORT: binder: use userspace pointer as base of buffer space
UPSTREAM: binder: fix kerneldoc header for struct binder_buffer
BACKPORT: binder: remove user_buffer_offset
UPSTREAM: binder: remove kernel vm_area for buffer space
UPSTREAM: binder: avoid kernel vm_area for buffer fixups
BACKPORT: binder: add function to copy binder object from buffer
BACKPORT: binder: add functions to copy to/from binder buffers
UPSTREAM: binder: create userspace-to-binder-buffer copy function
ANDROID: dm-bow: Add dm-bow feature
f2fs: set pin_file under CAP_SYS_ADMIN
f2fs: fix to avoid deadlock in f2fs_read_inline_dir()
f2fs: fix to adapt small inline xattr space in __find_inline_xattr()
f2fs: fix to do sanity check with inode.i_inline_xattr_size
f2fs: give some messages for inline_xattr_size
f2fs: don't trigger read IO for beyond EOF page
f2fs: fix to add refcount once page is tagged PG_private
f2fs: remove wrong comment in f2fs_invalidate_page()
f2fs: fix to use kvfree instead of kzfree
f2fs: print more parameters in trace_f2fs_map_blocks
f2fs: trace f2fs_ioc_shutdown
f2fs: fix to avoid deadlock of atomic file operations
f2fs: fix to dirty inode for i_mode recovery
f2fs: give random value to i_generation
f2fs: no need to take page lock in readdir
f2fs: fix to update iostat correctly in IPU path
f2fs: fix encrypted page memory leak
f2fs: make fault injection covering __submit_flush_wait()
f2fs: fix to retry fill_super only if recovery failed
f2fs: silence VM_WARN_ON_ONCE in mempool_alloc
f2fs: correct spelling mistake
f2fs: fix wrong #endif
f2fs: don't clear CP_QUOTA_NEED_FSCK_FLAG
f2fs: don't allow negative ->write_io_size_bits
f2fs: fix to check inline_xattr_size boundary correctly
Revert "f2fs: fix to avoid deadlock of atomic file operations"
Revert "f2fs: fix to check inline_xattr_size boundary correctly"
f2fs: do not use mutex lock in atomic context
f2fs: fix potential data inconsistence of checkpoint
f2fs: fix to avoid deadlock of atomic file operations
f2fs: fix to check inline_xattr_size boundary correctly
f2fs: jump to label 'free_node_inode' when failing from d_make_root()
f2fs: fix to document inline_xattr_size option
f2fs: fix to data block override node segment by mistake
f2fs: fix typos in code comments
f2fs: use xattr_prefix to wrap up
f2fs: sync filesystem after roll-forward recovery
f2fs: flush quota blocks after turnning it off
f2fs: avoid null pointer exception in dcc_info
f2fs: don't wake up too frequently, if there is lots of IOs
f2fs: try to keep CP_TRIMMED_FLAG after successful umount
f2fs: add quick mode of checkpoint=disable for QA
f2fs: run discard jobs when put_super
f2fs: fix to set sbi dirty correctly
f2fs: fix to initialize variable to avoid UBSAN/smatch warning
f2fs: UBSAN: set boolean value iostat_enable correctly
f2fs: add brackets for macros
f2fs: check if file namelen exceeds max value
f2fs: fix to trigger fsck if dirent.name_len is zero
f2fs: no need to check return value of debugfs_create functions
f2fs: export FS_NOCOW_FL flag to user
f2fs: check inject_rate validity during configuring
f2fs: remove set but not used variable 'err'
f2fs: fix compile warnings: 'struct *' declared inside parameter list
f2fs: change error code to -ENOMEM from -EINVAL
Conflicts:
drivers/md/Kconfig
kernel/sched/fair.c
Change-Id: I2c6ba055f1160864446c87507a7fd7c8249ad885
Signed-off-by: Ivaylo Georgiev <irgeorgiev@codeaurora.org>
Based on https://www.redhat.com/archives/dm-devel/2019-March/msg00025.html
Third version of dm-bow. Key changes:
Free list added
Support for block sizes other than 4k
Handles writes during trim phase, and overlapping trims
Integer overflow error
Support trims even if underlying device doesn't
Numerous small bug fixes
bow == backup on write
USE CASE:
dm-bow takes a snapshot of an existing file system before mounting.
The user may, before removing the device, commit the snapshot.
Alternatively the user may remove the device and then run a command
line utility to restore the device to its original state.
dm-bow does not require an external device
dm-bow efficiently uses all the available free space on the file system.
IMPLEMENTATION:
dm-bow can be in one of three states.
In state one, the free blocks on the device are identified by issuing
an FSTRIM to the filesystem.
In state two, any writes cause the overwritten data to be backup up
to the available free space. While in this state, the device can be
restored by unmounting the filesystem, removing the dm-bow device
and running a usermode tool over the underlying device.
In state three, the changes are committed, dm-bow is in pass-through
mode and the drive can no longer be restored.
It is planned to use this driver to enable restoration of a failed
update attempt on Android devices using ext4.
Test: Can boot Android with userdata mounted on this device. Can commit
userdata after SUW has run. Can then reboot, make changes and roll back.
Known issues:
Mutex is held around entire flush operation, including lengthy I/O. Plan
is to convert to state machine with pending queues.
Interaction with block encryption is unknown, especially with respect
to sector 0.
Bug: 119769411
Bug: 129280212
Test: Dogfooded on Wahoo.
Ran under Cuttlefish, running VtsKernelBowTest &
VtsKernelCheckpointTest tests against 4.19, 4.14 & 4.9 kernels
Change-Id: Id70988bbd797ebe3e76fc175094388b423c8da8c
Signed-off-by: Paul Lawrence <paullawrence@google.com>
* origin/tmp-7876320:
Linux 4.19-rc4
Code of Conduct: Let's revamp it.
x86/APM: Fix build warning when PROC_FS is not enabled
NFS: Don't open code clearing of delegation state
NFSv4.1 fix infinite loop on I/O.
NFSv4: Fix a tracepoint Oops in initiate_file_draining()
pNFS: Ensure we return the error if someone kills a waiting layoutget
NFSv4: Fix a tracepoint Oops in initiate_file_draining()
Revert "x86/mm/legacy: Populate the user page-table with user pgd's"
xen/gntdev: fix up blockable calls to mn_invl_range_start
xen: fix GCC warning and remove duplicate EVTCHN_ROW/EVTCHN_COL usage
xen: avoid crash in disable_hotplug_cpu
xen/balloon: add runtime control for scrubbing ballooned out pages
xen/manage: don't complain about an empty value in control/sysrq node
asm-generic: io: Fix ioport_map() for !CONFIG_GENERIC_IOMAP && CONFIG_INDIRECT_PIO
mm: get rid of vmacache_flush_all() entirely
MAINTAINERS: Make Dennis the percpu tree maintainer
pstore: Fix incorrect persistent ram buffer mapping
drm/nouveau/devinit: fix warning when PMU/PRE_OS is missing
null_blk: fix zoned support for non-rq based operation
cifs: read overflow in is_valid_oplock_break()
nfp: flower: reject tunnel encap with ipv6 outer headers for offloading
nfp: flower: fix vlan match by checking both vlan id and vlan pcp
tipc: check return value of __tipc_dump_start()
s390/qeth: don't dump past end of unknown HW header
s390/qeth: use vzalloc for QUERY OAT buffer
s390/qeth: switch on SG by default for IQD devices
s390/qeth: indicate error when netdev allocation fails
x86/efi: Load fixmap GDT in efi_call_phys_epilog() before setting %cr3
x86/xen: Disable CPU0 hotplug for Xen PV
tracing/Makefile: Fix handling redefinition of CC_FLAGS_FTRACE
cifs: integer overflow in in SMB2_ioctl()
CIFS: fix wrapping bugs in num_entries()
cifs: prevent integer overflow in nxt_dir_entry()
s390/zcrypt: remove VLA usage from the AP bus
firmware: Fix security issue with request_firmware_into_buf()
vmbus: don't return values for uninitalized channels
fpga: dfl: fme: fix return value check in in pr_mgmt_init()
misc: hmc6352: fix potential Spectre v1
Tools: hv: Fix a bug in the key delete code
misc: ibmvsm: Fix wrong assignment of return code
android: binder: fix the race mmap and alloc_new_buf_locked
mei: bus: need to unlink client before freeing
mei: bus: fix hw module get/put balance
mei: fix use-after-free in mei_cl_write
mei: ignore not found client in the enumeration
rds: fix two RCU related problems
r8169: Clear RTL_FLAG_TASK_*_PENDING when clearing RTL_FLAG_TASK_ENABLED
erspan: fix error handling for erspan tunnel
erspan: return PACKET_REJECT when the appropriate tunnel is not found
tcp: rate limit synflood warnings further
MIPS: lantiq: dma: add dev pointer
xtensa: enable SG chaining in Kconfig
xtensa: remove unnecessary KBUILD_SRC ifeq conditional
PCI: Fix enabling of PASID on RC integrated endpoints
IB/hfi1,PCI: Allow bus reset while probing
PCI: Fix faulty logic in pci_reset_bus()
x86/EISA: Don't probe EISA bus for Xen PV guests
drm/amdgpu: fix error handling in amdgpu_cs_user_fence_chunk
perf tools: Fix maps__find_symbol_by_name()
tools headers uapi: Update tools's copy of linux/if_link.h
blk-cgroup: increase number of supported policies
staging: vboxvideo: Change address of scanout buffer on page-flip
staging: vboxvideo: Fix IRQs no longer working
of: fix phandle cache creation for DTs with no phandles
tools headers uapi: Update tools's copy of linux/vhost.h
tools headers uapi: Update tools's copies of kvm headers
drm/i915/overlay: Allocate physical registers from stolen
tools headers uapi: Update tools's copy of drm/drm.h
tools headers uapi: Update tools's copy of asm-generic/unistd.h
tools headers uapi: Update tools's copy of linux/perf_event.h
PCI: pciehp: Fix hot-add vs powerfault detection order
switchtec: Fix Spectre v1 vulnerability
Revert "PCI: Add ACS quirk for Intel 300 series"
MAINTAINERS: Add Gustavo Pimentel as DesignWare PCI maintainer
MAINTAINERS: Add entries for PPC64 RPA PCI hotplug drivers
arm64: kernel: arch_crash_save_vmcoreinfo() should depend on CONFIG_CRASH_CORE
arm64: jump_label.h: use asm_volatile_goto macro instead of "asm goto"
Revert "printk: make sure to print log on console."
drm/amdgpu: move PSP init prior to IH in gpu reset
drm/amdgpu: Fix SDMA hang in prt mode v2
drm/amdgpu: fix amdgpu_mn_unlock() in the CS error path
hexagon: modify ffs() and fls() to return int
arch/hexagon: fix kernel/dma.c build warning
netfilter: xt_hashlimit: use s->file instead of s->private
netfilter: nfnetlink_queue: Solve the NFQUEUE/conntrack clash for NF_REPEAT
netfilter: cttimeout: ctnl_timeout_find_get() returns incorrect pointer to type
netfilter: conntrack: timeout interface depend on CONFIG_NF_CONNTRACK_TIMEOUT
netfilter: conntrack: reset tcp maxwin on re-register
dm thin metadata: try to avoid ever aborting transactions
Revert "cdc-acm: implement put_char() and flush_chars()"
usb: Change usb_of_get_companion_dev() place to usb/common
usb: xhci: fix interrupt transfer error happened on MTK platforms
qmi_wwan: Support dynamic config on Quectel EP06
drm/i915/bdw: Increase IPS disable timeout to 100ms
ethernet: renesas: convert to SPDX identifiers
staging: gasket: TODO: re-implement using UIO
tty: hvc: hvc_write() fix break condition
tty: hvc: hvc_poll() fix read loop batching
tty: hvc: hvc_poll() fix read loop hang
x86/doc: Fix Documentation/x86/earlyprintk.txt
perf/core: Force USER_DS when recording user stack data
locking/ww_mutex: Fix spelling mistake "cylic" -> "cyclic"
locking/lockdep: Delete unnecessary #include
tools/lib/lockdep: Add dummy task_struct state member
tools/lib/lockdep: Add empty nmi.h
tools/lib/lockdep: Update Sasha Levin email to MSFT
ovl: fix oopses in ovl_fill_super() failure paths
staging/fbtft: Update TODO and mailing lists
sched/fair: Fix kernel-doc notation warning
jump_label: Fix typo in warning message
sched/fair: Fix load_balance redo for !imbalance
sched/fair: Fix scale_rt_capacity() for SMT
sched/fair: Fix vruntime_normalized() for remote non-migration wakeup
sched/pelt: Fix update_blocked_averages() for RT and DL classes
sched/topology: Set correct NUMA topology type
sched/debug: Fix potential deadlock when writing to sched_features
staging: erofs: rename superblock flags (MS_xyz -> SB_xyz)
locking/mutex: Fix mutex debug call and ww_mutex documentation
perf/UAPI: Clearly mark __PERF_SAMPLE_CALLCHAIN_EARLY as internal use
perf/x86/intel: Add support/quirk for the MISPREDICT bit on Knights Landing CPUs
ip: frags: fix crash in ip_do_fragment()
net/tls: Set count of SG entries if sk_alloc_sg returns -ENOSPC
net: ena: fix incorrect usage of memory barriers
net: ena: fix missing calls to READ_ONCE
net: ena: fix missing lock during device destruction
net: ena: fix potential double ena_destroy_device()
net: ena: fix device destruction to gracefully free resources
net: ena: fix driver when PAGE_SIZE == 64kB
net: ena: fix surprise unplug NULL dereference kernel crash
fs/cifs: require sha512
fs/cifs: suppress a string overflow warning
tcp: really ignore MSG_ZEROCOPY if no SO_ZEROCOPY
net_sched: properly cancel netlink dump on failure
xen/netfront: fix waiting for xenbus state change
r8169: set TxConfig register after TX / RX is enabled, just like RxConfig
tipc: call start and done ops directly in __tipc_nl_compat_dumpit()
dm raid: bump target version, update comments and documentation
dm raid: fix RAID leg rebuild errors
dm raid: fix rebuild of specific devices by updating superblock
dm raid: fix stripe adding reshape deadlock
drm/nouveau/disp/gm200-: enforce identity-mapped SOR assignment for LVDS/eDP panels
drm/nouveau/disp: fix DP disable race
drm/nouveau/disp: move eDP panel power handling
drm/nouveau/disp: remove unused struct member
drm/nouveau/TBDdevinit: don't fail when PMU/PRE_OS is missing from VBIOS
drm/nouveau/mmu: don't attempt to dereference vmm without valid instance pointer
drm/nouveau: fix oops in client init failure path
drm/nouveau: Fix nouveau_connector_ddc_detect()
drm/nouveau/drm/nouveau: Don't forget to cancel hpd_work on suspend/unload
drm/nouveau/drm/nouveau: Prevent handling ACPI HPD events too early
drm/nouveau: Reset MST branching unit before enabling
drm/nouveau: Only write DP_MSTM_CTRL when needed
drm/nouveau: Remove useless poll_enable() call in drm_load()
drm/nouveau: Remove useless poll_disable() call in switcheroo_set_state()
drm/nouveau: Remove useless poll_enable() call in switcheroo_set_state()
drm/nouveau: Fix deadlocks in nouveau_connector_detect()
drm/nouveau/drm/nouveau: Use pm_runtime_get_noresume() in connector_detect()
drm/nouveau/drm/nouveau: Fix deadlock with fb_helper with async RPM requests
drm/nouveau: Remove duplicate poll_enable() in pmops_runtime_suspend()
drm/nouveau/drm/nouveau: Fix bogus drm_kms_helper_poll_enable() placement
RDMA/mlx4: Ensure that maximal send/receive SGE less than supported by HW
RDMA/cma: Protect cma dev list with lock
xtensa: ISS: don't allocate memory in platform_setup
dm raid: fix reshape race on small devices
dm: disable CRYPTO_TFM_REQ_MAY_SLEEP to fix a GFP_KERNEL recursion deadlock
HID: i2c-hid: Don't reset device upon system resume
net/iucv: declare iucv_path_table_empty() as static
net/af_iucv: fix skb handling on HiperTransport xmit error
net/af_iucv: drop inbound packets with invalid flags
net/sched: fix memory leak in act_tunnel_key_init()
tipc: orphan sock in tipc_release()
drm/i915/gvt: Fix the incorrect length of child_device_config issue
net/mlx5: Fix possible deadlock from lockdep when adding fte to fg
net/mlx5e: Ethtool steering, fix udp source port value
net/mlx5: Check for error in mlx5_attach_interface
net/mlx5: Consider PCI domain in search for next dev
net/mlx5: Fix not releasing read lock when adding flow rules
net/mlx5: E-Switch, Fix memory leak when creating switchdev mode FDB tables
net/mlx5: Use u16 for Work Queue buffer strides offset
net/mlx5: Use u16 for Work Queue buffer fragment size
net/mlx5: Fix debugfs cleanup in the device init/remove flow
net/mlx5: Fix use-after-free in self-healing flow
RDMA/uverbs: Fix error cleanup path of ib_uverbs_add_one()
bnxt_re: Fix couple of memory leaks that could lead to IOMMU call traces
IB/ipoib: Avoid a race condition between start_xmit and cm_rep_handler
nvmet-rdma: fix possible bogus dereference under heavy load
net: qca_spi: Fix race condition in spi transfers
be2net: Fix memory leak in be_cmd_get_profile_config()
mlxsw: spectrum_buffers: Set up a dedicated pool for BUM traffic
usb: cdc-wdm: Fix a sleep-in-atomic-context bug in service_outstanding_interrupt()
usb: misc: uss720: Fix two sleep-in-atomic-context bugs
usb: host: u132-hcd: Fix a sleep-in-atomic-context bug in u132_get_frame()
usb: Avoid use-after-free by flushing endpoints early in usb_set_interface()
linux/mod_devicetable.h: fix kernel-doc missing notation for typec_device_id
usb/typec: fix kernel-doc notation warning for typec_match_altmode
usb: Don't die twice if PCI xhci host is not responding in resume
usb: mtu3: fix error of xhci port id when enable U3 dual role
usb: uas: add support for more quirk flags
USB: Add quirk to support DJI CineSSD
usb: typec: fix kernel-doc parameter warning
usb/dwc3/gadget: fix kernel-doc parameter warning
USB: yurex: Check for truncation in yurex_read()
USB: yurex: Fix buffer over-read in yurex_write()
usb: host: xhci-plat: Iterate over parent nodes for finding quirks
xhci: Fix use after free for URB cancellation on a reallocated endpoint
USB: add quirk for WORLDE Controller KS49 or Prodipe MIDI 49C USB controller
usb: dwc2: Fix call location of dwc2_check_core_endianness
HID: sensor-hub: Restore fixup for Lenovo ThinkPad Helix 2 sensor hub report
HID: core: fix NULL pointer dereference
mmc: meson-mx-sdio: fix OF child-node lookup
riscv: Do not overwrite initrd_start and initrd_end
iw_cxgb4: only allow 1 flush on user qps
IB/core: Release object lock if destroy failed
RDMA/ucma: check fd type in ucma_migrate_id()
HID: core: fix grouping by application
HID: multitouch: fix Elan panels with 2 input modes declaration
dm verity: fix crash on bufio buffer that was allocated with vmalloc
mmc: omap_hsmmc: fix wakeirq handling on removal
s390/crypto: Fix return code checking in cbc_paes_crypt()
drm/i915/gvt: Fix life cycle reference on KVM mm
ovl: add ovl_fadvise()
iio: imu: st_lsm6dsx: take into account ts samples in wm configuration
Revert "iio: temperature: maxim_thermocouple: add MAX31856 part"
ipmi: Fix NULL pointer dereference in ssif_probe
netfilter: nf_tables: release chain in flushing set
netfilter: kconfig: nat related expression depend on nftables core
ipmi: Fix I2C client removal in the SSIF driver
ipmi: Move BT capabilities detection to the detect call
ipmi: Rework SMI registration failure
ipmi: kcs_bmc: don't change device name
perf annotate: Fix parsing aarch64 branch instructions after objdump update
perf probe powerpc: Ignore SyS symbols irrespective of endianness
vfs: implement readahead(2) using POSIX_FADV_WILLNEED
perf event-parse: Use fixed size string for comms
perf util: Fix bad memory access in trace info.
perf tools: Streamline bpf examples and headers installation
perf evsel: Fix potential null pointer dereference in perf_evsel__new_idx()
perf arm64: Fix include path for asm-generic/unistd.h
perf/hw_breakpoint: Simplify breakpoint enable in perf_event_modify_breakpoint
perf/hw_breakpoint: Enable breakpoint in modify_user_hw_breakpoint
perf/hw_breakpoint: Remove superfluous bp->attr.disabled = 0
perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set
perf tests: Add breakpoint modify tests
perf annotate: Properly interpret indirect call
vfs: add the fadvise() file operation
Documentation/filesystems: update documentation of file_operations
ovl: fix GPF in swapfile_activate of file from overlayfs over xfs
ovl: respect FIEMAP_FLAG_SYNC flag
scsi: qedi: Add the CRC size within iSCSI NVM image
scsi: iscsi: target: Fix conn_ops double free
scsi: iscsi: target: Set conn->sess to NULL when iscsi_login_set_conn_values fails
HID: hid-saitek: Add device ID for RAT 7 Contagion
pinctrl: madera: Fix possible NULL pointer with pdata config
pinctrl: ingenic: Fix group & function error checking
netfilter: nf_tables: rework ct timeout set support
netfilter: conntrack: place 'new' timeout in first location too
pinctrl: msm: Really mask level interrupts to prevent latching
usb: dwc3: pci: Fix return value check in dwc3_byt_enable_ulpi_refclock()
usb: gadget: udc: renesas_usb3: fix maxpacket size of ep0
usb: gadget: fotg210-udc: Fix memory leak of fotg210->ep[i]
USB: net2280: Fix erroneous synchronization change
usb: dwc3: of-simple: avoid unused function warnings
Revert "staging: erofs: disable compiling temporarile"
HID: core: fix memory leak on probe
HID: input: fix leaking custom input node name
HID: add support for Apple Magic Keyboards
HID: i2c-hid: Fix flooded incomplete report after S3 on Rayd touchscreen
HID: intel-ish-hid: Enable Sunrise Point-H ish driver
MAINTAINERS: Switch a maintainer for drivers/staging/gasket
staging: wilc1000: revert "fix TODO to compile spi and sdio components in single module"
USB: serial: ti_usb_3410_5052: fix array underflow in completion handler
USB: serial: io_ti: fix array underflow in completion handler
dmaengine: mic_x100_dma: use devm_kzalloc to fix an issue
netfilter: xt_checksum: ignore gso skbs
netfilter: xt_cluster: add dependency on conntrack module
netfilter: conntrack: remove duplicated include from nf_conntrack_proto_udp.c
Change-Id: I9fdae855388077fd5a44e66153c360a7ed1c7cc5
[rishabhb@codeaurora.org:Resolved minor merge conflicts].
Signed-off-by: Rishabh Bhatnagar <rishabhb@codeaurora.org>
Bump target version to reflect the documented fixes are available.
Also fix some code comments (typos and clarity).
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
This is a wrap-up of three patches pending upstream approval.
I'm bundling them because they are interdependent, and it'll be
easier to drop it on rebase later.
1. dm: allow a dm-fs-style device to be shared via dm-ioctl
Integrates feedback from Alisdair, Mike, and Kiyoshi.
Two main changes occur here:
- One function is added which allows for a programmatically created
mapped device to be inserted into the dm-ioctl hash table. This binds
the device to a name and, optional, uuid which is needed by udev and
allows for userspace management of the mapped device.
- dm_table_complete() was extended to handle all of the final
functional changes required for the table to be operational once
called.
2. init: boot to device-mapper targets without an initr*
Add a dm= kernel parameter modeled after the md= parameter from
do_mounts_md. It allows for device-mapper targets to be configured at
boot time for use early in the boot process (as the root device or
otherwise). It also replaces /dev/XXX calls with major:minor opportunistically.
The format is dm="name uuid ro,table line 1,table line 2,...". The
parser expects the comma to be safe to use as a newline substitute but,
otherwise, uses the normal separator of space. Some attempt has been
made to make it forgiving of additional spaces (using skip_spaces()).
A mapped device created during boot will be assigned a minor of 0 and
may be access via /dev/dm-0.
An example dm-linear root with no uuid may look like:
root=/dev/dm-0 dm="lroot none ro, 0 4096 linear /dev/ubdb 0, 4096 4096 linear /dv/ubdc 0"
Once udev is started, /dev/dm-0 will become /dev/mapper/lroot.
Older upstream threads:
http://marc.info/?l=dm-devel&m=127429492521964&w=2http://marc.info/?l=dm-devel&m=127429499422096&w=2http://marc.info/?l=dm-devel&m=127429493922000&w=2
Latest upstream threads:
https://patchwork.kernel.org/patch/104859/https://patchwork.kernel.org/patch/104860/https://patchwork.kernel.org/patch/104861/
Bug: 27175947
Signed-off-by: Will Drewry <wad@chromium.org>
Review URL: http://codereview.chromium.org/2020011
Change-Id: I92bd53432a11241228d2e5ac89a3b20d19b05a31
[AmitP: Refactored the original changes based on upstream changes,
commit e52347bd66 ("Documentation/admin-guide: split the kernel parameter list to a separate file")]
Signed-off-by: Amit Pundir <amit.pundir@linaro.org>
The metadata low watermark threshold is set by the kernel. But the
kernel depends on userspace to extend the thinpool metadata device when
the threshold is crossed.
Since the metadata low watermark threshold is not visible to userspace,
upon receiving an event, userspace cannot tell that the kernel wants the
metadata device extended, instead of some other eventing condition.
Making it visible (but not settable) enables userspace to affirmatively
know the kernel is asking for a metadata device extension, by comparing
metadata_low_watermark against nr_free_blocks_metadata, also reported in
status.
Current solutions like dmeventd have their own thresholds for extending
the data and metadata devices, and both devices are checked against
their thresholds on each event. This lessens the value of the kernel-set
threshold, since userspace will either extend the metadata device sooner,
when receiving another event; or will receive the metadata lowater event
and do nothing, if dmeventd's threshold is less than the kernel's.
(This second case is dangerous. The metadata lowater event will not be
re-sent, so no further event will be generated before the metadata
device is out if space, unless some other event causes userspace to
recheck its thresholds.)
Signed-off-by: Andy Grover <agrover@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
When using external metadata device and internal hash, recalculate the
checksums when the device is created - so that dm-integrity doesn't
have to overwrite the device. The superblock stores the last position
when the recalculation ended, so that it is properly restarted.
Integrity tags that haven't been recalculated yet are ignored.
Also bump the target version.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Add a new class for dm-delay that delays flush requests. Previously,
flushes were delayed as writes, but it caused problems if the user
needed to create a device with one or a few slow sectors for the purpose
of testing - all flushes would be forwarded to this device and delayed,
and that skews the test results. Fix this by allowing to select 0 delay
for flushes.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Documentation/device-mapper-/thin-provisioning.txt's "Status" section no
longer reflected the current fitness level of DM thin-provisioning.
That is, DM thinp is no longer "EXPERIMENTAL". It has since seen
considerable improvement, has been fairly widely deployed and has
performed in a robust manner.
Update Documentation to dispel concern raised by potential DM thinp
users.
Reported-by: Drew Hastings <dhastings@crucialwebhost.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Add an optional parameter "start_sector" to allow the start of the
device to be offset by the specified number of 512-byte sectors. The
sectors below this offset are not used by the writecache device and are
left to be used for disk labels and/or userspace metadata (e.g. lvm).
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
The writecache target caches writes on persistent memory or SSD.
It is intended for databases or other programs that need extremely low
commit latency.
The writecache target doesn't cache reads because reads are supposed to
be cached in page cache in normal RAM.
If persistent memory isn't available this target can still be used in
SSD mode.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Colin Ian King <colin.king@canonical.com> # fix missing goto
Signed-off-by: Ross Zwisler <ross.zwisler@linux.intel.com> # fix compilation issue with !DAX
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> # use msecs_to_jiffies
Acked-by: Dan Williams <dan.j.williams@intel.com> # reworks to unify ARM and x86 flushing
Signed-off-by: Mike Snitzer <msnitzer@redhat.com>
Due to user confusion, clarify that it doesn't make sense to try to
create a thin-pool with "read_only" mode enabled.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
This allows platforms that are CPU/memory contrained to verify data
blocks only the first time they are read from the data device, rather
than every time. As such, it provides a reduced level of security
because only offline tampering of the data device's content will be
detected, not online tampering.
Hash blocks are still verified each time they are read from the hash
device, since verification of hash blocks is less performance critical
than data blocks, and a hash block will not be verified any more after
all the data blocks it covers have been verified anyway.
This option introduces a bitset that is used to check if a block has
been validated before or not. A block can be validated more than once
as there is no thread protection for the bitset.
These changes were developed and tested on entry-level Android Go
devices.
Signed-off-by: Patrik Torstensson <totte@google.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
In commit f8350daf7a ("dm cache: tune migration throttling") the
value for DEFAULT_MIGRATION_THRESHOLD was decreased from 204800 to
2048. Edit device-mapper/cache.txt to reflect the correct default
value for migration_threshold.
Signed-off-by: John Pittman <jpittman@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Make the format string for thinpool status more correct.
Swap the order of two items to correspond with reality.
Signed-off-by: mulhern <amulhern@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Fixes:
1. The use of "exceeds" when the opposite of exceeds, falls below,
was meant.
2. Properly speaking, a table can not exceed a threshold.
It emphasizes the important point, which is that it is the userspace
daemon's responsibility to check for low free space when a device
is resumed, since it won't get a special event indicating low free
space in that situation.
Signed-off-by: mulhern <amulhern@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
The 'mq' policy is no longer the default policy, and the default policy,
'smq', does not store hit counts.
Signed-off-by: mulhern <amulhern@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Use possessive pronoun where appropriate, instead of contraction.
Signed-off-by: mulhern <amulhern@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Add a note to snapshot.txt that the origin target must be suspended when
loading or unloading the snapshot target.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
This device mapper "unstriped" target remaps and unstripes I/O so it
is issued solely on a single drive in a HW RAID0 or dm-striped target.
In a 4 drive HW RAID0 the striped target exposes 1/4th of the LBA range
as a virtual drive. Each I/O to that virtual drive will only be issued
to the 1 drive that was selected of the 4 drives in the HW RAID0.
This unstriped target is most useful for Intel NVMe drives that have
multiple cores but that do not have firmware control to pin separate LBA
ranges to each discrete cpu core.
Signed-off-by: Scott Bauer <scott.bauer@intel.com>
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Acked-by: Keith Busch <keith.busch@intel.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
In order to avoid redoing synchronization/recovery/reshape partially,
the raid set got frozen until after all passed in table line flags had
been cleared. The related table reload sequence had to be precisely
followed, or reshaping may lead to data corruption caused by the active
mapping carrying on with a reshape when the inactive mapping already
had retrieved a stale reshape position.
Harden by retrieving the actual resync/recovery/reshape position
during resume whilst the active table is suspended thus avoiding
to keep the raid set frozen altogether. This prevents superfluous
redoing of an already resynchronized or recovered segment and,
most importantly, potential for redoing of an already reshaped
segment causing data corruption.
Fixes: d39f0010e ("dm raid: fix raid_resume() to keep raid set frozen as needed")
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
There are three important fields that indicate the overall health and
status of an array: dev_health, sync_ratio, and sync_action. They tell
us the condition of the devices in the array, and the degree to which
the array is synchronized.
This commit fixes a condition that is reported incorrectly. When a member
of the array is being rebuilt or a new device is added, the "recover"
process is used to synchronize it with the rest of the array. When the
process is complete, but the sync thread hasn't yet been reaped, it is
possible for the state of MD to be:
mddev->recovery = [ MD_RECOVERY_RUNNING MD_RECOVERY_RECOVER MD_RECOVERY_DONE ]
curr_resync_completed = <max dev size> (but not MaxSector)
and all rdevs to be In_sync.
This causes the 'array_in_sync' output parameter that is passed to
rs_get_progress() to be computed incorrectly and reported as 'false' --
or not in-sync. This in turn causes the dev_health status characters to
be reported as all 'a', rather than the proper 'A'.
This can cause erroneous output for several seconds at a time when tools
will want to be checking the condition due to events that are raised at
the end of a sync process. Fix this by properly calculating the
'array_in_sync' return parameter in rs_get_progress().
Also, remove an unnecessary intermediate 'recovery_cp' variable in
rs_get_progress().
Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Bumo dm-raid target version to 1.12.1 to reflect that commit cc27b0c78c
("md: fix deadlock between mddev_suspend() and md_write_start()") is
available.
This version change allows userspace to detect that MD fix is available.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
The dm-zoned device mapper target provides transparent write access
to zoned block devices (ZBC and ZAC compliant block devices).
dm-zoned hides to the device user (a file system or an application
doing raw block device accesses) any constraint imposed on write
requests by the device, equivalent to a drive-managed zoned block
device model.
Write requests are processed using a combination of on-disk buffering
using the device conventional zones and direct in-place processing for
requests aligned to a zone sequential write pointer position.
A background reclaim process implemented using dm_kcopyd_copy ensures
that conventional zones are always available for executing unaligned
write requests. The reclaim process overhead is minimized by managing
buffer zones in a least-recently-written order and first targeting the
oldest buffer zones. Doing so, blocks under regular write access (such
as metadata blocks of a file system) remain stored in conventional
zones, resulting in no apparent overhead.
dm-zoned implementation focus on simplicity and on minimizing overhead
(CPU, memory and storage overhead). For a 14TB host-managed disk with
256 MB zones, dm-zoned memory usage per disk instance is at most about
3 MB and as little as 5 zones will be used internally for storing metadata
and performing buffer zone reclaim operations. This is achieved using
zone level indirection rather than a full block indirection system for
managing block movement between zones.
dm-zoned primary target is host-managed zoned block devices but it can
also be used with host-aware device models to mitigate potential
device-side performance degradation due to excessive random writing.
Zoned block devices can be formatted and checked for use with the dm-zoned
target using the dmzadm utility available at:
https://github.com/hgst/dm-zoned-tools
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Hannes Reinecke <hare@suse.com>
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
[Mike Snitzer partly refactored Damien's original work to cleanup the code]
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
whether blocks should migrate to/from the cache. The bio-prison-v2
interface supports this improvement by enabling direct dispatch of
work to workqueues rather than having to delay the actual work
dispatch to the DM cache core. So the dm-cache policies are much more
nimble by being able to drive IO as they see fit. One immediate
benefit from the improved latency is a cache that should be much more
adaptive to changing workloads.
- Add a new DM integrity target that emulates a block device that has
additional per-sector tags that can be used for storing integrity
information.
- Add a new authenticated encryption feature to the DM crypt target that
builds on the capabilities provided by the DM integrity target.
- Add MD interface for switching the raid4/5/6 journal mode and update
the DM raid target to use it to enable aid4/5/6 journal write-back
support.
- Switch the DM verity target over to using the asynchronous hash crypto
API (this helps work better with architectures that have access to
off-CPU algorithm providers, which should reduce CPU utilization).
- Various request-based DM and DM multipath fixes and improvements from
Bart and Christoph.
- A DM thinp target fix for a bio structure leak that occurs for each
discard IFF discard passdown is enabled.
- A fix for a possible deadlock in DM bufio and a fix to re-check the
new buffer allocation watermark in the face of competing admin changes
to the 'max_cache_size_bytes' tunable.
- A couple DM core cleanups.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJZB6vtAAoJEMUj8QotnQNaoicIALuZTLElgAzxzA28cfk1+1Ea
Gd09CfJ3M6cvk/YGUU7WwiSYIwu16yOJALG4sLcYnEmUCzvKfFPcl/RpeSJHPpYM
0aVXa6NIJw7K2r3C17toiK2DRMHYw6QU843WeWI93vBW13lDJklNJL9fM7GBEOLH
NMSNw2mAq9ajtLlnJhM3ZfhloA7/u/jektvlBO1AA3RQ5Kx1cXVXFPqN7FdRfcqp
4RuEMe9faAadlXLsj3bia5IBmF/W0Qza6JilP+NLKLWB4fm7LZDjN/k+TsHWMa9e
cGR73TgUGLMBJX+sDJy8R3oeBG9JZkFVkD7I30eCjzyhSOs/54XNYQ23EkqHJU0=
=9Ryi
-----END PGP SIGNATURE-----
Merge tag 'for-4.12/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm
Pull device mapper updates from Mike Snitzer:
- A major update for DM cache that reduces the latency for deciding
whether blocks should migrate to/from the cache. The bio-prison-v2
interface supports this improvement by enabling direct dispatch of
work to workqueues rather than having to delay the actual work
dispatch to the DM cache core. So the dm-cache policies are much more
nimble by being able to drive IO as they see fit. One immediate
benefit from the improved latency is a cache that should be much more
adaptive to changing workloads.
- Add a new DM integrity target that emulates a block device that has
additional per-sector tags that can be used for storing integrity
information.
- Add a new authenticated encryption feature to the DM crypt target
that builds on the capabilities provided by the DM integrity target.
- Add MD interface for switching the raid4/5/6 journal mode and update
the DM raid target to use it to enable aid4/5/6 journal write-back
support.
- Switch the DM verity target over to using the asynchronous hash
crypto API (this helps work better with architectures that have
access to off-CPU algorithm providers, which should reduce CPU
utilization).
- Various request-based DM and DM multipath fixes and improvements from
Bart and Christoph.
- A DM thinp target fix for a bio structure leak that occurs for each
discard IFF discard passdown is enabled.
- A fix for a possible deadlock in DM bufio and a fix to re-check the
new buffer allocation watermark in the face of competing admin
changes to the 'max_cache_size_bytes' tunable.
- A couple DM core cleanups.
* tag 'for-4.12/dm-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm: (50 commits)
dm bufio: check new buffer allocation watermark every 30 seconds
dm bufio: avoid a possible ABBA deadlock
dm mpath: make it easier to detect unintended I/O request flushes
dm mpath: cleanup QUEUE_IF_NO_PATH bit manipulation by introducing assign_bit()
dm mpath: micro-optimize the hot path relative to MPATHF_QUEUE_IF_NO_PATH
dm: introduce enum dm_queue_mode to cleanup related code
dm mpath: verify __pg_init_all_paths locking assumptions at runtime
dm: verify suspend_locking assumptions at runtime
dm block manager: remove an unused argument from dm_block_manager_create()
dm rq: check blk_mq_register_dev() return value in dm_mq_init_request_queue()
dm mpath: delay requeuing while path initialization is in progress
dm mpath: avoid that path removal can trigger an infinite loop
dm mpath: split and rename activate_path() to prepare for its expanded use
dm ioctl: prevent stack leak in dm ioctl call
dm integrity: use previously calculated log2 of sectors_per_block
dm integrity: use hex2bin instead of open-coded variant
dm crypt: replace custom implementation of hex2bin()
dm crypt: remove obsolete references to per-CPU state
dm verity: switch to using asynchronous hash crypto API
dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
...
The DM integrity block size can now be 512, 1k, 2k or 4k. Using larger
blocks reduces metadata handling overhead. The block size can be
configured at table load time using the "block_size:<value>" option;
where <value> is expressed in bytes (defult is still 512 bytes).
It is safe to use larger block sizes with DM integrity, because the
DM integrity journal makes sure that the whole block is updated
atomically even if the underlying device doesn't support atomic writes
of that size (e.g. 4k block ontop of a 512b device).
Depends-on: 2859323e ("block: fix blk_integrity_register to use template's interval_exp if not 0")
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Some coding style changes.
Fix a bug that the array test_tag has insufficient size if the digest
size of internal has is bigger than the tag size.
The function __fls is undefined for zero argument, this patch fixes
undefined behavior if the user sets zero interleave_sectors.
Fix the limit of optional arguments to 8.
Don't allocate crypt_data on the stack to avoid a BUG with debug kernel.
Rename all optional argument names to have underscores rather than
dashes.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Commit 63c32ed4af ("dm raid: add raid4/5/6 journaling support") added
journal support to close the raid4/5/6 "write hole" -- in terms of
writethrough caching.
Introduce a "journal_mode" feature and use the new
r5c_journal_mode_set() API to add support for switching the journal
device's cache mode between write-through (the current default) and
write-back.
NOTE: If the journal device is not layered on resilent storage and it
fails, write-through mode will cause the "write hole" to reoccur. But
if the journal fails while in write-back mode it will cause data loss
for any dirty cache entries unless resilent storage is used for the
journal.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Commit 3a1c1ef2f ("dm raid: enhance status interface and fixup
takeover/raid0") added new table line arguments and introduced an
ordering flaw. The sequence of the raid10_copies and raid10_format
raid parameters got reversed which causes lvm2 userspace to fail by
falsely assuming a changed table line.
Sequence those 2 parameters as before so that old lvm2 can function
properly with new kernels by adjusting the table line output as
documented in Documentation/device-mapper/dm-raid.txt.
Also, add missing version 1.10.1 highlight to the documention.
Fixes: 3a1c1ef2f ("dm raid: enhance status interface and fixup takeover/raid0")
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
In recovery mode, we don't:
- replay the journal
- check checksums
- allow writes to the device
This mode can be used as a last resort for data recovery. The
motivation for recovery mode is that when there is a single error in the
journal, the user should not lose access to the whole device.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Add optional "sector_size" parameter that specifies encryption sector
size (atomic unit of block device encryption).
Parameter can be in range 512 - 4096 bytes and must be power of two.
For compatibility reasons, the maximal IO must fit into the page limit,
so the limit is set to the minimal page size possible (4096 bytes).
NOTE: this device cannot yet be handled by cryptsetup if this parameter
is set.
IV for the sector is calculated from the 512 bytes sector offset unless
the iv_large_sectors option is used.
Test script using dmsetup:
DEV="/dev/sdb"
DEV_SIZE=$(blockdev --getsz $DEV)
KEY="9c1185a5c5e9fc54612808977ee8f548b2258d31ddadef707ba62c166051b9e3cd0294c27515f2bccee924e8823ca6e124b8fc3167ed478bca702babe4e130ac"
BLOCK_SIZE=4096
# dmsetup create test_crypt --table "0 $DEV_SIZE crypt aes-xts-plain64 $KEY 0 $DEV 0 1 sector_size:$BLOCK_SIZE"
# dmsetup table --showkeys test_crypt
Signed-off-by: Milan Broz <gmazyland@gmail.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
For the new authenticated encryption we have to support generic composed
modes (combination of encryption algorithm and authenticator) because
this is how the kernel crypto API accesses such algorithms.
To simplify the interface, we accept an algorithm directly in crypto API
format. The new format is recognised by the "capi:" prefix. The
dmcrypt internal IV specification is the same as for the old format.
The crypto API cipher specifications format is:
capi:cipher_api_spec-ivmode[:ivopts]
Examples:
capi:cbc(aes)-essiv:sha256 (equivalent to old aes-cbc-essiv:sha256)
capi:xts(aes)-plain64 (equivalent to old aes-xts-plain64)
Examples of authenticated modes:
capi:gcm(aes)-random
capi:authenc(hmac(sha256),xts(aes))-random
capi:rfc7539(chacha20,poly1305)-random
Authenticated modes can only be configured using the new cipher format.
Note that this format allows user to specify arbitrary combinations that
can be insecure. (Policy decision is done in cryptsetup userspace.)
Authenticated encryption algorithms can be of two types, either native
modes (like GCM) that performs both encryption and authentication
internally, or composed modes where user can compose AEAD with separate
specification of encryption algorithm and authenticator.
For composed mode with HMAC (length-preserving encryption mode like an
XTS and HMAC as an authenticator) we have to calculate HMAC digest size
(the separate authentication key is the same size as the HMAC digest).
Introduce crypt_ctr_auth_cipher() to parse the crypto API string to get
HMAC algorithm and retrieve digest size from it.
Also, for HMAC composed mode we need to parse the crypto API string to
get the cipher mode nested in the specification. For native AEAD mode
(like GCM), we can use crypto_tfm_alg_name() API to get the cipher
specification.
Because the HMAC composed mode is not processed the same as the native
AEAD mode, the CRYPT_MODE_INTEGRITY_HMAC flag is no longer needed and
"hmac" specification for the table integrity argument is removed.
Signed-off-by: Milan Broz <gmazyland@gmail.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Allow the use of per-sector metadata, provided by the dm-integrity
module, for integrity protection and persistently stored per-sector
Initialization Vector (IV). The underlying device must support the
"DM-DIF-EXT-TAG" dm-integrity profile.
The per-bio integrity metadata is allocated by dm-crypt for every bio.
Example of low-level mapping table for various types of use:
DEV=/dev/sdb
SIZE=417792
# Additional HMAC with CBC-ESSIV, key is concatenated encryption key + HMAC key
SIZE_INT=389952
dmsetup create x --table "0 $SIZE_INT integrity $DEV 0 32 J 0"
dmsetup create y --table "0 $SIZE_INT crypt aes-cbc-essiv:sha256 \
11ff33c6fb942655efb3e30cf4c0fd95f5ef483afca72166c530ae26151dd83b \
00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff \
0 /dev/mapper/x 0 1 integrity:32:hmac(sha256)"
# AEAD (Authenticated Encryption with Additional Data) - GCM with random IVs
# GCM in kernel uses 96bits IV and we store 128bits auth tag (so 28 bytes metadata space)
SIZE_INT=393024
dmsetup create x --table "0 $SIZE_INT integrity $DEV 0 28 J 0"
dmsetup create y --table "0 $SIZE_INT crypt aes-gcm-random \
11ff33c6fb942655efb3e30cf4c0fd95f5ef483afca72166c530ae26151dd83b \
0 /dev/mapper/x 0 1 integrity:28:aead"
# Random IV only for XTS mode (no integrity protection but provides atomic random sector change)
SIZE_INT=401272
dmsetup create x --table "0 $SIZE_INT integrity $DEV 0 16 J 0"
dmsetup create y --table "0 $SIZE_INT crypt aes-xts-random \
11ff33c6fb942655efb3e30cf4c0fd95f5ef483afca72166c530ae26151dd83b \
0 /dev/mapper/x 0 1 integrity:16:none"
# Random IV with XTS + HMAC integrity protection
SIZE_INT=377656
dmsetup create x --table "0 $SIZE_INT integrity $DEV 0 48 J 0"
dmsetup create y --table "0 $SIZE_INT crypt aes-xts-random \
11ff33c6fb942655efb3e30cf4c0fd95f5ef483afca72166c530ae26151dd83b \
00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff \
0 /dev/mapper/x 0 1 integrity:48:hmac(sha256)"
Both AEAD and HMAC protection authenticates not only data but also
sector metadata.
HMAC protection is implemented through autenc wrapper (so it is
processed the same way as an authenticated mode).
In HMAC mode there are two keys (concatenated in dm-crypt mapping
table). First is the encryption key and the second is the key for
authentication (HMAC). (It is userspace decision if these keys are
independent or somehow derived.)
The sector request for AEAD/HMAC authenticated encryption looks like this:
|----- AAD -------|------ DATA -------|-- AUTH TAG --|
| (authenticated) | (auth+encryption) | |
| sector_LE | IV | sector in/out | tag in/out |
For writes, the integrity fields are calculated during AEAD encryption
of every sector and stored in bio integrity fields and sent to
underlying dm-integrity target for storage.
For reads, the integrity metadata is verified during AEAD decryption of
every sector (they are filled in by dm-integrity, but the integrity
fields are pre-allocated in dm-crypt).
There is also an experimental support in cryptsetup utility for more
friendly configuration (part of LUKS2 format).
Because the integrity fields are not valid on initial creation, the
device must be "formatted". This can be done by direct-io writes to the
device (e.g. dd in direct-io mode). For now, there is available trivial
tool to do this, see: https://github.com/mbroz/dm_int_tools
Signed-off-by: Milan Broz <gmazyland@gmail.com>
Signed-off-by: Ondrej Mosnacek <omosnacek@gmail.com>
Signed-off-by: Vashek Matyas <matyas@fi.muni.cz>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
The dm-integrity target emulates a block device that has additional
per-sector tags that can be used for storing integrity information.
A general problem with storing integrity tags with every sector is that
writing the sector and the integrity tag must be atomic - i.e. in case of
crash, either both sector and integrity tag or none of them is written.
To guarantee write atomicity the dm-integrity target uses a journal. It
writes sector data and integrity tags into a journal, commits the journal
and then copies the data and integrity tags to their respective location.
The dm-integrity target can be used with the dm-crypt target - in this
situation the dm-crypt target creates the integrity data and passes them
to the dm-integrity target via bio_integrity_payload attached to the bio.
In this mode, the dm-crypt and dm-integrity targets provide authenticated
disk encryption - if the attacker modifies the encrypted device, an I/O
error is returned instead of random data.
The dm-integrity target can also be used as a standalone target, in this
mode it calculates and verifies the integrity tag internally. In this
mode, the dm-integrity target can be used to detect silent data
corruption on the disk or in the I/O path.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Signed-off-by: Milan Broz <gmazyland@gmail.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Fix typos and add the following to the scripts/spelling.txt:
explictely||explicitly
Link: http://lkml.kernel.org/r/1481573103-11329-25-git-send-email-yamada.masahiro@socionext.com
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
If "metadata2" is provided as a table argument when creating/loading a
cache target a more compact metadata format, with separate dirty bits,
is used. "metadata2" improves speed of shutting down a cache target.
Signed-off-by: Joe Thornber <ejt@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Add md raid4/5/6 journaling support (upstream commit bac624f3f8 started
the implementation) which closes the write hole (i.e. non-atomic updates
to stripes) using a dedicated journal device.
Background:
raid4/5/6 stripes hold N data payloads per stripe plus one parity raid4/5
or two raid6 P/Q syndrome payloads in an in-memory stripe cache.
Parity or P/Q syndromes used to recover any data payloads in case of a disk
failure are calculated from the N data payloads and need to be updated on the
different component devices of the raid device. Those are non-atomic,
persistent updates. Hence a crash can cause failure to update all stripe
payloads persistently and thus cause data loss during stripe recovery.
This problem gets addressed by writing whole stripe cache entries (together with
journal metadata) to a persistent journal entry on a dedicated journal device.
Only if that journal entry is written successfully, the stripe cache entry is
updated on the component devices of the raid device (i.e. writethrough type).
In case of a crash, the entry can be recovered from the journal and be written
again thus ensuring consistent stripe payload suitable to data recovery.
Future dependencies:
once writeback caching being worked on to compensate for the throughput
implictions involved with writethrough overhead is supported with journaling
in upstream, an additional patch based on this one will support it in dm-raid.
Journal resilience related remarks:
because stripes are recovered from the journal in case of a crash, the
journal device better be resilient. Resilience becomes mandatory with
future writeback support, because loosing the working set in the log
means data loss as oposed to writethrough, were the loss of the
journal device 'only' reintroduces the write hole.
Fix comment on data offsets in parse_dev_params() and initialize
new_data_offset as well.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
This fix addresses the following 3 failure scenarios:
1) If a (transiently) inaccessible metadata device is being passed into the
constructor (e.g. a device tuple '254:4 254:5'), it is processed as if
'- -' was given. This erroneously results in a status table line containing
'- -', which mistakenly differs from what has been passed in. As a result,
userspace libdevmapper puts the device tuple seperate from the RAID device
thus not processing the dependencies properly.
2) False health status char 'A' instead of 'D' is emitted on the status
status info line for the meta/data device tuple in this metadata device
failure case.
3) If the metadata device is accessible when passed into the constructor
but the data device (partially) isn't, that leg may be set faulty by the
raid personality on access to the (partially) unavailable leg. Restore
tried in a second raid device resume on such failed leg (status char 'D')
fails after the (partial) leg returned.
Fixes for aforementioned failure scenarios:
- don't release passed in devices in the constructor thus allowing the
status table line to e.g. contain '254:4 254:5' rather than '- -'
- emit device status char 'D' rather than 'A' for the device tuple
with the failed metadata device on the status info line
- when attempting to restore faulty devices in a second resume, allow the
device hot remove function to succeed by setting the device to not in-sync
In case userspace intentionally passes '- -' into the constructor to avoid that
device tuple (e.g. to split off a raid1 leg temporarily for later re-addition),
the status table line will correctly show '- -' and the status info line will
provide a '-' device health character for the non-defined device tuple.
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
According to `man blockdev':
--getsize
Print device size (32-bit!) in sectors.
Deprecated in favor of the --getsz option.
...
--getsz
Get size in 512-byte sectors.
Hence, occurrences of `--getsize' should be replaced with `--getsz',
which this commit has achieved as follows:
$ cd "$repo"
$ git grep -l -e --getsz
Documentation/device-mapper/delay.txt
Documentation/device-mapper/dm-crypt.txt
Documentation/device-mapper/linear.txt
Documentation/device-mapper/log-writes.txt
Documentation/device-mapper/striped.txt
Documentation/device-mapper/switch.txt
$ cd Documentation/device-mapper
$ sed -i s/getsize/getsz/g *
Signed-off-by: Michael Witten <mfwitten@gmail.com>
Signed-off-by: Jiri Kosina <jkosina@suse.cz>
The kernel key service is a generic way to store keys for the use of
other subsystems. Currently there is no way to use kernel keys in dm-crypt.
This patch aims to fix that. Instead of key userspace may pass a key
description with preceding ':'. So message that constructs encryption
mapping now looks like this:
<cipher> [<key>|:<key_string>] <iv_offset> <dev_path> <start> [<#opt_params> <opt_params>]
where <key_string> is in format: <key_size>:<key_type>:<key_description>
Currently we only support two elementary key types: 'user' and 'logon'.
Keys may be loaded in dm-crypt either via <key_string> or using
classical method and pass the key in hex representation directly.
dm-crypt device initialised with a key passed in hex representation may be
replaced with key passed in key_string format and vice versa.
(Based on original work by Andrey Ryabinin)
Signed-off-by: Ondrej Kozina <okozina@redhat.com>
Reviewed-by: David Howells <dhowells@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
dm-raid 1.9.0 fails to activate existing RAID4/10 devices that have the
old superblock format (which does not have takeover/reshaping support
that was added via commit 33e53f0685).
Fix validation path for old superblocks by reverting to the old raid4
layout and basing checks on mddev->new_{level,layout,...} members in
super_init_validation().
Cc: stable@vger.kernel.org # 4.8
Signed-off-by: Heinz Mauelshagen <heinzm@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
Since commit 63a4cc2486, bio->bi_rw contains flags in the lower
portion and the op code in the higher portions. This means that
old code that relies on manually setting bi_rw is most likely
going to be broken. Instead of letting that brokeness linger,
rename the member, to force old and out-of-tree code to break
at compile time instead of at runtime.
No intended functional changes in this commit.
Signed-off-by: Jens Axboe <axboe@fb.com>
