Memory-Mapping Operations Within Transactions

It is perfectly legal to execute memory-mapping operations (including mmap() and shmat) within a lock-based critical section, and, at least in principle, from within an RCU read-side critical section. What happens when you attempt to execute such an operation from within a transaction? More to the point, what happens if the memory region being remapped contains some variables participating in the current thread's transaction? And what if this memory region contains variables particpating in some other thread's transaction?

It should not be necessary to consider cases where the TM system's metadata is remapped, as most locking primitives do not define the outcome of remapping the lock variables themselves.

Here are some options available to TM:

  1. Memory remapping is illegal within a transaction, and will result in all enclosing transactions being aborted. This does simplify things somewhat, but also requires that TM interoperate with synchronization primitives that do tolerate remapping from within their critical sections.
  2. Memory remapping is illegal within a transaction, and the compiler is enlisted to enforce this prohibition.
  3. Memory mapping is legal within a transaction, but aborts all other transactions having variables in the region mapped over.
  4. Memory mapping is legal within a transaction, but the mapping operation will fail if the region being mapped overlaps with the current transaction's footprint.
  5. All memory-mapping operations, whether within or outside a transaction, check the region being mapped against the memory footprint of all transactions in the system. If there is overlap, then the memory-mapping operation fails.
  6. The effect of memory-mapping operations that overlap the memory footprint of any transaction in the system is determined by the TM conflict manager, which might dynamically determine whether to fail the memory-mapping operation or abort any conflicting transactions.

It is important to note that munmap() is also a memory-mapping primitive, but that this primitive leaves the relevant region of memory unmapped, which could have additional interesting implications. (Thanks to Josh Triplett for pointing this primitive out.)