Hi Kevin, thanks, that helps. Let me answer your last post directly first, because we now have concrete data for both the SCSI and BlockStorage paths. Regarding the ring size: agreed. We were deliberately conservative because we initially treated the shared region more like scarce driver memory. Your clarification changes that assumption. A substantially larger ring is clearly possible and may improve batching and headroom. However, our measurements also support your other point: increasing the ring would not fundamentally remove the bottleneck while the original SCSI requests remain page-sized. Using normal mounted-volume I/O on the same ~152 MiB file on our Intel test system, we measured: Initial request-by-request WRITE: 0.32–0.36 MB/s Shared-memory ring WRITE: 2.5–2.9 MB/s Shared-memory ring READ: 5.8–8.0 MB/s 1 GbE payload ceiling: ~110 MB/s The shared ring improved WRITE by roughly 7–9x. But almost all framework requests remained 4 KB: WRITE: 4 KB 37,238 16 KB 18 64 KB 0 READ: 4 KB 38,315 16 KB 2 64 KB 22 Our application-side coalescer reduced 92,026 original requests to 91,876 wire commands, only about 0.16%. By the time requests reach us they are already separate active SCSI tasks, so a larger ring can keep more requests in flight but cannot remove the per-task framework lifecycle. That gives the scale you asked for: on Intel, the current single-page behavior costs us roughly one to two orders of magnitude compared with normal 1 GbE NAS throughput. QD64, multiple ITTs, ImmediateData, larger Data-In PDUs and the shared ring all work, but they cannot compensate for paying the full SCSI/DriverKit lifecycle for almost every 4 KB request. On Apple Silicon the 16 KB page size reduces the request count by 4x, but the same limitation remains. For a NAS initiator, that is still not product-viable. So I agree that SCSIControllerDriverKit is probably the most useful path to fix first. If the kernel starts issuing larger requests while UserGetDataBuffer remains unchanged, our SCSI DEXT should require little architectural change. There is, however, one new result from the BlockStorage path that may be relevant. Our IOUserBlockStorageDevice implementation is essentially complete apart from one very small but critical missing bridge. We already have dynamic geometry from the real iSCSI session, RMB=0 / native fixed-disk presentation, /dev/diskN, publish/unpublish, READ/WRITE/FLUSH orchestration, queue depth, shared-memory App/DEXT transport, real iSCSI READ/WRITE, completion handling and PR / single-writer arbitration. The remaining problem is specifically this callback: DoAsyncReadWrite( bool isRead, uint32_t requestID, uint64_t dmaAddr, uint64_t size, uint64_t lba, uint64_t numOfBlocks, IOUserStorageOptions options) DriverKit 25.5 documents dmaAddr only as: DMA address of the data buffer Unfortunately, the documentation does not explain how a software-backed BlockStorage driver is supposed to access the data behind that DMA address from CPU context. We initially tested whether dmaAddr might be CPU-dereferenceable inside the DEXT. Runtime proved that assumption wrong. READ completion repeatedly crashed the DEXT at: memcpy(reinterpret_cast<void *>(dmaAddr), ...) with: EXC_BAD_ACCESS KERN_INVALID_ADDRESS After repeated IOUserServer crashes, macOS eventually panicked with: Driver IOUserServer(com.aviontex.iscsi...) has crashed too many times (reason 2:11) We stopped testing and inspected the actual Xcode 26.5 / DriverKit 25.5 SDK instead of making further assumptions. We checked IOUserBlockStorageDevice.iig, the generated header, the private StartDev interface, exported BlockStorageDeviceDriverKit symbols, the generic IODMACommand API and the wider DriverKit headers. We could not find any public equivalent of: GetDataBuffer(...) GetDMACommand(...) dmaAddr -> IOMemoryDescriptor dmaAddr -> IODMACommand Map/Resolve/LookupDMAAddress(...) This is particularly noticeable because other DriverKit families expose descriptors explicitly when CPU-side access is intended. DoAsyncUnmap() in the same BlockStorage class receives an IOMemoryDescriptor *, and SCSIControllerDriverKit explicitly provides UserGetDataBuffer(). IODMACommand::PerformOperation() initially looked promising. DriverKit documents it as a way to perform CPU access to a prepared DMA mapping, for example to/from a driver-allocated bounce buffer. But that method operates on a specific prepared IODMACommand instance. IOUserBlockStorageDevice::DoAsyncReadWrite() gives us only the resulting DMA address, not the IODMACommand or IOMemoryDescriptor that owns that mapping. Interestingly, our older experimental BlockStorage implementation had already identified exactly this gap. It contained a proposed: dmaAddr -> avx_descriptorForDMA() -> IOMemoryDescriptor path, but avx_descriptorForDMA() was deliberately left as a stub returning nullptr until a real BlockStorage-family API could be identified. So the BlockStorage path is now in a rather frustrating position: from our side it is almost finished, and the only missing connection is: framework-created DMA mapping | dmaAddr | ??? | CPU-accessible request bytes | shared ring | iSCSI transport We searched the public DriverKit 25.5 SDK for that bridge and cannot find one. So the remaining question is now very narrow: Is this absence intentional? Is IOUserBlockStorageDevice designed on the assumption that dmaAddr is handed to DMA-capable hardware, with no supported CPU-access path for a software-backed device? Or is there an intended BlockStorage-specific mechanism to access the already-prepared mapping that is not exposed or documented in the public SDK? If such a mechanism exists, the BlockStorage path may genuinely be only one small missing API connection away from working and would avoid the single-page SCSI limitation entirely. If it does not exist, then the picture is finally clear: SCSIControllerDriverKit: CPU-accessible request descriptor available, but I/O is currently page-bound. BlockStorageDeviceDriverKit: the desired block-I/O model is available, but READ/WRITE exposes only a DMA address with no documented CPU-accessible descriptor path. In that case, your comment that SCSIControllerDriverKit is the path most likely to be fixed first makes complete sense, and we would freeze the BlockStorage work rather than build another unsupported workaround around the DMA contract. Thanks again for helping us narrow this down. At this point the remaining BlockStorage issue is no longer a large architectural problem on our side, but literally this one missing DMA-to-CPU access bridge. Best regards, Torsten

Hi @lazarro! That’s exactly our goal. We want the native iSCSI path to be really fast. From my side it no longer feels like a question of if Apple will provide the missing descriptor support, but rather when. I’m hoping they play along because without that change this path simply won’t become what it needs to be. I’m keeping my fingers crossed for all of us ;) I don’t expect Kevin to be able to answer the timing question here in the forum anytime soon. We will probably both have to wait a bit and it might unfortunately take some more time. Best regards, Torsten

Hi Kevin, We now have the measurements we promised. My previous post was still based on calculations and on the assumption that the single-page SCSI path might remain viable if per-request overhead stayed low enough. On our Intel test system, that assumption does not hold. Functional result SCSIControllerDriverKit works functionally for a native software-backed iSCSI device. We validated an RMB=0 fixed disk backed by a real 4.29 TB Synology LUN with 512-byte blocks, native IOMedia and APFS mounting, verified READ and WRITE traffic, clean target creation and removal, QD64, bundled task intake, multiple iSCSI ITTs, 64 KB Data-In PDUs and ImmediateData. This is a mounted native macOS disk carrying real file I/O, not an INQUIRY-only probe. Measured performance The same approximately 152 MiB file was used throughout. Path / direction Elapsed time Effective rate Initial request-by-request path, WRITE 7-8 minutes 0.32-0.36 MB/s Shared-memory ring, WRITE 52.5-62 seconds 2.5-2.9 MB/s Shared-memory ring, READ 19.8-26.3 seconds 5.8-8.0 MB/s 1 GbE payload ceiling, context only ~1.4 seconds ~110 MB/s The ring improved representative WRITE performance by roughly 7-9x. That is a real gain, but a result of only a few MB/s is still not viable for a LAN-connected NAS block device. Why we built the ring The initial request-by-request path took seven to eight minutes for 152 MiB, so we built a substantial workaround to determine whether the App/DEXT handoff was the primary bottleneck. The DEXT now creates and shares a roughly 16 MB IOBufferMemoryDescriptor containing a request queue, a completion queue, 64 request slots, 64 completion slots and 64 payload slots of 256 KB each. The path supports QD64, bundled DriverKit intake, multiple ITTs, out-of-order completion, doorbells, completion kicks and ImmediateData. The data path is effectively: SCSI task -> UserGetDataBuffer -> request mapping -> shared staging slot -> userspace iSCSI -> completion ring -> READ copy-back -> individual framework completion This removed the old payload-sized UserClient transport and the QD1 bottleneck. It did not remove the framework lifecycle of each original SCSI task. What remained Despite advertising 256 KB through Block Limits VPD, representative epochs were still almost entirely 4 KB tasks: Task size WRITE READ exact 4 KB 37,238 38,315 exact 16 KB 18 2 exact 64 KB 0 22 exact 128/256 KB 0 0 other small 10 25 An application-side coalescer was byte-correct, but 92,026 original requests became 91,876 wire commands, a reduction of only 0.16%. The requests were already individually active rather than accumulating as a mergeable batch. The limiting granularity therefore originates above the ring. The single-page restriction is the blocker Every workaround still pays one complete framework lifecycle per page-sized task: callback, UserGetDataBuffer, descriptor and mapping ownership, data movement and individual completion. QD64 can overlap these lifecycles; it cannot remove them. For a 152 MiB transfer, the arithmetic is: Request size Request count 4 KB 38,912 16 KB 9,728 64 KB 2,432 256 KB 608 We have not yet measured Apple Silicon, but its 16 KB page does not change the verdict. Even assuming ideal 16 KB tasks, the same file still requires almost ten thousand complete framework lifecycles. A larger system page reduces the count but does not remove the page-bound architecture. Measuring Silicon would refine the number, not the conclusion, because the constraint we need removed is single-page, not 4 KB specifically. What we actually need A self-created IOBufferMemoryDescriptor only describes DEXT-owned staging memory. It is not the descriptor of the current DoAsyncReadWrite request, and it does not make dmaAddr a documented CPU-accessible pointer. For a software-backed block device, we need the request-scoped memory object: a documented, CPU-accessible, multi-page descriptor with defined length, direction, mapping, synchronization, ownership and asynchronous lifetime. DoAsyncUnmap already carries an IOMemoryDescriptor in the same IOUserBlockStorageDevice class, while DoAsyncReadWrite exposes only dmaAddr. Such an API would not guarantee line-rate performance. It would remove the artificial requirement that a software network block device be forced through an IODMACommand/DART-dependent single-page workaround. FSKit We considered the FSKit/raw-DiskImage approach seriously, but for a native iSCSI initiator it is not an equivalent solution. The device it produces is a raw disk image, not a SCSI device. It exposes disk-image block semantics, so the behavior our initiator implements at the SCSI transport level has nowhere to live: Persistent Reservations, sense data, Unit Attention, proper SCSI error reporting, task management and task ordering. FSKit models a filesystem, and the disk-image indirection repurposes it as a byte-backing store, so the result is a block device layered over a file rather than a native one. It is also fragile as a product foundation. The block device exists only while the FSKit mount and the hdiutil attachment stay alive, so an extension crash, an app update or an unclean teardown takes the device, and anything mounted on it, with it. For byte movement it may well be faster than the current single-page SCSI path, and we are not disputing that. It is a different, non-native architecture that discards the SCSI device model our product is built on. SCSIControllerDriverKit is therefore functionally viable for this use case, but the current page-bound request path is not product-viable for us. The shared ring makes the workaround substantially better; it cannot remove the reason the workaround exists. Without the single-page restriction, performance is an engineering problem. With it, performance is an API-architecture problem. Does this measured result match your technical view of the remaining constraint? If it does, we are left waiting for a kernel fix: a request-scoped, multi-page descriptor. For a software device the clean home for it is BlockStorageDeviceDriverKit, which sidesteps IODMACommand entirely; without a multi-page path there or in SCSIControllerDriverKit, every path we have stays pinned to one page per request, 4 KB on Intel and 16 KB on Apple Silicon. Such a change would ship only in a future macOS, which puts Intel-based Macs permanently outside this feature. That is a trade-off we can live with. Best regards, Torsten

We ran the whole thing through on paper first. Everything below is calculation and assumption, not measurement. The real numbers come once we measure, but the calculations already let us make a call. Short version: we build on SCSI and validate it. FSKit stays as a last resort. Extending BlockStorage with a descriptor path looks like the clean fix. 1. SCSI DEXT On the single-page limit we did the math instead of guessing. Assuming 16 KB per request on Apple Silicon, a 20 GB transfer comes to 1,310,720 requests (on Intel at 4 KB it would be 5,242,880). Whether that becomes a problem depends on per-request latency. Working it through, in theory it looks like this: Latency per request 20 GB @ 16 KB Effective throughput 10 us 13 s 1.5 GB/s 50 us 66 s 305 MB/s 100 us 131 s 153 MB/s 140 us 183 s 110 MB/s <- 1 GbE line 200 us 262 s 76 MB/s 500 us 11 min 31 MB/s If those numbers hold, the single-page limit stays acceptable up to roughly 140 us per request, which keeps us at or above the 1 GbE ceiling of about 110 MB/s. Our target market of SOHO and home NAS runs mostly on 1 GbE and Wi-Fi anyway, where real-world throughput sits at or below that line. And if there is a way to lift the single-page constraint, that only improves the picture. 2. FSKit This is the option you raised, and on raw speed you are probably right: it avoids the DMA path completely. We took the suggestion seriously and ran the numbers. Since FSKit has no fixed transfer size like the single-page SCSI path, the only thing we can really compare is request count. Assuming cluster-sized I/O, a 20 GB transfer looks like this: 20 GB transfer, requests by I/O size (assumption) SCSI single-page 16 KB 1,310,720 requests FSKit 64 KB 327,680 requests (4x fewer) FSKit 128 KB 163,840 requests (8x fewer) FSKit 1 MB 20,480 requests (64x fewer) On request count FSKit in theory clearly wins. The trade-off is higher per-request overhead through the VFS layer, so which approach is faster in the end we cannot say without measuring both. What makes FSKit unattractive for us is not throughput, but what it costs at the layer above it. It hands us file offsets instead of SCSI semantics. Reservation handling, sense data, command ordering and error semantics would all have to be rebuilt on top of a layer that no longer speaks SCSI. And more important, our own in-house iSCSI protocol extensions (iSCSI-over-TLS or iSCSI-via-Remote) are built on native iSCSI. A SCSI or BlockStorage DEXT keeps that native layer underneath them. FSKit replaces it, and we cannot yet say what that does to those extensions, but building on a file abstraction instead of the native wire is a risk we would rather not take. If nothing else works we would have to, but it would set our development back considerably. 3. Extending BlockStorage with a descriptor path This would be additive and entitlement-gated. Leaving architecture, design, conception, testing and review aside completely and looking purely at the implementation effort, our estimate is less than a week of native code with no impact, since it is an extension and not a change. The reasoning: DoAsyncUnmap already carries an IOMemoryDescriptor across the DriverKit boundary in the same class, while DoAsyncReadWrite carries only a bare address one method away. So the plumbing to pass a descriptor already exists in the family. The descriptor also has to stay alive in the request until CompleteIO regardless, so it looks reachable via request ID. More broadly, this looks like a gap and not a missing capability. Other DriverKit families hand the dext a real IOMemoryDescriptor for their data path. Even SCSIController does, through UserGetDataBuffer, which is exactly what we are relying on. BlockStorage is the one that does not: its DoAsyncReadWrite passes only a bare address, with no descriptor equivalent anywhere in the class. And storage is exactly the area where getting the bytes wrong costs data. Our conclusion Of the three paths, only the descriptor extension actually solves this rather than working around it. SCSI seems to work but stays capped. FSKit works but breaks the native semantics. The descriptor path is the only one that removes the underlying reason this device class still needs a kext. Laid out plainly, this is why it looks like the right call to us, and why we think it is the sensible one for the platform too: Apple users finally get a native iSCSI solution. The missing piece is only the descriptor path in the family. It makes the DMA problem moot instead of fixing it. No IODMACommand, no segments, no DART, no single-page limit, and no impact on the general DMA path that every driver depends on. It moves a whole device class off kexts. iSCSI on macOS means a kext today, and on Apple Silicon that means Reduced Security. The descriptor path removes the technical reason for that, for everyone, not just us. It strengthens platform security. Nobody has to weaken their machine to reach their own storage. It gives people a reason to move to Apple Silicon. A fix lands from macOS 27 onward, which is Apple Silicon only, so native high-throughput iSCSI becomes a capability of the newer machines rather than a reason to stay on a kext. Realistically a change like this would take maybe six to nine months through the normal release cycle. That would put it in macOS 27 or later. Older Intel machines would keep running on the single-page SCSI path on Tahoe, without regression. Nobody would lose a working setup. That is a bigger outcome than one product, and that is why we keep coming back to it. We are building on SCSI now and will test whether it holds up. If it is good enough for SOHO and home NAS traffic, we can live with it, and none of this blocks us. We would of course like to know how long something like this might take, but we know that is not a question you can answer. The one thing we are asking for is your technical read. Does this match how you see the constraints today? And is there a fundamental blocker in this approach that we are missing? Best regards, Torsten

Hi Kevin, Both bugs are filed: FB23814013 FB23814092 If you need more detail on either, or want anything in a different form, just say the word and I will add it. Filing them was the easy part. We spent most of yesterday testing, so the data was already sitting there. Thanks again for requesting SCSIControllerDriverKit on our behalf. That was not something I expected and it is appreciated. We will keep testing and give you an update once we know more, probably sometime next week. Best regards & enjoy your weekend!!! Torsten

Hi Kevin, I'm afraid I have to disagree with one detail. Please give me two seconds to enjoy the moment. You know, the Big Five for Life: getting married, having children, planting a tree, building a house and correcting an Apple DTS engineer. :-) All right, moment over. Seriously, though: thank you for revisiting this so openly, investigating the alternatives and trying to move the underlying issues forward internally. Your revised overall conclusion appears to be correct: SCSIControllerDriverKit is viable for this use case, at least on the system we have now tested. One detail may need clarification, though: the descriptor API path. In the public DriverKit 25.5 headers, UserMapHBAData does not return a memory descriptor. Its only output is the unique controller task identifier: UserMapHBAData( uint32_t *uniqueTaskID ); The descriptor itself is retrieved later through: UserGetDataBuffer( targetID, controllerTaskID, &buffer ); when called from UserProcessParallelTask, using the request's fControllerTaskIdentifier. Perhaps this is the mechanism you meant. UserMapHBAData establishes the task identity, and UserGetDataBuffer subsequently retrieves the task's IOBufferMemoryDescriptor. As we are still testing, we would like to keep the conclusions narrow and clearly distinguish between what we have measured and what remains open. Current test configuration Hardware: Intel Mac macOS: 26.5.2 Xcode: 26.5 DriverKit SDK: 25.5 IOClass: IOUserSCSIParallelInterfaceController IOProviderClass: IOUserResources There is no PCI or Thunderbolt provider. The signed DEXT does not request the PCI transport entitlement. The extension was installed and activated successfully. The controller became registered, matched and active. The following lifecycle and reporting steps completed: UserInitializeController UserReportHBAConstraints UserStartController UserInitializeTargetForID The framework then submitted real SCSI commands through UserProcessParallelTask. Inside those callbacks: UserGetDataBuffer -> kIOReturnSuccess -> non-null IOBufferMemoryDescriptor GetAddressRange -> kIOReturnSuccess -> nonzero DEXT-local address -> expected transfer length We initially performed a descriptor-only probe without touching the buffer. That succeeded. We then performed a strictly bounded standard INQUIRY write using the address returned by GetAddressRange. The observed sequence was: INQUIRY allocation length 6 -> wrote 6 bytes -> completed GOOD INQUIRY allocation length 36 -> wrote 36 bytes -> completed GOOD macOS consumed the initial six bytes and subsequently requested the complete 36-byte INQUIRY response. The resulting IORegistry properties contained the exact data written by the DEXT: Vendor: AVIONTEX Product: iSCSI4NAS VIRT Revision: 0001 The native storage stack then advanced through: IOSCSIParallelInterfaceDevice IOSCSITargetDevice IOSCSIHierarchicalLogicalUnit IOSCSIPeripheralDeviceType00 IOBlockStorageServices IOBlockStorageDriver It is now issuing subsequent discovery commands, including READ CAPACITY(10). There were no panics and no DEXT crashes. The DriverKit crash counter remained at zero. For this specific configuration, we have therefore demonstrated: IOUserResources -> IOUserSCSIParallelInterfaceController -> UserProcessParallelTask -> UserGetDataBuffer -> IOBufferMemoryDescriptor -> GetAddressRange -> bounded byte write -> successful SCSI completion -> data consumed and interpreted by the macOS storage stack This confirms your revised conclusion that the SCSI path is possible today. It also shows that the original fBufferIOVMAddr limitation does not prevent the DEXT from accessing the request data, because UserGetDataBuffer provides a separate descriptor-based path. We still do not use or dereference fBufferIOVMAddr. Single-page limitation Your warning about the single-page DMA limitation remains open. Our successful transfers were only: 6 bytes 36 bytes These results therefore neither confirm nor disprove a single-page limitation. We have not yet tested a cross-page request, Scatter/Gather I/O or real READ/WRITE commands. Could you clarify the exact configuration Apple currently recommends to enforce the single-page restriction? In particular, should a virtual HBA use: maximum segment count read = 1 maximum segment count write = 1 maximum segment byte count read = runtime system page size maximum segment byte count write = runtime system page size maxTransferSize = runtime system page size Should any additional alignment constraint be applied? And should "one page" always use the runtime system page size rather than a fixed 4096-byte value, particularly when validating the same implementation across Intel and Apple Silicon? BlockStorageDeviceDriverKit and FSKit We also agree with your assessment of BlockStorageDeviceDriverKit. Its current DoAsyncReadWrite interface exposes only the DMA address and does not provide an equivalent to UserGetDataBuffer. We will file the two requested Feedback Assistant reports: A descriptor-only read/write path for BlockStorageDeviceDriverKit A documented descriptor-only, multi-page I/O path for SCSIControllerDriverKit We will post both feedback numbers here once they have been submitted. We will also retain the FSKit/raw-DiskImage design as a fallback and potential performance comparison. For now, however, the SCSI path has progressed far enough that we would like to finish validating it before changing architectures. Scope For completeness, the results above currently apply only to: Intel macOS 26.5.2 DriverKit 25.5 development signing Developer Mode SIP disabled The following remain separate validation steps: Apple Silicon SIP-enabled standard security distribution provisioning multi-page I/O real READ/WRITE traffic long-running stability This is an early status report, not a final result. We will keep testing and keep you posted as we go, including the feedback numbers once both reports are filed. One more thing, and I mean it. When we started, we did not see the SCSI option at all. In hindsight we were probably too fixated on BlockStorageDeviceDriverKit, because from where we sit that is the more logical family for this product. It took both the entitlement process and your pointers to send us back to the SCSI DEXT and look at it properly. So thank you again. Your correction and the pointer back toward the SCSI path appear to have saved the architecture.

Thanks Kevin, that was fast and clear, and I appreciate you taking the time to look at it. I really hope you are wrong, and not just for selfish reasons. If neither storage family can carry a network backed block device, then there is no DEXT path to iSCSI on macOS at all. Not for us, not for anyone. It stays a kext, and on Apple Silicon that means telling users to drop to Reduced Security to reach their own storage. Nobody should have to weaken their machine to mount a disk. As a product that is simply not shippable, so for us that road ends there. The selfish reason is easier to explain: three years of my life are in this thing. So you can imagine I read your reply twice, went for a walk, and read it a third time. Right now the outcome is either "we built something that cannot exist" or "the DTS engineer is wrong about one detail". I know which one I am betting on, though I admit the odds are not in my favour. No argument from me either way, you know this stack better than I ever will. But I would rather find out than debate it. We are going to test this thoroughly now and see what actually happens, and as soon as I know more I will come back with the details. Thanks again.