Post

Replies

Boosts

Views

Activity

Reply to System Panic with IOUserSCSIParallelInterfaceController during Dispatch Queue Configuration
Hi Kevin, Thank you for your valuable insights. Following your advice, we have refactored our driver to use a "wrapper" pattern where UserProcessBundledParallelTasks serves as a high-performance entry point that forwards commands to our core dispatch logic. To eliminate potential race conditions, we have also moved our interrupt handling to the Default Dispatch Queue using kIOServiceDefaultQueueName. This ensures that command submission and completion are strictly serialized. Here are the key implementation details showing how we unified the dispatch logic: 1. Legacy Entry Point (Single Task): We extracted our core logic into a helper method, DispatchTaskInternal, passing 0xFFFF as a placeholder for the Slot Index. kern_return_t MyDriver::UserProcessParallelTask_Impl( SCSIUserParallelTask parallelRequest, uint32_t *response, OSAction *completion) { // Forward to unified internal dispatcher with no slot index (Legacy Mode) return DispatchTaskInternal(parallelRequest, response, completion, 0xFFFF); } 2. Bundled Submission Path: In our UserProcessBundledParallelTasks implementation, we iterate through the indices, create a local object copy of the request (as you suggested), and call the same internal dispatcher. void MyDriver::UserProcessBundledParallelTasks_Impl( const uint16_t parallelRequestSlotIndices[32], uint16_t parallelRequestSlotIndicesCount, OSAction * completion) { for (uint16_t i = 0; i < parallelRequestSlotIndicesCount; i++) { uint16_t slotIndex = parallelRequestSlotIndices[i]; const SCSIUserParallelTask& sharedReq = ivars->fCommandBuffers[slotIndex]; // Create a local copy to utilize existing processing logic SCSIUserParallelTask localReq = sharedReq; uint32_t response; // DispatchTaskInternal performs a completion->retain() for EVERY command, // ensuring the action stays alive regardless of the bundled count. DispatchTaskInternal(localReq, &response, completion, slotIndex); } } 3. Unified Asynchronous Completion Path: Regardless of the mode, we now use the stable ParallelTaskCompletion within the interrupt handler (ISR), followed by the corresponding release(). // Inside the Asynchronous Interrupt Handler (serialized on Default Queue) SCSIUserParallelResponse taskResponse = {}; // ... Populate status, TaskID, and Version ... // Signal completion to the framework ParallelTaskCompletion(completion, taskResponse); // Balance the reference count performed during submission completion->release(); Observations and Current Bottleneck: If we intercept a command (like TEST UNIT READY) and call the completion logic "synchronously" within the UserProcessBundledParallelTasks scope, the system works perfectly and immediately issues the next command (e.g., INQUIRY). As soon as the completion is triggered from the asynchronous interrupt path—even though it is serialized on the same Default Queue—invoking ParallelTaskCompletion (or BundledParallelTaskCompletion) causes the DEXT to crash immediately (Corpse / Address size fault). Questions: Since we have ensured an independent retain/release cycle for every command and used a serialized queue, why does the OSAction handle appear to become invalid once it enters an asynchronous callback context in Bundled mode? Is there a hidden thread-context or lifecycle restriction on the OSAction provided during a bundled call that differs from legacy mode? We have verified that the legacy mode remains 100% stable under the same serialized queue configuration. We would appreciate any further guidance on why the "Sync succeeds, Async crashes" behavior persists in the high-performance path. Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Jan ’26
Reply to System Panic with IOUserSCSIParallelInterfaceController during Dispatch Queue Configuration
Hi Kevin, Thank you for your previous guidance. We have shifted our driver architecture to the UserProcessBundledParallelTasks model with Shared Command/Response Buffers to optimize I/O performance. But, we are encountering a persistent Kernel Panic / DEXT Crash (Corpse) immediately after switching to the Bundled mode, preventing the target discovery and initialization sequence from completing, while the legacy mode (UserProcessParallelTask) is rock-solid and stable. Implementation Details: Memory Mapping: Successfully implemented UserMapBundledParallelTaskCommandAndResponseBuffers. DEXT correctly obtains the virtual addresses for both buffers. Dispatching: Inside UserProcessBundledParallelTasks, we iterate through the parallelRequestSlotIndices, reading from the shared command buffer and dispatching tasks to the hardware. Completion: Upon hardware completion, we populate the shared response buffer in the asynchronous path (ISR/Poll) and invoke BundledParallelTaskCompletion with the corresponding OSAction. The Critical Race Condition Observed: Due to the very low latency of our hardware for specific commands (e.g., TEST UNIT READY). Logs reveal a severe timing conflict: The interrupt handler (Completion path) is triggered and successfully invokes BundledParallelTaskCompletion, which returns normally. Crucially, at this exact microsecond, the original UserProcessBundledParallelTasks call (Submission path) has not yet finished its loop or returned to the system. Immediately after both paths eventually return, the DEXT process crashes (Corpse), subsequently triggering a Kernel Panic. Troubleshooting Steps Taken (Issue Persists): Memory Protection: Clamped Sense Data to exactly 18 bytes (SCSI fixed format) to prevent any potential buffer overflows in the shared response buffer. Array Sizing: Ensured that the index array passed to BundledParallelTaskCompletion is a fixed-size uint16_t[32] to align with the .iig declaration and ensure RPC serialization safety. Reference Counting: Tested both calling and omitting release() on the OSAction in the completion path. The crash occurs regardless of manual release. Memory Barriers: Implemented __atomic_thread_fence(__ATOMIC_SEQ_CST) to ensure shared buffer writes are visible before signaling completion. Questions: In Bundled mode, does DriverKit support a scenario where the Completion RPC returns before the Submission RPC has finished its execution? Does this cause an internal state-machine conflict within IOUserSCSIParallelInterfaceController? What is the exact ownership model for the OSAction * action in Bundled mode? Is the action "consumed" by BundledParallelTaskCompletion, or is the driver still responsible for retain/release management as in legacy mode? Any advice on how to resolve this would be very helpful. Thanks! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Jan ’26
Reply to [DriverKit SCSI] SCSI probe stalls for Target ID > 0 with IOUserSCSIParallelInterfaceController
Hi Kevin, Thank you very much for your guidance. I am happy to report that the issue has been resolved. I would like to share the root cause and the final solution here in case it helps other developers facing similar issues. As you suspected, it was indeed a bug in our code regarding capability reporting. Although we were successfully calling AsyncCreateTargetForID(1), the macOS SCSI stack was silently aborting the probe sequence after the initial TEST UNIT READY command. The issue was located in our implementation of UserReportHighestSupportedDeviceID. We were mistakenly hardcoding the return value to 0: // Buggy implementation kern_return_t IMPL(DRV_MAIN_CLASS_NAME, UserReportHighestSupportedDeviceID) { *id = 0; // Reported that the HBA only supports ID 0 return kIOReturnSuccess; } Because the HBA reported a maximum supported ID of 0, the macOS SCSI Family treated Target 1 as an "out-of-bounds" device. While the object was instantiated and visible in ioreg, the kernel refused to Register or Match it. This explained why Target 1 remained in a !registered state and never received an INQUIRY command. We updated the method to return the correct maximum ID supported by our controller (63 to support up to 64 targets): // Corrected implementation kern_return_t IMPL(DRV_MAIN_CLASS_NAME, UserReportHighestSupportedDeviceID) { *id = 63; // Support for IDs 0 through 63 return kIOReturnSuccess; } Immediately after this change, Target 1 moved to a registered, matched state in the I/O Registry. The SCSI stack successfully issued the INQUIRY command, and diskutil list now correctly identifies both RAID volumes. Thank you again! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Jan ’26
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, Just a quick update: I have uploaded the source code (SetupHBAConstraints_Workaround.cpp) to the bug report (FB21256805). This file confirms that we are indeed defining all the required keys (including kIOMaximumSegmentByteCount) before calling UserReportHBAConstraints, and it also demonstrates the SetProperties workaround we implemented to bypass the issue. Best regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Dec ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, I have filed the bug report as requested. The Feedback ID is FB21256805. I've included the IORegistry dumps showing the missing keys when using the standard API versus the successful injection using the SetProperties workaround. To answer your specific question: "Are you sure you're setting all of the required key set, particularly 'kIOMaximumSegmentByteCountWriteKey'?" Yes. We confirmed that kIOMaximumSegmentByteCountWriteKey was correctly set to 65536 (64 KB), and kIOMaximumSegmentCountWriteKey was set to 129. Thanks again for your help in narrowing this down. Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Dec ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, Apologies, but it seems our previous celebration was premature. I must retract the conclusion from my last post regarding the issue being resolved. Here is the current situation: While our fixes regarding maxTransferSize (1MB) and SGL splitting have indeed achieved a 100% pass rate for CLI tools (cp and fio) across all file sizes, we have discovered that Finder still fails with Error -36 during file copies. Based on our latest findings, I would appreciate your insights on the following: CLI (cp, fio): Read/Write operations are 100% successful, even with large files (e.g., 4GB). Finder: Copying "Pre-existing files" (Cold Data): Success for files < 64MB. Copying "Freshly created files" (Dirty Data/Hot Cache): Immediate failure (Error -36) for files > 1MB. We suspected that "freshly created files" were highly fragmented in memory, causing the DEXT to construct a Scatter/Gather List (SGL) that exceeded our hardware limit (129 Segments). To verify this, we implemented "Trap Logs" inside the DEXT to monitor the SGCount. The logs proved this hypothesis wrong. Finder is not sending requests that exceed the limit. At the exact moment Finder reports the error, the DEXT is receiving a series of extremely small and clean requests, not fragmented large I/O: Request sizes are mostly 4KB (0x1000) or 16KB (0x4000). The SGCount for each request is only 0 or 1. This is far below our hardware limit of 129. This leaves us with a confusing contradiction: CLI (fio/cp): Sends larger chunks (e.g., 512KB), SGCount = 1 (likely due to contiguous allocation by fio). Result: Success. Finder: Sends tiny chunks (4KB/16KB), SGCount = 1. Result: Failure (Error -36). To rule out configuration variables, we have set the DEXT's UserReportHBAConstraints to match our stable Legacy KEXT exactly: IOMaximumByteCount = 512 KB (0x80000) IOMaximumSegmentCount = 129 Since the logs prove that Finder's requests fully comply with our reported Constraints, why would DriverKit or macOS determine a failure? Does Finder's I/O behavior trigger specific SCSI command sequences or timing issues that CLI tools do not? We observed that Finder issues multiple TEST UNIT READY and SYNCHRONIZE CACHE commands immediately before the write failure. Does this imply that some error handling mechanism is being triggered prematurely? Thank you again for your time and assistance. Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to Archive Fails: Conflict with Existing KEXT Developer ID Certificate
Hi everyone, After extensive testing and verification, we have finally resolved the persistent DriverKit archiving and signing errors. The core issue lies in the fact that Xcode enforces distinctly different certificate requirements for DriverKit targets during the "Build Phase" versus the "Distribution Phase." We are sharing our successful experience to help other developers facing the same trap. 💡 Core Conclusion Do not attempt to configure the Distribution Profile directly in Xcode’s Target settings, as this will cause the build to fail. The correct workflow is: Use "Development Certificates (Auto)" during the Build/Archive phase, and only swap in the "Distribution Certificates (Manual)" during the Organizer Distribution phase. 🔍 Root Cause Analysis The DriverKit Linker mandates linking against the Development SDK during the build phase. If you force a manual Distribution Profile in Xcode’s "Signing & Capabilities" tab, Xcode will fail because it cannot find a matching development certificate (often resulting in errors like Signing Certificate: None), making it impossible to generate an Archive. Step-by-Step Guide Phase 1: Xcode Project Settings Go to the Signing & Capabilities tab of your DEXT Target. Check "Automatically manage signing". Ensure Xcode generates a Team Provisioning Profile (Development) without errors. Note: This step is crucial to satisfy the compiler's requirement for the Development SDK. Phase 2: Generating the Archive Select Product -> Archive. Since you are using Auto Signing (Development), the build should complete successfully and appear in the Organizer. Phase 3: Organizer Distribution In the Organizer, click Distribute App -> Direct Distribution (or your preferred distribution method). When asked for the signing method, you select "Manually manage signing". In the Profile selection screen: App Target: Select the corresponding App Distribution Profile. DEXT Target: Manually select the "DriverKit Distribution Profile" you created in the Developer Portal. 🛠 Verification After exporting the App, use the following command to check the DEXT entitlements: codesign -d --entitlements :- "Path/To/Your.app/Contents/Library/SystemExtensions/com.your.driver.dext" Success Indicators: It includes your required hardware entitlements (e.g., com.apple.developer.driverkit.family.scsicontroller). It does NOT include com.apple.security.get-task-allow. (The absence of this flag confirms that the debug marker has been removed and successfully replaced with a production signature). Hope this strategy saves everyone some valuable time!
Nov ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, Regarding your previous analysis of the KEXT configuration, specifically where you pointed out the IOMaximumByteCount is actually 0x80000 (512KB) in this post. We observed the runtime logs of the legacy KEXT, and the results align perfectly with your observation. Even when copying a 2GB file, the KEXT actually splits the request into 512KB (0x80000) chunks per transfer. [ProcessParallelTask] [KEXT_CHECK] WRITE (0x2a) | T:0 L:0 | Size: 524288 bytes (512 KB) This confirms that our current 1MB limit in the DEXT actually exceeds the behavior of the legacy driver. Since the current setting has passed all data integrity verifications, we consider the transfer size issue resolved and will not pursue larger transfer sizes at this stage. As per your recommendation, we are currently implementing the UserProcessBundledParallelTasks architecture to strictly follow DriverKit best practices and optimize performance (it is currently causing the DEXT to crash, so we are in the middle of debugging it). Thanks again for your precise insights! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, The data corruption issue has been resolved! Special thanks for asking the key question: "So what's IS there?" This prompted us to shift our focus from high-level hash verification to inspecting the raw bytes written to the disk. This investigation revealed that the root cause was not related to cache flushing, but rather a hardware limitation regarding single transfer lengths. Here is a summary of our findings and the solution: By using a Python script to verify the data on disk byte-by-byte, we discovered: When macOS coalesced writes into 2MB chunks (as we had previously set maxTransferSize to 128MB), data corruption consistently began exactly at Offset 1MB + 16KB within the command (manifesting as 0x00 or garbage data). The LSI 3108 controller/firmware seems cannot correctly handle a single SCSI command with a data length exceeding 1MB in the DEXT environment. We implemented a two-layer fix: In UserGetDMASpecification, we explicitly set maxTransferSize to 1MB (1,048,576 bytes). This forces macOS to split large I/O requests into smaller chunks that the hardware can safely digest. To align with SCSI best practices and ensure maximum stability, we implemented logic within the driver to further split these 1MB buffers into multiple 64KB SGL Segments when populating hardware descriptors. With these fixes applied, all previously failing test scenarios (including cp + sync and fio, with file sizes ranging from 256MB up to 5GB) now PASS 100%. The checksums match perfectly. Thanks again for your guidance — it pointed us directly to the core issue! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, TL;DR (Core Issues) 100% Reproducible: We used two completely independent test methods (cp + sync and fio buffered mode), and both consistently reproduced the data corruption issue under Buffered I/O. Clear Failure Threshold: Both tests point to the exact same failure boundary: file sizes 128MB (inclusive) and below pass, but 256MB (inclusive) and above consistently fail. Key Clue Confirmed: The fio test again reported the exact same error as our initial finding: data corruption starts at a very precise offset 1048576 (1MB) location. You mentioned last time that this offset was "odd"; now we can confirm this is not a coincidence, but a core characteristic of the problem. Test 1: System-Level File Copy Integrity Test (cp + sync) This test aims to simulate standard user operations to rule out tool-specific issues. Methodology: We used a shell script to automate cp for file copying, used sync to force the system to flush I/O caches, and finally compared the SHA256 hashes of the source and destination files. Results: 128MB File: Success. Hashes match perfectly. 256MB File: Failure. Hash mismatch, confirming data corruption during the copy process. Conclusion: This proves that the issue lies within the system I/O stack, and our DEXT has a defect when handling standard buffered writes. Test 2: fio Buffered I/O Integrity Test This test leverages fio's robust verification capabilities to obtain lower-level error details. Settings & Expected Behavior Settings (buffered_integrity_test.fio): We removed direct=1 to force fio to use the OS file cache, consistent with the behavior of cp. We set do_verify=1 and verify=sha256 to require data integrity checks after I/O operations. fio Mechanism: The verification process consists of two independent phases (it does not verify-as-it-writes): Write Phase: fio writes the entire 256MB data (each 1MB block carries a unique header for verification) to the file. Verify Phase: After writing concludes, fio reads back the entire 256MB starting from the beginning (offset 0), checking the headers block by block. Test Results The results match the cp test exactly: 128MB passes, 256MB fails. During the 256MB failure, we received the following error message: [*] Testing file size: 256M ... verify: bad header ... at file ... offset 1048576, length 1048576 fio: ... error=Illegal byte sequence ... [!!!] TEST FAILED at file size: 256M Error Message Analysis The fio error report indicates the following: verify: bad header: This means the data block read back by fio during the "Verify Phase" has a corrupted header. at file ... offset 1048576: This tells us: fio successfully verified the first 1MB (offset 0 to 1048575). The error occurred when fio attempted to verify the second 1MB data block (starting at offset 1,048,576). This indicates that our driver or controller starts failing when handling I/O requests that cross this 1MB address boundary. Synthesis & Next Steps We suspect there is a bug in our DEXT driver when handling Buffered I/O writes. The trigger conditions are: The I/O must be Buffered I/O. The total I/O for a single file must exceed a threshold (> 128MB). When these conditions are met, the driver or hardware errs when processing write requests that cross the 1MB byte boundary, causing corruption for all subsequent data. This is likely related to the processing flow of the SYNCHRONIZE CACHE command, and how I/O requests are segmented or addressed in memory. Our Next Plan: Following your suggestion, we will implement detailed logging within the DEXT. Our goal is to capture all SCSI commands and their parameters (specifically LBA and transfer length) entering and exiting the driver during the failed 256MB fio test. We will focus on observing exactly what happens when I/O requests touch and cross this critical 1MB location. Best Regards, Charles (Attachments include the test scripts and full logs for both cp and fio.) run_copy_verify_test.sh run_copy_verify_test.log integrity_test.fio run_fio_tests.sh run_fio_tests.log
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, Final update: It works! We have successfully resolved the issues related to I/O splitting and size. The solution was to: Correctly and completely implement all 7 required keys in UserReportHBAConstraints, based on the KEXT's configuration. Set maxTransferSize in UserGetDMASpecification to a sufficiently large value (>= SegmentCount * SegmentByteCount). Once we implemented these two changes, everything fell into place. Our DEXT can now handle large I/O requests of any size (we've successfully tested up to 1GB in a single transfer). dd bs=2m now works as expected, and diskutil can also successfully partition and format the disk. We observed that the IOMaximumBlockCount property on IOSCSIPeripheralDeviceType00 still shows as 0xffff, but as you anticipated, this value does not seem to affect the behavior of IOBreaker. Our tests confirm that our hardware can handle the resulting 512KB segments directly. We could not have solved this problem without your persistent and incredibly precise guidance. Thank you for taking the time to analyze our ioreg files and for explaining the complex behaviors of the IOKit storage stack and DriverKit. Thank you again! Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, We have an update after completing the DEXT configuration as you advised. The final result is that any dd command with a block size larger than 512KB to the rdisk node still fails, just as before. The tests and configurations that led to this result are detailed below. Final Configuration (We have tried all approaches): Based on the setProperty calls from our KEXT's source code and the properties from the .ioreg analysis, we have implemented the following four-part configuration in our DEXT: HBA Level (UserReportHBAConstraints): We implemented this function to report all key properties from the KEXT's HBA layer (e.g., kIOMaximumByteCountWriteKey = 512KB). LUN Level (UserCreateTargetForID): In AsyncCreateTargetForID, we built a personalities dictionary for each LUN and attempted to set IOMaximumBlockCountWriteKey = 0x400 through it. Identity Declaration (UserInitializeController): In UserInitializeController(), we set the Protocol Characteristics dictionary via this->SetProperties. Low-Level Sync (UserGetDMASpecification): We also set maxTransferSize to 512KB to ensure consistency with the HBA layer. (We have confirmed that IORegistryExplorer shows the Protocol Characteristics were set successfully, but IOMaximumBlockCountWrite remains 0xffff.) dd Test Results: After applying all the configurations above, we ran the following tests on the /dev/rdisk node: dd bs=64k count=1 -> Succeeded dd bs=128k count=1 -> Succeeded dd bs=512k count=1 -> Succeeded dd bs=768k count=1 -> Failed (Input/output error) Our Analysis: The success of bs=512k indicates that our property configuration at the HBA level (UserReportHBAConstraints + UserGetDMASpecification) is effective. IOKit allows a single request up to 512KB to pass through. The failure of bs=768k, however, shows that IOKit's IOBreaker splitting functionality has not been successfully activated. If it had been, we should have at least seen the first 512KB sub-request in our DEXT's log, but in fact, we saw nothing. This seems to indicate that merely declaring size limits and protocol characteristics is insufficient to get IOBreaker to work for an IOUserSCSIParallelInterfaceController type DEXT. Our Question: You previously described a two-layer splitting model: IOBreaker ... segments the transfer... Your DEXT receives ... a single large physical segment. Your DEXT breaks that physical segment into individual transfers that match whatever your hardware requires. Combining all our experimental evidence (especially our inability to influence IOMaximumBlockCountWrite from the DEXT), we believe the reason our KEXT succeeded is that its parent class implemented step 3, whereas our DEXT has not. We would like to confirm with you: Our plan for the next step is to manually write a "second-layer I/O splitter" from scratch within our UserProcessParallelTask_Impl (for example, to split a received 512KB request into eight 64KB hardware commands). Is this the expected and correct solution for handling such hardware limits in DriverKit? Thank you very much! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, Thank you for the detailed explanation and the pointer to IOBlockStorageDriver::breakUpRequest() and IOBreaker. That is incredibly helpful information. As you requested, I have filed a Feedback report containing the .ioreg snapshots from both the KEXT and DEXT configurations. A sysdiagnose log, generated by Feedback Assistant during the submission process, is also attached. The Feedback ID is: FB20924370 We look forward to your analysis of the registry files. Best regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, Thank you again for your guidance. We have an important update based on your advice. Following your suggestion to "compare property configurations," we analyzed our KEXT's source code, as we could not find the relevant properties in the KEXT's IORegistryExplorer snapshot. We then attempted to replicate the KEXT's behavior in our DEXT, which has led us to a final, specific contradiction that we hope you can help us resolve. Here is a summary of our findings and final questions: 1 - KEXT Source Code Contains setProperty Calls We discovered that our KEXT's Start() function programmatically sets several IOBlockStorageCharacteristics using setProperty. The key value is: // In KEXT's Start() function: #define MAX_IO_LENGTH (512 * 1024) // 512KB setProperty(kIOMaximumByteCountWriteKey, (UInt64)MAX_IO_LENGTH, 64); // ... plus other similar properties. 2 - DEXT Fails Despite Replicating KEXT's Configuration Based on this finding and your previous advice, we implemented the following in our DEXT: We implemented SetupHBAConstraints() and call the UserReportHBAConstraints virtual function. Inside it, we populated the dictionary with the exact same values found in our KEXT's source, including setting kIOMaximumByteCountWriteKey to 512KB. We also kept our UserGetDMASpecification's maxTransferSize at 64KB, as this reflects our hardware's actual single DMA transaction limit. However, a dd bs=128k command to the rdisk node still fails with an Input/output error. 3 - The Final Contradiction: KEXT's "Hidden" Splitting Behavior This is the core of our confusion. When we test the KEXT with a large I/O (dd bs=10m), our driver logs show that IOKit is, in fact, splitting the request into 64KB chunks, not the 512KB size we specified in the KEXT's source code. This leads to our final questions: 1 - Why does IOKit split I/O into 64KB chunks for our KEXT, even when kIOMaximumByteCountWriteKey is set to 512KB? Is there another, higher-priority property or mechanism (perhaps related to Protocol Characteristics) that is forcing this smaller split size? 2 - In the DEXT environment, this 64KB splitting behavior is not being triggered. Instead, it appears the upper layers of IOKit are respecting our 512KB hint (by not splitting a 128KB request), while the lower layers reject it for exceeding the 64KB maxTransferSize. How can we correctly replicate the KEXT's robust "64KB splitting" behavior in our DEXT? We are now unable to explain the discrepancy between the KEXT's declared properties and its actual I/O splitting behavior. Any clarification you can provide would be immensely helpful. Best regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Nov ’25
Reply to System Panic with IOUserSCSIParallelInterfaceController during Dispatch Queue Configuration
Hi Kevin, Thank you for your valuable insights. Following your advice, we have refactored our driver to use a "wrapper" pattern where UserProcessBundledParallelTasks serves as a high-performance entry point that forwards commands to our core dispatch logic. To eliminate potential race conditions, we have also moved our interrupt handling to the Default Dispatch Queue using kIOServiceDefaultQueueName. This ensures that command submission and completion are strictly serialized. Here are the key implementation details showing how we unified the dispatch logic: 1. Legacy Entry Point (Single Task): We extracted our core logic into a helper method, DispatchTaskInternal, passing 0xFFFF as a placeholder for the Slot Index. kern_return_t MyDriver::UserProcessParallelTask_Impl( SCSIUserParallelTask parallelRequest, uint32_t *response, OSAction *completion) { // Forward to unified internal dispatcher with no slot index (Legacy Mode) return DispatchTaskInternal(parallelRequest, response, completion, 0xFFFF); } 2. Bundled Submission Path: In our UserProcessBundledParallelTasks implementation, we iterate through the indices, create a local object copy of the request (as you suggested), and call the same internal dispatcher. void MyDriver::UserProcessBundledParallelTasks_Impl( const uint16_t parallelRequestSlotIndices[32], uint16_t parallelRequestSlotIndicesCount, OSAction * completion) { for (uint16_t i = 0; i < parallelRequestSlotIndicesCount; i++) { uint16_t slotIndex = parallelRequestSlotIndices[i]; const SCSIUserParallelTask& sharedReq = ivars->fCommandBuffers[slotIndex]; // Create a local copy to utilize existing processing logic SCSIUserParallelTask localReq = sharedReq; uint32_t response; // DispatchTaskInternal performs a completion->retain() for EVERY command, // ensuring the action stays alive regardless of the bundled count. DispatchTaskInternal(localReq, &response, completion, slotIndex); } } 3. Unified Asynchronous Completion Path: Regardless of the mode, we now use the stable ParallelTaskCompletion within the interrupt handler (ISR), followed by the corresponding release(). // Inside the Asynchronous Interrupt Handler (serialized on Default Queue) SCSIUserParallelResponse taskResponse = {}; // ... Populate status, TaskID, and Version ... // Signal completion to the framework ParallelTaskCompletion(completion, taskResponse); // Balance the reference count performed during submission completion->release(); Observations and Current Bottleneck: If we intercept a command (like TEST UNIT READY) and call the completion logic "synchronously" within the UserProcessBundledParallelTasks scope, the system works perfectly and immediately issues the next command (e.g., INQUIRY). As soon as the completion is triggered from the asynchronous interrupt path—even though it is serialized on the same Default Queue—invoking ParallelTaskCompletion (or BundledParallelTaskCompletion) causes the DEXT to crash immediately (Corpse / Address size fault). Questions: Since we have ensured an independent retain/release cycle for every command and used a serialized queue, why does the OSAction handle appear to become invalid once it enters an asynchronous callback context in Bundled mode? Is there a hidden thread-context or lifecycle restriction on the OSAction provided during a bundled call that differs from legacy mode? We have verified that the legacy mode remains 100% stable under the same serialized queue configuration. We would appreciate any further guidance on why the "Sync succeeds, Async crashes" behavior persists in the high-performance path. Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Jan ’26
Reply to System Panic with IOUserSCSIParallelInterfaceController during Dispatch Queue Configuration
Hi Kevin, Thank you for your previous guidance. We have shifted our driver architecture to the UserProcessBundledParallelTasks model with Shared Command/Response Buffers to optimize I/O performance. But, we are encountering a persistent Kernel Panic / DEXT Crash (Corpse) immediately after switching to the Bundled mode, preventing the target discovery and initialization sequence from completing, while the legacy mode (UserProcessParallelTask) is rock-solid and stable. Implementation Details: Memory Mapping: Successfully implemented UserMapBundledParallelTaskCommandAndResponseBuffers. DEXT correctly obtains the virtual addresses for both buffers. Dispatching: Inside UserProcessBundledParallelTasks, we iterate through the parallelRequestSlotIndices, reading from the shared command buffer and dispatching tasks to the hardware. Completion: Upon hardware completion, we populate the shared response buffer in the asynchronous path (ISR/Poll) and invoke BundledParallelTaskCompletion with the corresponding OSAction. The Critical Race Condition Observed: Due to the very low latency of our hardware for specific commands (e.g., TEST UNIT READY). Logs reveal a severe timing conflict: The interrupt handler (Completion path) is triggered and successfully invokes BundledParallelTaskCompletion, which returns normally. Crucially, at this exact microsecond, the original UserProcessBundledParallelTasks call (Submission path) has not yet finished its loop or returned to the system. Immediately after both paths eventually return, the DEXT process crashes (Corpse), subsequently triggering a Kernel Panic. Troubleshooting Steps Taken (Issue Persists): Memory Protection: Clamped Sense Data to exactly 18 bytes (SCSI fixed format) to prevent any potential buffer overflows in the shared response buffer. Array Sizing: Ensured that the index array passed to BundledParallelTaskCompletion is a fixed-size uint16_t[32] to align with the .iig declaration and ensure RPC serialization safety. Reference Counting: Tested both calling and omitting release() on the OSAction in the completion path. The crash occurs regardless of manual release. Memory Barriers: Implemented __atomic_thread_fence(__ATOMIC_SEQ_CST) to ensure shared buffer writes are visible before signaling completion. Questions: In Bundled mode, does DriverKit support a scenario where the Completion RPC returns before the Submission RPC has finished its execution? Does this cause an internal state-machine conflict within IOUserSCSIParallelInterfaceController? What is the exact ownership model for the OSAction * action in Bundled mode? Is the action "consumed" by BundledParallelTaskCompletion, or is the driver still responsible for retain/release management as in legacy mode? Any advice on how to resolve this would be very helpful. Thanks! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Jan ’26
Reply to [DriverKit SCSI] SCSI probe stalls for Target ID > 0 with IOUserSCSIParallelInterfaceController
Hi Kevin, Thank you very much for your guidance. I am happy to report that the issue has been resolved. I would like to share the root cause and the final solution here in case it helps other developers facing similar issues. As you suspected, it was indeed a bug in our code regarding capability reporting. Although we were successfully calling AsyncCreateTargetForID(1), the macOS SCSI stack was silently aborting the probe sequence after the initial TEST UNIT READY command. The issue was located in our implementation of UserReportHighestSupportedDeviceID. We were mistakenly hardcoding the return value to 0: // Buggy implementation kern_return_t IMPL(DRV_MAIN_CLASS_NAME, UserReportHighestSupportedDeviceID) { *id = 0; // Reported that the HBA only supports ID 0 return kIOReturnSuccess; } Because the HBA reported a maximum supported ID of 0, the macOS SCSI Family treated Target 1 as an "out-of-bounds" device. While the object was instantiated and visible in ioreg, the kernel refused to Register or Match it. This explained why Target 1 remained in a !registered state and never received an INQUIRY command. We updated the method to return the correct maximum ID supported by our controller (63 to support up to 64 targets): // Corrected implementation kern_return_t IMPL(DRV_MAIN_CLASS_NAME, UserReportHighestSupportedDeviceID) { *id = 63; // Support for IDs 0 through 63 return kIOReturnSuccess; } Immediately after this change, Target 1 moved to a registered, matched state in the I/O Registry. The SCSI stack successfully issued the INQUIRY command, and diskutil list now correctly identifies both RAID volumes. Thank you again! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Jan ’26
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, Just a quick update: I have uploaded the source code (SetupHBAConstraints_Workaround.cpp) to the bug report (FB21256805). This file confirms that we are indeed defining all the required keys (including kIOMaximumSegmentByteCount) before calling UserReportHBAConstraints, and it also demonstrates the SetProperties workaround we implemented to bypass the issue. Best regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Dec ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, I have filed the bug report as requested. The Feedback ID is FB21256805. I've included the IORegistry dumps showing the missing keys when using the standard API versus the successful injection using the SetProperties workaround. To answer your specific question: "Are you sure you're setting all of the required key set, particularly 'kIOMaximumSegmentByteCountWriteKey'?" Yes. We confirmed that kIOMaximumSegmentByteCountWriteKey was correctly set to 65536 (64 KB), and kIOMaximumSegmentCountWriteKey was set to 129. Thanks again for your help in narrowing this down. Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Dec ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, Apologies, but it seems our previous celebration was premature. I must retract the conclusion from my last post regarding the issue being resolved. Here is the current situation: While our fixes regarding maxTransferSize (1MB) and SGL splitting have indeed achieved a 100% pass rate for CLI tools (cp and fio) across all file sizes, we have discovered that Finder still fails with Error -36 during file copies. Based on our latest findings, I would appreciate your insights on the following: CLI (cp, fio): Read/Write operations are 100% successful, even with large files (e.g., 4GB). Finder: Copying "Pre-existing files" (Cold Data): Success for files < 64MB. Copying "Freshly created files" (Dirty Data/Hot Cache): Immediate failure (Error -36) for files > 1MB. We suspected that "freshly created files" were highly fragmented in memory, causing the DEXT to construct a Scatter/Gather List (SGL) that exceeded our hardware limit (129 Segments). To verify this, we implemented "Trap Logs" inside the DEXT to monitor the SGCount. The logs proved this hypothesis wrong. Finder is not sending requests that exceed the limit. At the exact moment Finder reports the error, the DEXT is receiving a series of extremely small and clean requests, not fragmented large I/O: Request sizes are mostly 4KB (0x1000) or 16KB (0x4000). The SGCount for each request is only 0 or 1. This is far below our hardware limit of 129. This leaves us with a confusing contradiction: CLI (fio/cp): Sends larger chunks (e.g., 512KB), SGCount = 1 (likely due to contiguous allocation by fio). Result: Success. Finder: Sends tiny chunks (4KB/16KB), SGCount = 1. Result: Failure (Error -36). To rule out configuration variables, we have set the DEXT's UserReportHBAConstraints to match our stable Legacy KEXT exactly: IOMaximumByteCount = 512 KB (0x80000) IOMaximumSegmentCount = 129 Since the logs prove that Finder's requests fully comply with our reported Constraints, why would DriverKit or macOS determine a failure? Does Finder's I/O behavior trigger specific SCSI command sequences or timing issues that CLI tools do not? We observed that Finder issues multiple TEST UNIT READY and SYNCHRONIZE CACHE commands immediately before the write failure. Does this imply that some error handling mechanism is being triggered prematurely? Thank you again for your time and assistance. Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to Archive Fails: Conflict with Existing KEXT Developer ID Certificate
Hi everyone, After extensive testing and verification, we have finally resolved the persistent DriverKit archiving and signing errors. The core issue lies in the fact that Xcode enforces distinctly different certificate requirements for DriverKit targets during the "Build Phase" versus the "Distribution Phase." We are sharing our successful experience to help other developers facing the same trap. 💡 Core Conclusion Do not attempt to configure the Distribution Profile directly in Xcode’s Target settings, as this will cause the build to fail. The correct workflow is: Use "Development Certificates (Auto)" during the Build/Archive phase, and only swap in the "Distribution Certificates (Manual)" during the Organizer Distribution phase. 🔍 Root Cause Analysis The DriverKit Linker mandates linking against the Development SDK during the build phase. If you force a manual Distribution Profile in Xcode’s "Signing & Capabilities" tab, Xcode will fail because it cannot find a matching development certificate (often resulting in errors like Signing Certificate: None), making it impossible to generate an Archive. Step-by-Step Guide Phase 1: Xcode Project Settings Go to the Signing & Capabilities tab of your DEXT Target. Check "Automatically manage signing". Ensure Xcode generates a Team Provisioning Profile (Development) without errors. Note: This step is crucial to satisfy the compiler's requirement for the Development SDK. Phase 2: Generating the Archive Select Product -> Archive. Since you are using Auto Signing (Development), the build should complete successfully and appear in the Organizer. Phase 3: Organizer Distribution In the Organizer, click Distribute App -> Direct Distribution (or your preferred distribution method). When asked for the signing method, you select "Manually manage signing". In the Profile selection screen: App Target: Select the corresponding App Distribution Profile. DEXT Target: Manually select the "DriverKit Distribution Profile" you created in the Developer Portal. 🛠 Verification After exporting the App, use the following command to check the DEXT entitlements: codesign -d --entitlements :- "Path/To/Your.app/Contents/Library/SystemExtensions/com.your.driver.dext" Success Indicators: It includes your required hardware entitlements (e.g., com.apple.developer.driverkit.family.scsicontroller). It does NOT include com.apple.security.get-task-allow. (The absence of this flag confirms that the debug marker has been removed and successfully replaced with a production signature). Hope this strategy saves everyone some valuable time!
Replies
Boosts
Views
Activity
Nov ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, Regarding your previous analysis of the KEXT configuration, specifically where you pointed out the IOMaximumByteCount is actually 0x80000 (512KB) in this post. We observed the runtime logs of the legacy KEXT, and the results align perfectly with your observation. Even when copying a 2GB file, the KEXT actually splits the request into 512KB (0x80000) chunks per transfer. [ProcessParallelTask] [KEXT_CHECK] WRITE (0x2a) | T:0 L:0 | Size: 524288 bytes (512 KB) This confirms that our current 1MB limit in the DEXT actually exceeds the behavior of the legacy driver. Since the current setting has passed all data integrity verifications, we consider the transfer size issue resolved and will not pursue larger transfer sizes at this stage. As per your recommendation, we are currently implementing the UserProcessBundledParallelTasks architecture to strictly follow DriverKit best practices and optimize performance (it is currently causing the DEXT to crash, so we are in the middle of debugging it). Thanks again for your precise insights! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, The data corruption issue has been resolved! Special thanks for asking the key question: "So what's IS there?" This prompted us to shift our focus from high-level hash verification to inspecting the raw bytes written to the disk. This investigation revealed that the root cause was not related to cache flushing, but rather a hardware limitation regarding single transfer lengths. Here is a summary of our findings and the solution: By using a Python script to verify the data on disk byte-by-byte, we discovered: When macOS coalesced writes into 2MB chunks (as we had previously set maxTransferSize to 128MB), data corruption consistently began exactly at Offset 1MB + 16KB within the command (manifesting as 0x00 or garbage data). The LSI 3108 controller/firmware seems cannot correctly handle a single SCSI command with a data length exceeding 1MB in the DEXT environment. We implemented a two-layer fix: In UserGetDMASpecification, we explicitly set maxTransferSize to 1MB (1,048,576 bytes). This forces macOS to split large I/O requests into smaller chunks that the hardware can safely digest. To align with SCSI best practices and ensure maximum stability, we implemented logic within the driver to further split these 1MB buffers into multiple 64KB SGL Segments when populating hardware descriptors. With these fixes applied, all previously failing test scenarios (including cp + sync and fio, with file sizes ranging from 256MB up to 5GB) now PASS 100%. The checksums match perfectly. Thanks again for your guidance — it pointed us directly to the core issue! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to DEXT (IOUserSCSIParallelInterfaceController): Direct I/O Succeeds, but Buffered I/O Fails with Data Corruption on Large File Copies
Hi Kevin, TL;DR (Core Issues) 100% Reproducible: We used two completely independent test methods (cp + sync and fio buffered mode), and both consistently reproduced the data corruption issue under Buffered I/O. Clear Failure Threshold: Both tests point to the exact same failure boundary: file sizes 128MB (inclusive) and below pass, but 256MB (inclusive) and above consistently fail. Key Clue Confirmed: The fio test again reported the exact same error as our initial finding: data corruption starts at a very precise offset 1048576 (1MB) location. You mentioned last time that this offset was "odd"; now we can confirm this is not a coincidence, but a core characteristic of the problem. Test 1: System-Level File Copy Integrity Test (cp + sync) This test aims to simulate standard user operations to rule out tool-specific issues. Methodology: We used a shell script to automate cp for file copying, used sync to force the system to flush I/O caches, and finally compared the SHA256 hashes of the source and destination files. Results: 128MB File: Success. Hashes match perfectly. 256MB File: Failure. Hash mismatch, confirming data corruption during the copy process. Conclusion: This proves that the issue lies within the system I/O stack, and our DEXT has a defect when handling standard buffered writes. Test 2: fio Buffered I/O Integrity Test This test leverages fio's robust verification capabilities to obtain lower-level error details. Settings & Expected Behavior Settings (buffered_integrity_test.fio): We removed direct=1 to force fio to use the OS file cache, consistent with the behavior of cp. We set do_verify=1 and verify=sha256 to require data integrity checks after I/O operations. fio Mechanism: The verification process consists of two independent phases (it does not verify-as-it-writes): Write Phase: fio writes the entire 256MB data (each 1MB block carries a unique header for verification) to the file. Verify Phase: After writing concludes, fio reads back the entire 256MB starting from the beginning (offset 0), checking the headers block by block. Test Results The results match the cp test exactly: 128MB passes, 256MB fails. During the 256MB failure, we received the following error message: [*] Testing file size: 256M ... verify: bad header ... at file ... offset 1048576, length 1048576 fio: ... error=Illegal byte sequence ... [!!!] TEST FAILED at file size: 256M Error Message Analysis The fio error report indicates the following: verify: bad header: This means the data block read back by fio during the "Verify Phase" has a corrupted header. at file ... offset 1048576: This tells us: fio successfully verified the first 1MB (offset 0 to 1048575). The error occurred when fio attempted to verify the second 1MB data block (starting at offset 1,048,576). This indicates that our driver or controller starts failing when handling I/O requests that cross this 1MB address boundary. Synthesis & Next Steps We suspect there is a bug in our DEXT driver when handling Buffered I/O writes. The trigger conditions are: The I/O must be Buffered I/O. The total I/O for a single file must exceed a threshold (> 128MB). When these conditions are met, the driver or hardware errs when processing write requests that cross the 1MB byte boundary, causing corruption for all subsequent data. This is likely related to the processing flow of the SYNCHRONIZE CACHE command, and how I/O requests are segmented or addressed in memory. Our Next Plan: Following your suggestion, we will implement detailed logging within the DEXT. Our goal is to capture all SCSI commands and their parameters (specifically LBA and transfer length) entering and exiting the driver during the failed 256MB fio test. We will focus on observing exactly what happens when I/O requests touch and cross this critical 1MB location. Best Regards, Charles (Attachments include the test scripts and full logs for both cp and fio.) run_copy_verify_test.sh run_copy_verify_test.log integrity_test.fio run_fio_tests.sh run_fio_tests.log
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, Final update: It works! We have successfully resolved the issues related to I/O splitting and size. The solution was to: Correctly and completely implement all 7 required keys in UserReportHBAConstraints, based on the KEXT's configuration. Set maxTransferSize in UserGetDMASpecification to a sufficiently large value (>= SegmentCount * SegmentByteCount). Once we implemented these two changes, everything fell into place. Our DEXT can now handle large I/O requests of any size (we've successfully tested up to 1GB in a single transfer). dd bs=2m now works as expected, and diskutil can also successfully partition and format the disk. We observed that the IOMaximumBlockCount property on IOSCSIPeripheralDeviceType00 still shows as 0xffff, but as you anticipated, this value does not seem to affect the behavior of IOBreaker. Our tests confirm that our hardware can handle the resulting 512KB segments directly. We could not have solved this problem without your persistent and incredibly precise guidance. Thank you for taking the time to analyze our ioreg files and for explaining the complex behaviors of the IOKit storage stack and DriverKit. Thank you again! Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, We have an update after completing the DEXT configuration as you advised. The final result is that any dd command with a block size larger than 512KB to the rdisk node still fails, just as before. The tests and configurations that led to this result are detailed below. Final Configuration (We have tried all approaches): Based on the setProperty calls from our KEXT's source code and the properties from the .ioreg analysis, we have implemented the following four-part configuration in our DEXT: HBA Level (UserReportHBAConstraints): We implemented this function to report all key properties from the KEXT's HBA layer (e.g., kIOMaximumByteCountWriteKey = 512KB). LUN Level (UserCreateTargetForID): In AsyncCreateTargetForID, we built a personalities dictionary for each LUN and attempted to set IOMaximumBlockCountWriteKey = 0x400 through it. Identity Declaration (UserInitializeController): In UserInitializeController(), we set the Protocol Characteristics dictionary via this->SetProperties. Low-Level Sync (UserGetDMASpecification): We also set maxTransferSize to 512KB to ensure consistency with the HBA layer. (We have confirmed that IORegistryExplorer shows the Protocol Characteristics were set successfully, but IOMaximumBlockCountWrite remains 0xffff.) dd Test Results: After applying all the configurations above, we ran the following tests on the /dev/rdisk node: dd bs=64k count=1 -> Succeeded dd bs=128k count=1 -> Succeeded dd bs=512k count=1 -> Succeeded dd bs=768k count=1 -> Failed (Input/output error) Our Analysis: The success of bs=512k indicates that our property configuration at the HBA level (UserReportHBAConstraints + UserGetDMASpecification) is effective. IOKit allows a single request up to 512KB to pass through. The failure of bs=768k, however, shows that IOKit's IOBreaker splitting functionality has not been successfully activated. If it had been, we should have at least seen the first 512KB sub-request in our DEXT's log, but in fact, we saw nothing. This seems to indicate that merely declaring size limits and protocol characteristics is insufficient to get IOBreaker to work for an IOUserSCSIParallelInterfaceController type DEXT. Our Question: You previously described a two-layer splitting model: IOBreaker ... segments the transfer... Your DEXT receives ... a single large physical segment. Your DEXT breaks that physical segment into individual transfers that match whatever your hardware requires. Combining all our experimental evidence (especially our inability to influence IOMaximumBlockCountWrite from the DEXT), we believe the reason our KEXT succeeded is that its parent class implemented step 3, whereas our DEXT has not. We would like to confirm with you: Our plan for the next step is to manually write a "second-layer I/O splitter" from scratch within our UserProcessParallelTask_Impl (for example, to split a received 512KB request into eight 64KB hardware commands). Is this the expected and correct solution for handling such hardware limits in DriverKit? Thank you very much! Best Regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, Thank you for the detailed explanation and the pointer to IOBlockStorageDriver::breakUpRequest() and IOBreaker. That is incredibly helpful information. As you requested, I have filed a Feedback report containing the .ioreg snapshots from both the KEXT and DEXT configurations. A sysdiagnose log, generated by Feedback Assistant during the submission process, is also attached. The Feedback ID is: FB20924370 We look forward to your analysis of the registry files. Best regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25
Reply to [DEXT Migration Issue] IOUserSCSIParallelInterfaceController fails to handle low-level I/O from `diskutil`
Hi Kevin, Thank you again for your guidance. We have an important update based on your advice. Following your suggestion to "compare property configurations," we analyzed our KEXT's source code, as we could not find the relevant properties in the KEXT's IORegistryExplorer snapshot. We then attempted to replicate the KEXT's behavior in our DEXT, which has led us to a final, specific contradiction that we hope you can help us resolve. Here is a summary of our findings and final questions: 1 - KEXT Source Code Contains setProperty Calls We discovered that our KEXT's Start() function programmatically sets several IOBlockStorageCharacteristics using setProperty. The key value is: // In KEXT's Start() function: #define MAX_IO_LENGTH (512 * 1024) // 512KB setProperty(kIOMaximumByteCountWriteKey, (UInt64)MAX_IO_LENGTH, 64); // ... plus other similar properties. 2 - DEXT Fails Despite Replicating KEXT's Configuration Based on this finding and your previous advice, we implemented the following in our DEXT: We implemented SetupHBAConstraints() and call the UserReportHBAConstraints virtual function. Inside it, we populated the dictionary with the exact same values found in our KEXT's source, including setting kIOMaximumByteCountWriteKey to 512KB. We also kept our UserGetDMASpecification's maxTransferSize at 64KB, as this reflects our hardware's actual single DMA transaction limit. However, a dd bs=128k command to the rdisk node still fails with an Input/output error. 3 - The Final Contradiction: KEXT's "Hidden" Splitting Behavior This is the core of our confusion. When we test the KEXT with a large I/O (dd bs=10m), our driver logs show that IOKit is, in fact, splitting the request into 64KB chunks, not the 512KB size we specified in the KEXT's source code. This leads to our final questions: 1 - Why does IOKit split I/O into 64KB chunks for our KEXT, even when kIOMaximumByteCountWriteKey is set to 512KB? Is there another, higher-priority property or mechanism (perhaps related to Protocol Characteristics) that is forcing this smaller split size? 2 - In the DEXT environment, this 64KB splitting behavior is not being triggered. Instead, it appears the upper layers of IOKit are respecting our 512KB hint (by not splitting a 128KB request), while the lower layers reject it for exceeding the 64KB maxTransferSize. How can we correctly replicate the KEXT's robust "64KB splitting" behavior in our DEXT? We are now unable to explain the discrepancy between the KEXT's declared properties and its actual I/O splitting behavior. Any clarification you can provide would be immensely helpful. Best regards, Charles
Topic: App & System Services SubTopic: Drivers Tags:
Replies
Boosts
Views
Activity
Nov ’25