Updating pending isochronous OUT buffers

I’m developing a low-latency USB audio-MIDI driver using AudioDriverKit and USBDriverKit for macOS and iPadOS.

I've observed occasional delays in USB completion callbacks. The manufacturer's official macOS driver uses IOUSBHost to queue longer isochronous transfers and update their mapped OUT buffers as audio samples become available.

I've reviewed the relevant IOKit and IOUSBHost documentation, but haven't established whether USBDriverKit supports an equivalent approach.

Basically, "yes". More specifically, USBDriverKit and the IOUSBHost framework are actually quite similar, in that what both actually "do” is control a USB device by sending commands to the in-kernel USB driver which ACTUALLY controls the device. The main difference/benefit DriverKit provides is the ability to inject services back INTO the kernel, not in the mechanics of how it actually interacts with its provider.

Does USBDriverKit permit modifying portions of a CreateIOBuffer OUT buffer while an IsochIO request is still pending, specifically to supply audio for future USB frames?

Yes, as this is basically the entire "point" of our Isochronous APIs. More specifically, the basic idea here is that what's actually slow/high latency here is:

• Setting up the chain of commands that you're doing to send.

• Receiving the completion for any given event.

...while the actual transmission of any given packet happens at interrupt time, basically make it as fast as it's possible to be.

SO, the idea here is that you transfer data by:

1. Scheduling a long series of transfers to occur at specific times "in the future".

2. Backfilling the data into each of those transfers "before" they're actually sent.

3. ...then looping back to #1 by scheduling more transfers before #1 finishes, allowing the transfer process to continue indefinitely.

If so, what synchronization requirements and controller-prefetch guarantees determine the latest safe time to update those regions?

I'm not sure what you mean here. The memory you're writing to has already been wired, so it can be modified any time "before" the actual transfer starts. Roughly when that transfer starts can be determined mathematically, but part of the "point" of this architecture is to avoid explicit synchronization between the data producer and the sending driver.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Thanks, Kevin. Your explanation has helped clarify the intended architecture.

To explain my earlier synchronization question: I'm using 32 ms isochronous OUT requests containing multiple scheduled microframes, with audio data written into the mapped buffers as it becomes available.

I was trying to establish whether the payload for a later microframe can still be updated after the request has begun transmitting, provided that particular microframe hasn't yet been sent.

In other words, does the safe-update boundary apply to the entire request or to each scheduled microframe?

I'm asking because this distinction affects how closely to transmission I can safely supply audio without increasing latency.

Thanks again,
Luke

Thanks, Kevin. Your explanation has helped clarify the intended architecture.

To explain my earlier synchronization question: I'm using 32 ms isochronous OUT requests containing multiple scheduled microframes, with audio data written into the mapped buffers as it becomes available.

Just to clarify, did you really mean "32 ms" (milliseconds)? If so, then I think the problem here is that your message count is WAY too small. Frankly, I'd expect our bulk transfer pipes to be able to do 30 transfers per second without much difficulty. For reference, TN3190 has specific rate guidance for particular transfer types, and synchronous data endpoints use this example:

"For example, if a device is using a sample rate of 48 kHz, it should transfer exactly 48 audio samples every millisecond as defined by the SOF tokens on the bus. If it is on a High Speed bus with a polling interval of 125 µs, it should transfer exactly 6 samples every 125 µs."

That translates to 8000 transfers per second. That's the kind of update frequency isochronous was designed to support.

I was trying to establish whether the payload for a later microframe can still be updated after the request has begun transmitting, provided that particular microframe hasn't yet been sent.

In other words, does the safe-update boundary apply to the entire request or to each scheduled microframe?

This turns into a philosophical question about what the word "transmit" means. The controller "owns" the DMA buffer from the moment the transfer “starts," and how exactly it manipulates it isn't really defined. My guess is that we packetize the entire buffer in a single pass, which means there isn't really any "window" for you to use. However, the real problem here is that the actual behavior is going to depend on the specifics of the controller implementation, which means there isn't anything you can really rely on here.

I’m asking because this distinction affects how closely to transmission I can safely supply audio without increasing latency.

So, this makes me think you are thinking in terms of milliseconds (ms) not microseconds (µs). Audio latency is typically measured in milliseconds, with "OK" being “10–20 ms" and "Great!" being “3–6 ms". The problem here is that 3 ms (the VERY best) is 3000 µs. In other words, at the BEST latency level, you're talking about 24 transmissions spaced every 125 µs. In other words, the audio system generally fills up multiple USB requests per callback because the USB system is firing transfers more frequently than the real-time thread is firing.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

For context, the device that I'm building the driver for is a BOSS SY-1000 Guitar Synthesizer, with eight channels of 32-bit audio input and eight channels of output at 48 kHz, alongside two MIDI ports.

Yes, I meant 32 ms requests, each containing 256 high-speed microframes at 125 µs intervals. So the actual USB packet cadence remains 8,000 per second, rather than approximately 31 transfers per second.

Thanks for your clarification about controller ownership. I had been treating the individual scheduled microframes within a pending request as independently writable, but I now understand that the controller's handling of the entire DMA buffer after the request starts isn't guaranteed.

I've consequently revised the implementation to use shorter, 1 ms isochronous OUT requests, with each payload finalized before submission and left unchanged until completion and safe retirement.

Initial physical testing confirms that this arrangement is functioning, although I'm still working through some unrelated input-continuity problems.

Thanks again for taking the time to explain all this. Your insights have been invaluable to this newb!

Regards,
Luke

For context, the device that I'm building the driver for is a BOSS SY-1000 Guitar Synthesizer, with eight channels of 32-bit audio input and eight channels of output at 48 kHz, alongside two MIDI ports.

Yes, I meant 32 ms requests, each containing 256 high-speed microframes at 125 µs intervals. So the actual USB packet cadence remains 8,000 per second, rather than approximately 31 transfers per second.

OK, so let me state things a bit. When you setup an isochronous transfer, you're providing a list of frames that will be transferred "in the future" and it's known and expected that you can/will be modifying the content of future frames "before" any given frame is sent.

SO, if the controller just sent frame "2" it's perfectly valid to be modifying frames 4+. It's also valid to be modifying frame "3", however, the problem here is with this question:

...guarantees determine the latest safe time to update those regions?

The problem here is that you can't really time things "precisely" at the microsecond scale. That is, on paper, it's true that:

• You can modifying frame 3 10µs after frame 2 was sent.

• You can't modify frame 3 130µs after frame 2 was sent (because frame 3 is "gone").

However, trying to PRECISELY time YOUR changes such that your modifications consistently occur "at" exactly 120µs isn't actually possible, as even the realtime thread is scheduling at a substantially longer time than that. In practice, the audio system is generally writing at least 2-4 ms "ahead" of the USB bus, which translate to being 100+ frames of audio that's already been buffered. That buffer then masks any variability in thread scheduling.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware

Hi Kevin,

Thanks for the clear info. Here's a follow-up after extensive driver testing on macOS (M1 Max, macOS 27.2).

In normal operation, my 1 ms OUT requests are submitted 13–16 ms ahead of the controller. In the two incidents I've captured in detail, the next audio had already been published about 14 ms early, but the dispatch queue that submits requests wasn't serviced for roughly 8–10 ms (inferred from event timing), and the next request then missed its frame.

For reference, the vendor's own macOS driver on the same Mac (a Core Audio HAL plug-in that uses IOUSBHost, as I mentioned in the other thread) shows 4.58 ms round trip at a 32-frame buffer, 5.92 ms at 64 frames, and 8.58 ms at 128 frames (as reported by the DAW, not verified as true RTL, but certainly in the ballpark), so the hardware isn't the limit.

That leaves me with three questions I haven't been able to resolve through testing and research:

1. You mentioned the audio system typically writes 2–4 ms ahead of the bus; stalls of 8–10 ms are well beyond that. What determines how long a DriverKit dispatch queue can go unscheduled, and is there a supported way to bound it (for example, a QoS or priority setting)?
2. Should I expect the same dispatch and scheduling behaviour on iPadOS as on macOS? The iPad Pro (M1) is the eventual product target.
3. A request whose first USB frame is only 1 frame ahead of the frame number I read from the OUT interface with GetFrameNumber just before submitting is rejected as IsoTooOld, while 2 frames ahead has been accepted. Is there a documented minimum lead for IsochIO, and is GetFrameNumber the right current-frame source to compare against (as opposed to CurrentMicroframe)?

I can share the recorder exports if that would help.

Thanks again,
Luke

but the dispatch queue that submits requests wasn't serviced for roughly 8–10 ms (inferred from event timing), and the next request then missed its frame.

That actually sounds pretty normal. The way to handle this is that your driver actually schedules 2-3 requests (each covering X ms), so that any processing glitches are "hidden" by the extra request. That means you'd need to stall for ~30+ ms to actually disrupt anything. Note that what I'm describing has no impact on audio latency.

That also means that this:

A request whose first USB frame is only 1 frame ahead of the frame number I read from the OUT interface with GetFrameNumber just before submitting is rejected as IsoTooOld, while 2 frames ahead has been accepted. Is there a documented minimum lead for IsochIO, and is GetFrameNumber the right current-frame source to compare against (as opposed to CurrentMicroframe)?

...isn't a question your DEXT would normally ever "ask", as the request you're currently scheduling should always be well AFTER the "next" request (which you already scheduled "earlier").

You mentioned the audio system typically writes 2–4 ms ahead of the bus;

You are confusing the process of writing to the bus with the process of scheduling those future writes. The "writes" the audio system makes are the isochronous requests you previously set up.

This happens:

stalls of 8–10 ms are well beyond that.

...because you didn't preschedule enough requests to "cover up" any delays.

What determines how long a DriverKit dispatch queue can go unscheduled, and is there a supported way to bound it (for example, a QoS or priority setting)?

Assuming you're talking about callbacks coming "out" of the kernel, the main cause of delays is other DEXT activity was running on the target queue, which then delayed its firing. However, the isochronous architecture was intentionally designed so that the answer to this question wouldn't actually matter- you just buffer a few additional requests so that any timing variation is irrelevant.

Should I expect the same dispatch and scheduling behavior on iPadOS as on macOS?

Yes.

__
Kevin Elliott
DTS Engineer, CoreOS/Hardware