Dive into the world of video on Apple platforms, exploring ways to integrate video functionalities within your iOS,iPadOS, macOS, tvOS, visionOS or watchOS app.

Video Documentation

Posts under Video subtopic

Post

Replies

Boosts

Views

Activity

Feature Request: Enable Multiple Simultaneous iPhone Cameras Through Continuity Camera
I use continuity camera. I use a clone iPhone 17 pro to have a multi-camera podcast setup. CamoCam gets A&V out of sync which creates WAY MORE work. Dear Apple Engineering and Product Development Team, I am requesting an important enhancement to Apple’s Continuity Camera technology. Currently, Continuity Camera limits users to a single iPhone camera connected to a Mac at a time. Apple should expand this functionality to allow two, three, four, or more iPhones to operate simultaneously as independent camera sources. Connect multiple iPhones simultaneously to a single Mac using USB or Wi-Fi. Recognize each iPhone & toggle between cameras. Thank you, Brandon Johnson
0
0
8
1h
Swift 6: Supported AVCaptureSession actor isolation and MainActor preview attachment
I’m developing a SwiftUI camera preview for iPhone and iPad using Xcode 27.2 and Swift 6.4.2. I’m looking for clarification about the supported concurrency boundary between an actor-owned AVCaptureSession and a MainActor-owned AVCaptureVideoPreviewLayer. Environment Xcode 27.2 (27B52) Apple Swift 6.4.2 iPhoneOS SDK 27.2 Deployment target: iOS 26.0 Swift 6 language mode Complete concurrency checking Default actor isolation: MainActor Approachable Concurrency enabled The isolated compiler reproductions also explicitly enable NonisolatedNonsendingByDefault, InferIsolatedConformances, and MemberImportVisibility. Architecture The intended architecture follows the general pattern demonstrated in Apple’s AVCam sample: A capture actor uses a dedicated DispatchSerialQueue executor. The capture actor creates, configures, starts, and stops AVCaptureSession. A MainActor adapter owns AVCaptureVideoPreviewLayer. The preview layer retains the same session after attachment. The capture actor retains its session reference because it must perform subsequent capture operations. Compiler behavior In a minimal reproduction, transferring the retained session directly to the MainActor adapter: await preview.attach(session)produces: Sources/CaptureActor.swift:19:23: error: sending 'session' risks causing data races [#RegionIsolation::SendingRisksDataRace]The compiler identifies a potential race between actor-isolated and MainActor-isolated uses. This reproduction produces one error and zero warnings. A second reproduction uses a small PreviewSource: Sendable value containing a private AVCaptureSession reference, with @preconcurrency import AVFoundation confined to that source file. Its MainActor method attaches the session to the preview layer. Under otherwise identical compiler settings, this second reproduction compiles with zero diagnostics. We understand that compiler acceptance does not establish runtime safety and that the compatibility mechanism does not itself synchronize access to the retained session. Apple AVCam reference Apple’s official AVCam sample demonstrates a similar interoperability pattern: CaptureService.swift: dedicated capture executor and retained preview source. PreviewSource.swift: compatibility import and MainActor attachment. CameraPreview.swift: preview-layer association. Reference: Apple’s AVCam: Building a Camera App sample. Pinned sample SHA-256: 5a1981f90bb30827121324d4b33198d58de9b9d68e3508b64f0aabc18bbc0e53The sample revision date is not established; we are not assuming this archive is identical to the latest downloadable revision. Questions 1. Supported Swift 6 interoperability pattern Is the narrow PreviewSource compatibility approach demonstrated by AVCam the recommended way to associate an actor-retained AVCaptureSession with a MainActor-owned AVCaptureVideoPreviewLayer under complete Swift 6 concurrency checking? Is there a newer recommended API, annotation, or ownership pattern? 2. Capture and preview synchronization Is the following operation sequence supported? Configure and commit the capture session on the dedicated capture executor. Attach the session to the preview layer on MainActor. Call startRunning() on the capture executor. Perform ordinary preview layout on MainActor while capture runs. Call stopRunning() on the capture executor. Detach the session from the preview layer on MainActor. Which preview-layer or preview-connection mutations, if any, must be excluded while startRunning() or stopRunning() is executing? 3. Supported detachment and release sequence After stopRunning() returns and MainActor executes: previewLayer.session = nilis it supported to release the remaining application-held session references once application-owned observers and asynchronous consumers have terminated? Does AVFoundation require or recommend any additional step before those references are released? We are asking about the supported framework lifecycle contract, not assuming that application-level cancellation drains internal AVFoundation work. Sources AVCam: Building a camera app Overview Apple’s official AVCam sample
0
1
238
2d
AVCaptureSession multitasking camera access on iPhone without VoIP background mode
I'm developing an iPhone screen-recording application that simultaneously records the front-facing camera and displays the live camera feed using Picture in Picture while the user navigates other apps. The implementation uses AVFoundation (AVCaptureSession) for camera capture and a system Picture in Picture controller for the floating window. I've encountered the following behavior on a physical iPhone: With voip declared in UIBackgroundModes, isMultitaskingCameraAccessSupported returns true, and the front camera continues capturing while another app is foregrounded. Without voip, isMultitaskingCameraAccessSupported returns false. When the application enters the background, the capture session is interrupted and camera frames stop. Screen recording and the Picture in Picture window continue, but the camera feed freezes or becomes blank. Apple App Review has confirmed that declaring voip without actual VoIP functionality is not permitted under Guideline 2.5.4. I understand and respect that restriction and do not want to misuse background modes. My question: Is there a supported public API, entitlement, or configuration that allows a non-VoIP iPhone application to continue capturing front-camera video while another application is in the foreground? Does Picture in Picture provide any supported mechanism for this use case, or is multitasking camera capture unavailable to non-VoIP applications on iPhone? I'm looking for a supported implementation rather than a workaround that violates App Review guidelines. Any guidance from Apple's AVFoundation engineers would be greatly appreciated.
2
0
375
3d
Thin border around VideoPlayer
A thin gray border appears around assets presented using VideoPlayer. It was tested on iPhone 8, iPhone 13 mini, and iPhone 15, and the border only appeared on the 13 mini. The iOS version didn't matter, we checked the 13 mini on both iOS 18 and iOS 26 and the border appeared on both. I created a minimal version of the app with just the VideoPlayer and the asset and the frame still appeared. We tried different assets and it was visible on all of them. import SwiftUI import AVKit struct ContentView: View { private let player = AVPlayer(url: Bundle.main.url(forResource: "sample", withExtension: "mp4")!) var body: some View { VideoPlayer(player: player) .frame(width: 300, height: 200) } }I found someone that had the same exact problem on Reddit and StackOverflow: https://www.reddit.com/r/swift/comments/8s87hm/weird_borders_around_avplayerviewcontroller/ https://stackoverflow.com/questions/50927561/weird-borders-around-avplayerviewcontroller and their solution included setting: playerLayer.shouldRasterize = true playerLayer.rasterizationScale = UIScreen.main.scaleThis fixed the issue but isn't suitable for our app due to offscreen rendering performance. They also found that issue on other devices, noting: "After testing this code on multiple devices (actual physical devices, not simulators), this bug appears to happen only on Plus devices (I tested it on i8 on iOS 11, i8+ on iOS 11, iX on iOS 11 and i6+ on iOS 10)." The bug doesn't appear on simulator, isn't visible on a screenshot, the frame disappears when using accessibility zoom. When we encoded the video file to Apple ProRes the border disappeared, but the size of the video bumped from 342 KB to 6.3 MB so we couldn't do that. I'm attaching photo from this person's post since our issue looked exactly the same. Has anyone had a similar issue and maybe found another workaround?
0
0
245
4d
Official API for Apple Log / Apple Log 2 to SDR Rec.709 and HDR HLG Conversion?
Hello, I'm developing a video application for Apple platforms and would like to implement accurate Apple Log and Apple Log 2 color conversion using official Apple APIs. Final Cut Pro and Final Cut Camera support SDR Rec.709 and HDR Rec.2020 HLG monitoring. I have found Apple's official Apple Log white papers and Rec.709 LUTs, but I could not find an official HDR HLG LUT or a public API that provides equivalent color conversion. I have three questions: Does Apple provide a public API through AVFoundation, Core Image, ColorSync, or another framework to convert Apple Log / Apple Log 2 to SDR Rec.709 and HDR Rec.2020 HLG? Are the color transformations used by Final Cut Pro and Final Cut Camera available to third-party developers, either through public APIs or official LUT resources? If no such API exists, is implementing the transfer functions and gamut conversions from Apple's white papers, followed by standard SDR/HDR rendering, the recommended approach? My goal is to preserve the original Apple Log footage while allowing independent SDR/HDR monitoring without modifying the recorded data. Any guidance, documentation, or sample code from Apple engineers would be greatly appreciated. Thank you!
1
0
353
1w
Audio cut-out
Installed the latest beta releases of TVos and Home pod OS. 27.2 (24k122) The problem of audio cutting out whilst video keeps playing persists. The set up is: Mac with homeshare enabled, MP4 video in TV app on Mac encoded with ffmpeg, Apple tv 4k tvos 27, tv, Two homepods 27, in stereo pair. Problem: When playing a video from the mac on the tv via the appletv on the computers app the audio cuts out whilst the video keeps playing. Pausing and playing the video restores the video only for it to cut-out again and again. Switching audio from homepods to tv speakers there is no cut-out. Other sources play fine through the apple tv, streaming appletv, iplayer etc. The homepods work fine when watching tv with homepods using a satellite box. This only started when the TvOS was upgraded to 27, 26 was fine. It's not a problem with the Macs as I use two, one an old Mac pro and a new Mac mini, the problem is on both. I don't think it's a problem with the homepods as the play music and other sources fine. I think it's the update in TvOS from 26 to 27 that introduced the bug/problem. I'm going to submit another feedback report with the above details again.
0
0
218
1w
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
3
5
1.3k
1w
Automatic PiP (Picture in Picture) with MusicKit JS in WKWebView
Our iOS app plays Apple Music catalog videos using MusicKit JS v3 in WKWebView. We use a WebView, with the current video visible. Tests are on an iPhone 16e running iOS 26.6.2, built with SDK 27.1. Inline playback and PiP are allowed, background audio is enabled, and the system’s automatic PiP setting is on. A real button click calling video.requestPictureInPicture() successfully opens PiP, which continues playing when returning to the Home Screen. However, leaving the app without opening PiP manually produces audio only. We tested both public video PiP APIs, including an early request from UIScene.willDeactivateNotification. In the latest trace, AVKit reports a visible, supported and eligible source, then reads ForegroundInactive and immediately reports “Failed to start picture in picture.” The video briefly reports PiP mode before returning inline; no PiP window appears. Our WebKit audio session also changes from ambient to auto on deactivation; its contribution has not been isolated. Is automatic inline PiP supported for MusicKit JS videos in WKWebView? Is there a supported configuration or initialization hook to prepare it while the scene is still active? We do not own WebKit’s underlying controller to set canStartPictureInPictureAutomaticallyFromInline. What lifecycle and audio-session setup does Apple recommend? If this behavior is unexpected, which minimal reproduction and diagnostics would help DTS investigate?
3
0
760
3w
Contextual image/text overlays on FairPlay video in Picture in Picture
I'm looking for a supported way to show occasional contextual information over a FairPlay-protected video in system PiP (Picture in Picture) on iOS and macOS. For example, I'd like to briefly display a small icon or image and a short text label to give the viewer useful context. I want to leave the video stream unchanged and keep normal protected playback, with the additional information displayed as a separate UI layer. Standard protected playback in PiP works. My question is how to include that extra UI in the actual system PiP window, rather than only in the player view inside the app. Does AVKit provide a supported way to display and update image/text overlays on top of an AVPlayer-based PiP presentation while preserving FairPlay playback? If not, is this a general limitation of system PiP, or an additional restriction for protected video? What would be the recommended approach for this use case, including images as well as text?
0
0
576
4w
MusicKit JS loses PiP continuity between queued music videos in a background WKWebView
We are trying to play consecutive Apple Music videos in Picture in Picture from a native iOS app. Our minimal reproduction uses one WKWebView, one MusicKit JS v3 instance and one preloaded queue. There are no parallel players or transitions implemented by our app. Setup: Inline playback enabled; allowsPictureInPictureMediaPlayback reads true. UIBackgroundModes includes audio. Initial PiP is opened through a user gesture and works correctly. Reproduced on iOS 26.6.2 and iOS 27.0; failure details vary between runs. await music.setQueue({ musicVideos: ["1445284008", "1258705880", "290361007"] }); await music.play(); The user opens PiP, goes to the Home screen and lets the first video finish naturally. At the queue boundary, MusicKit replaces its HTMLVideoElement. The existing PiP window closes, and the next video does not inherit the presentation. We also tested requesting PiP once on the new element, after it becomes ready, including a variant waiting for playing. In affected background runs, the element briefly reports the WebKit PiP mode, then receives leavepictureinpicture without enterpictureinpicture. The standard requestPictureInPicture() promise remains pending during observation. Native logs report a failed new PiP start while the source scene is backgrounded. The equivalent explicit handoff succeeds in the foreground. In one background configuration, the next video continued audibly, while the PiP window still closed. Explicit native playback audio-session configuration did not resolve the continuity issue. Is there a supported way to: Preserve the underlying media element/PiP presentation across MusicKit queue items? Transfer an existing PiP session to the replacement element while backgrounded? Obtain a reliable error or rejection when native PiP startup fails but the JavaScript promise remains pending? We can provide a minimal reproducer and correlated JavaScript/native logs.
4
0
755
4w
Playing Apple Music / MusicKit videos on Apple TV
Hello, Apple currently does not provide any option to play MusicKit catalog videos through the standard Apple Music API nor any AVPlayer option on any platform. The only solution to play MusicKit videos is to use MusicKit on the Web through a WKWebView. This comes with a lot of constraints for developing native Apps and trying to perfectly integrate with the OS. tvOS do not support WebKit, meaning we can't use a WKWebView and MusicKit on the Web. We have no known way to develop an Apple TV App to play Apple Music videos. Is there any chance Apple will soon : integrate videos in Apple Music API ? or, at least, make WebKit at least available for use with MusicKit on the Web ? Thank you.
1
0
619
Oct ’26
V.app intermittently fails to display embedded artwork in local MP4/M4V TV episodes
We are developing a macOS application that tags local MP4/M4V movie and TV episode files for playback in the macOS TV app. We have encountered an intermittent artwork-display issue that we have been unable to explain from the file contents alone. The observed behavior The files contain embedded TV metadata and JPEG artwork. The files play correctly in the TV app. However, the TV app sometimes displays “No Artwork” for an episode even though the artwork is embedded in the file. More interestingly, the artwork can subsequently appear without modifying the file at all. For example, closing and reopening the TV app, or changing views, can cause artwork that was previously shown as missing to appear. This makes it difficult to determine whether the issue is related to the file itself, TV app indexing, or the TV app's metadata/artwork cache. Controlled experiment We created nine fresh files from three identical sets of TV episodes: 3 files tagged by our application (Flickis) 3 files tagged by a Python-based tagger 3 files tagged by iFlicks The same episodes were used across the three sets. The same artwork was used. All files were imported into the same macOS TV app environment. The artwork behavior was intermittent across all three groups. For example, some files initially displayed artwork and others displayed “No Artwork”. In several cases the artwork appeared later after restarting the TV app, without any modification to the media file. Container structure We also extracted the hierarchical atom structure of all nine files, without analysing payloads or attempting to interpret the results. The three pipelines produce clearly different container structures. The most notable recurring difference is: Flickis:
ftyp → moov → mdat Python tagger:
ftyp → [free] → moov → [free] → mdat iFlicks:
ftyp → free → mdat → mdat → moov Thus, the iFlicks files consistently have the moov atom after the media data, whereas Flickis and the Python tagger place moov before mdat. All three approaches contain embedded artwork (covr) and TV-related metadata. We are deliberately not assuming that moov placement is the cause. It is simply the most obvious systematic structural difference revealed by the experiment. What we would like to understand Does the macOS TV app have specific requirements or expectations regarding the container structure or metadata layout when indexing artwork for local MP4/M4V TV episodes? In particular: Does TV.app rely on a particular moov/mdat arrangement when indexing local media? Are there specific requirements for covr artwork that are not apparent from the public AVFoundation metadata APIs? Does TV.app impose requirements on the hierarchy or placement of iTunes/QuickTime metadata atoms? Are free atoms or padding relevant to artwork indexing? Is artwork indexing performed asynchronously or cached in a way that could explain artwork appearing later without the underlying file changing? Is there a recommended way to generate local MP4/M4V files that maximizes compatibility with TV.app's artwork indexing? We are particularly interested in whether there is a documented or recommended container/metadata layout for local TV episodes imported into the macOS TV app, rather than streamed or purchased content. Any guidance on the expected container structure or the TV app's indexing behavior would be greatly appreciated.
0
0
328
Sep ’26
AVPlayer reports unexpected ~26-hour timeline
Hello, We are seeing an issue where AVPlayer sometimes reports time values close to 95,443 seconds (~26.5 hours), even though the actual content duration is much shorter. For example, in normal cases we see: currentTime: ~355 seconds duration: 6958 seconds seekableTimeRange: 0...6958 seconds However, in some cases AVPlayer reports values around: currentTime: ~95443 seconds duration: ~95449 seconds We checked the underlying MPEG-TS segments with ffprobe. The video and audio PTS values are around 6426 seconds and are consistent with the HLS EXTINF durations, so we do not see a 95,443-second timestamp directly in the media segments. Could you please help us understand: 1- Why can AVPlayer expose values close to 95,443 seconds? 2- Could this be related to MPEG-TS PTS wrap-around or AVFoundation timeline mapping? 3- How does AVPlayer calculate currentTime and AVPlayerItem.duration for HLS MPEG-TS content? Interestingly, Audio Only / Now Playing mode shows the expected timeline, while video playback can expose the incorrect values. Thank you.
0
0
356
Sep ’26
Synchronization between displays (M6/M5 Pro)
Hello, Apple recently announced that M6 and M5 Pro are featuring genlock support through USB-C to enables precise synchronization between a display and camera. Regarding displays : would this be possible to frame-lock several displays with this feature ? I mean outputting through several Thunderbolt and synchronizing every output so they works exactly at the same frequency and show exactly the same frame at the same time. If so, how would this work ? Is there some documentation about genlocking displays ? Thank you for your help.
1
1
256
Sep ’26
Screen capture of other apps without the broadcast picker — any public API?
Two quick questions about ReplayKit on iOS: Is there any public API or entitlement that lets an app capture the screen while another app is in the foreground without presenting RPSystemBroadcastPickerView each session? Is custom-content Picture-in-Picture (AVSampleBufferDisplayLayer with AVPictureInPictureController) considered acceptable for a non-video status overlay, or is Live Activities the intended surface for that? Thanks.
0
0
432
Sep ’26
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
1
0
548
Sep ’26
Is there a supported way to read on-screen text with the user's consent? (worker-safety / income-transparency app)
Hello, I am an independent developer in Argentina building an app for rideshare drivers, and I would like to know what the supported approach is on iOS before I build the wrong thing. The problem When a trip offer appears, the driver sees a fare, a distance and a duration. What they do not see is what is left after the cost of running their own car — fuel, maintenance, and very often the weekly rent they pay for the vehicle. Here those costs are high and change constantly, and drivers routinely accept trips that lose them money without realising it. They have about three seconds to decide, while driving. Our app reads the numbers already visible on the driver's own screen, subtracts that particular driver's cost per kilometre, and shows one figure: what this trip actually leaves them per hour. In our own field measurements with two drivers over several weeks, being able to see that number raised their hourly earnings and — just as important — kept their eyes off mental arithmetic at the wheel. What we will not do, on purpose I want to be explicit about this, because I know other tools in this category cut corners: We never ask for the driver's Uber or DiDi username or password, and we never connect to or automate their account. Some tools do, and that puts the driver's account — their livelihood — at risk of deactivation. We will not build that. We never accept or decline a trip for the driver. The decision and the tap are always theirs. We never store passenger data: no name, no rating, no photo, no exact address. Text recognition runs entirely on device, offline. The captured image is discarded immediately and never leaves the phone or reaches any server. No advertising, no data sales, no third-party trackers. The app does not read anything the driver is not already looking at, it only reads while the driver has explicitly turned it on for their shift, and it exists for one purpose: so that someone working long hours knows what an hour of their work is actually worth. My question On Android this is done with MediaProjection, with explicit user consent each session and a persistent system indicator. On iOS I understand a Broadcast Upload Extension is not intended for background processing, and that there is no overlay across apps. Is there any supported path on iOS for an app to read, with the user's explicit and repeated consent, text the user is already looking at on their own screen, in order to give that same user an immediate assessment of their own work? Accessibility APIs, a system extension point, anything. If there is no supported path, I would rather know now and design the iOS version around what is allowed, instead of shipping something that gets rejected or that quietly breaks the rules. Thank you.
1
0
466
Sep ’26
Feature Request: Enable Multiple Simultaneous iPhone Cameras Through Continuity Camera
I use continuity camera. I use a clone iPhone 17 pro to have a multi-camera podcast setup. CamoCam gets A&V out of sync which creates WAY MORE work. Dear Apple Engineering and Product Development Team, I am requesting an important enhancement to Apple’s Continuity Camera technology. Currently, Continuity Camera limits users to a single iPhone camera connected to a Mac at a time. Apple should expand this functionality to allow two, three, four, or more iPhones to operate simultaneously as independent camera sources. Connect multiple iPhones simultaneously to a single Mac using USB or Wi-Fi. Recognize each iPhone & toggle between cameras. Thank you, Brandon Johnson
Replies
0
Boosts
0
Views
8
Activity
1h
Swift 6: Supported AVCaptureSession actor isolation and MainActor preview attachment
I’m developing a SwiftUI camera preview for iPhone and iPad using Xcode 27.2 and Swift 6.4.2. I’m looking for clarification about the supported concurrency boundary between an actor-owned AVCaptureSession and a MainActor-owned AVCaptureVideoPreviewLayer. Environment Xcode 27.2 (27B52) Apple Swift 6.4.2 iPhoneOS SDK 27.2 Deployment target: iOS 26.0 Swift 6 language mode Complete concurrency checking Default actor isolation: MainActor Approachable Concurrency enabled The isolated compiler reproductions also explicitly enable NonisolatedNonsendingByDefault, InferIsolatedConformances, and MemberImportVisibility. Architecture The intended architecture follows the general pattern demonstrated in Apple’s AVCam sample: A capture actor uses a dedicated DispatchSerialQueue executor. The capture actor creates, configures, starts, and stops AVCaptureSession. A MainActor adapter owns AVCaptureVideoPreviewLayer. The preview layer retains the same session after attachment. The capture actor retains its session reference because it must perform subsequent capture operations. Compiler behavior In a minimal reproduction, transferring the retained session directly to the MainActor adapter: await preview.attach(session)produces: Sources/CaptureActor.swift:19:23: error: sending 'session' risks causing data races [#RegionIsolation::SendingRisksDataRace]The compiler identifies a potential race between actor-isolated and MainActor-isolated uses. This reproduction produces one error and zero warnings. A second reproduction uses a small PreviewSource: Sendable value containing a private AVCaptureSession reference, with @preconcurrency import AVFoundation confined to that source file. Its MainActor method attaches the session to the preview layer. Under otherwise identical compiler settings, this second reproduction compiles with zero diagnostics. We understand that compiler acceptance does not establish runtime safety and that the compatibility mechanism does not itself synchronize access to the retained session. Apple AVCam reference Apple’s official AVCam sample demonstrates a similar interoperability pattern: CaptureService.swift: dedicated capture executor and retained preview source. PreviewSource.swift: compatibility import and MainActor attachment. CameraPreview.swift: preview-layer association. Reference: Apple’s AVCam: Building a Camera App sample. Pinned sample SHA-256: 5a1981f90bb30827121324d4b33198d58de9b9d68e3508b64f0aabc18bbc0e53The sample revision date is not established; we are not assuming this archive is identical to the latest downloadable revision. Questions 1. Supported Swift 6 interoperability pattern Is the narrow PreviewSource compatibility approach demonstrated by AVCam the recommended way to associate an actor-retained AVCaptureSession with a MainActor-owned AVCaptureVideoPreviewLayer under complete Swift 6 concurrency checking? Is there a newer recommended API, annotation, or ownership pattern? 2. Capture and preview synchronization Is the following operation sequence supported? Configure and commit the capture session on the dedicated capture executor. Attach the session to the preview layer on MainActor. Call startRunning() on the capture executor. Perform ordinary preview layout on MainActor while capture runs. Call stopRunning() on the capture executor. Detach the session from the preview layer on MainActor. Which preview-layer or preview-connection mutations, if any, must be excluded while startRunning() or stopRunning() is executing? 3. Supported detachment and release sequence After stopRunning() returns and MainActor executes: previewLayer.session = nilis it supported to release the remaining application-held session references once application-owned observers and asynchronous consumers have terminated? Does AVFoundation require or recommend any additional step before those references are released? We are asking about the supported framework lifecycle contract, not assuming that application-level cancellation drains internal AVFoundation work. Sources AVCam: Building a camera app Overview Apple’s official AVCam sample
Replies
0
Boosts
1
Views
238
Activity
2d
AVCaptureSession multitasking camera access on iPhone without VoIP background mode
I'm developing an iPhone screen-recording application that simultaneously records the front-facing camera and displays the live camera feed using Picture in Picture while the user navigates other apps. The implementation uses AVFoundation (AVCaptureSession) for camera capture and a system Picture in Picture controller for the floating window. I've encountered the following behavior on a physical iPhone: With voip declared in UIBackgroundModes, isMultitaskingCameraAccessSupported returns true, and the front camera continues capturing while another app is foregrounded. Without voip, isMultitaskingCameraAccessSupported returns false. When the application enters the background, the capture session is interrupted and camera frames stop. Screen recording and the Picture in Picture window continue, but the camera feed freezes or becomes blank. Apple App Review has confirmed that declaring voip without actual VoIP functionality is not permitted under Guideline 2.5.4. I understand and respect that restriction and do not want to misuse background modes. My question: Is there a supported public API, entitlement, or configuration that allows a non-VoIP iPhone application to continue capturing front-camera video while another application is in the foreground? Does Picture in Picture provide any supported mechanism for this use case, or is multitasking camera capture unavailable to non-VoIP applications on iPhone? I'm looking for a supported implementation rather than a workaround that violates App Review guidelines. Any guidance from Apple's AVFoundation engineers would be greatly appreciated.
Replies
2
Boosts
0
Views
375
Activity
3d
Thin border around VideoPlayer
A thin gray border appears around assets presented using VideoPlayer. It was tested on iPhone 8, iPhone 13 mini, and iPhone 15, and the border only appeared on the 13 mini. The iOS version didn't matter, we checked the 13 mini on both iOS 18 and iOS 26 and the border appeared on both. I created a minimal version of the app with just the VideoPlayer and the asset and the frame still appeared. We tried different assets and it was visible on all of them. import SwiftUI import AVKit struct ContentView: View { private let player = AVPlayer(url: Bundle.main.url(forResource: "sample", withExtension: "mp4")!) var body: some View { VideoPlayer(player: player) .frame(width: 300, height: 200) } }I found someone that had the same exact problem on Reddit and StackOverflow: https://www.reddit.com/r/swift/comments/8s87hm/weird_borders_around_avplayerviewcontroller/ https://stackoverflow.com/questions/50927561/weird-borders-around-avplayerviewcontroller and their solution included setting: playerLayer.shouldRasterize = true playerLayer.rasterizationScale = UIScreen.main.scaleThis fixed the issue but isn't suitable for our app due to offscreen rendering performance. They also found that issue on other devices, noting: "After testing this code on multiple devices (actual physical devices, not simulators), this bug appears to happen only on Plus devices (I tested it on i8 on iOS 11, i8+ on iOS 11, iX on iOS 11 and i6+ on iOS 10)." The bug doesn't appear on simulator, isn't visible on a screenshot, the frame disappears when using accessibility zoom. When we encoded the video file to Apple ProRes the border disappeared, but the size of the video bumped from 342 KB to 6.3 MB so we couldn't do that. I'm attaching photo from this person's post since our issue looked exactly the same. Has anyone had a similar issue and maybe found another workaround?
Replies
0
Boosts
0
Views
245
Activity
4d
Official API for Apple Log / Apple Log 2 to SDR Rec.709 and HDR HLG Conversion?
Hello, I'm developing a video application for Apple platforms and would like to implement accurate Apple Log and Apple Log 2 color conversion using official Apple APIs. Final Cut Pro and Final Cut Camera support SDR Rec.709 and HDR Rec.2020 HLG monitoring. I have found Apple's official Apple Log white papers and Rec.709 LUTs, but I could not find an official HDR HLG LUT or a public API that provides equivalent color conversion. I have three questions: Does Apple provide a public API through AVFoundation, Core Image, ColorSync, or another framework to convert Apple Log / Apple Log 2 to SDR Rec.709 and HDR Rec.2020 HLG? Are the color transformations used by Final Cut Pro and Final Cut Camera available to third-party developers, either through public APIs or official LUT resources? If no such API exists, is implementing the transfer functions and gamut conversions from Apple's white papers, followed by standard SDR/HDR rendering, the recommended approach? My goal is to preserve the original Apple Log footage while allowing independent SDR/HDR monitoring without modifying the recorded data. Any guidance, documentation, or sample code from Apple engineers would be greatly appreciated. Thank you!
Replies
1
Boosts
0
Views
353
Activity
1w
Audio cut-out
Installed the latest beta releases of TVos and Home pod OS. 27.2 (24k122) The problem of audio cutting out whilst video keeps playing persists. The set up is: Mac with homeshare enabled, MP4 video in TV app on Mac encoded with ffmpeg, Apple tv 4k tvos 27, tv, Two homepods 27, in stereo pair. Problem: When playing a video from the mac on the tv via the appletv on the computers app the audio cuts out whilst the video keeps playing. Pausing and playing the video restores the video only for it to cut-out again and again. Switching audio from homepods to tv speakers there is no cut-out. Other sources play fine through the apple tv, streaming appletv, iplayer etc. The homepods work fine when watching tv with homepods using a satellite box. This only started when the TvOS was upgraded to 27, 26 was fine. It's not a problem with the Macs as I use two, one an old Mac pro and a new Mac mini, the problem is on both. I don't think it's a problem with the homepods as the play music and other sources fine. I think it's the update in TvOS from 26 to 27 that introduced the bug/problem. I'm going to submit another feedback report with the above details again.
Replies
0
Boosts
0
Views
218
Activity
1w
iOS 27 beta: Opening Notification Center pauses AVPlayer playback
Since iOS 27 beta we are seeing a behavior change in our video streaming app and I would like to know whether others can reproduce it. Behavior on iOS 27 beta: Fully opening the Notification Center pauses playback. Audio continues for about 5 more seconds, then the app is suspended. Closing the Notification Center leaves the player paused. On iOS 26 the same build keeps playing in this situation. Setup: AVQueuePlayer playing video with audio, not muted audiovisualBackgroundPlaybackPolicy = .automatic (default) AVAudioSession category .playback, UIBackgroundModes: audio entitlement AVPictureInPictureController attached with canStartPictureInPictureAutomaticallyFromInline = true Filed as FB23893965 Questions: Can anyone reproduce this on iOS 27 beta? Is this intentional or a regression?
Replies
3
Boosts
5
Views
1.3k
Activity
1w
Automatic PiP (Picture in Picture) with MusicKit JS in WKWebView
Our iOS app plays Apple Music catalog videos using MusicKit JS v3 in WKWebView. We use a WebView, with the current video visible. Tests are on an iPhone 16e running iOS 26.6.2, built with SDK 27.1. Inline playback and PiP are allowed, background audio is enabled, and the system’s automatic PiP setting is on. A real button click calling video.requestPictureInPicture() successfully opens PiP, which continues playing when returning to the Home Screen. However, leaving the app without opening PiP manually produces audio only. We tested both public video PiP APIs, including an early request from UIScene.willDeactivateNotification. In the latest trace, AVKit reports a visible, supported and eligible source, then reads ForegroundInactive and immediately reports “Failed to start picture in picture.” The video briefly reports PiP mode before returning inline; no PiP window appears. Our WebKit audio session also changes from ambient to auto on deactivation; its contribution has not been isolated. Is automatic inline PiP supported for MusicKit JS videos in WKWebView? Is there a supported configuration or initialization hook to prepare it while the scene is still active? We do not own WebKit’s underlying controller to set canStartPictureInPictureAutomaticallyFromInline. What lifecycle and audio-session setup does Apple recommend? If this behavior is unexpected, which minimal reproduction and diagnostics would help DTS investigate?
Replies
3
Boosts
0
Views
760
Activity
3w
Contextual image/text overlays on FairPlay video in Picture in Picture
I'm looking for a supported way to show occasional contextual information over a FairPlay-protected video in system PiP (Picture in Picture) on iOS and macOS. For example, I'd like to briefly display a small icon or image and a short text label to give the viewer useful context. I want to leave the video stream unchanged and keep normal protected playback, with the additional information displayed as a separate UI layer. Standard protected playback in PiP works. My question is how to include that extra UI in the actual system PiP window, rather than only in the player view inside the app. Does AVKit provide a supported way to display and update image/text overlays on top of an AVPlayer-based PiP presentation while preserving FairPlay playback? If not, is this a general limitation of system PiP, or an additional restriction for protected video? What would be the recommended approach for this use case, including images as well as text?
Replies
0
Boosts
0
Views
576
Activity
4w
MusicKit JS loses PiP continuity between queued music videos in a background WKWebView
We are trying to play consecutive Apple Music videos in Picture in Picture from a native iOS app. Our minimal reproduction uses one WKWebView, one MusicKit JS v3 instance and one preloaded queue. There are no parallel players or transitions implemented by our app. Setup: Inline playback enabled; allowsPictureInPictureMediaPlayback reads true. UIBackgroundModes includes audio. Initial PiP is opened through a user gesture and works correctly. Reproduced on iOS 26.6.2 and iOS 27.0; failure details vary between runs. await music.setQueue({ musicVideos: ["1445284008", "1258705880", "290361007"] }); await music.play(); The user opens PiP, goes to the Home screen and lets the first video finish naturally. At the queue boundary, MusicKit replaces its HTMLVideoElement. The existing PiP window closes, and the next video does not inherit the presentation. We also tested requesting PiP once on the new element, after it becomes ready, including a variant waiting for playing. In affected background runs, the element briefly reports the WebKit PiP mode, then receives leavepictureinpicture without enterpictureinpicture. The standard requestPictureInPicture() promise remains pending during observation. Native logs report a failed new PiP start while the source scene is backgrounded. The equivalent explicit handoff succeeds in the foreground. In one background configuration, the next video continued audibly, while the PiP window still closed. Explicit native playback audio-session configuration did not resolve the continuity issue. Is there a supported way to: Preserve the underlying media element/PiP presentation across MusicKit queue items? Transfer an existing PiP session to the replacement element while backgrounded? Obtain a reliable error or rejection when native PiP startup fails but the JavaScript promise remains pending? We can provide a minimal reproducer and correlated JavaScript/native logs.
Replies
4
Boosts
0
Views
755
Activity
4w
Playing Apple Music / MusicKit videos on Apple TV
Hello, Apple currently does not provide any option to play MusicKit catalog videos through the standard Apple Music API nor any AVPlayer option on any platform. The only solution to play MusicKit videos is to use MusicKit on the Web through a WKWebView. This comes with a lot of constraints for developing native Apps and trying to perfectly integrate with the OS. tvOS do not support WebKit, meaning we can't use a WKWebView and MusicKit on the Web. We have no known way to develop an Apple TV App to play Apple Music videos. Is there any chance Apple will soon : integrate videos in Apple Music API ? or, at least, make WebKit at least available for use with MusicKit on the Web ? Thank you.
Replies
1
Boosts
0
Views
619
Activity
Oct ’26
V.app intermittently fails to display embedded artwork in local MP4/M4V TV episodes
We are developing a macOS application that tags local MP4/M4V movie and TV episode files for playback in the macOS TV app. We have encountered an intermittent artwork-display issue that we have been unable to explain from the file contents alone. The observed behavior The files contain embedded TV metadata and JPEG artwork. The files play correctly in the TV app. However, the TV app sometimes displays “No Artwork” for an episode even though the artwork is embedded in the file. More interestingly, the artwork can subsequently appear without modifying the file at all. For example, closing and reopening the TV app, or changing views, can cause artwork that was previously shown as missing to appear. This makes it difficult to determine whether the issue is related to the file itself, TV app indexing, or the TV app's metadata/artwork cache. Controlled experiment We created nine fresh files from three identical sets of TV episodes: 3 files tagged by our application (Flickis) 3 files tagged by a Python-based tagger 3 files tagged by iFlicks The same episodes were used across the three sets. The same artwork was used. All files were imported into the same macOS TV app environment. The artwork behavior was intermittent across all three groups. For example, some files initially displayed artwork and others displayed “No Artwork”. In several cases the artwork appeared later after restarting the TV app, without any modification to the media file. Container structure We also extracted the hierarchical atom structure of all nine files, without analysing payloads or attempting to interpret the results. The three pipelines produce clearly different container structures. The most notable recurring difference is: Flickis:
ftyp → moov → mdat Python tagger:
ftyp → [free] → moov → [free] → mdat iFlicks:
ftyp → free → mdat → mdat → moov Thus, the iFlicks files consistently have the moov atom after the media data, whereas Flickis and the Python tagger place moov before mdat. All three approaches contain embedded artwork (covr) and TV-related metadata. We are deliberately not assuming that moov placement is the cause. It is simply the most obvious systematic structural difference revealed by the experiment. What we would like to understand Does the macOS TV app have specific requirements or expectations regarding the container structure or metadata layout when indexing artwork for local MP4/M4V TV episodes? In particular: Does TV.app rely on a particular moov/mdat arrangement when indexing local media? Are there specific requirements for covr artwork that are not apparent from the public AVFoundation metadata APIs? Does TV.app impose requirements on the hierarchy or placement of iTunes/QuickTime metadata atoms? Are free atoms or padding relevant to artwork indexing? Is artwork indexing performed asynchronously or cached in a way that could explain artwork appearing later without the underlying file changing? Is there a recommended way to generate local MP4/M4V files that maximizes compatibility with TV.app's artwork indexing? We are particularly interested in whether there is a documented or recommended container/metadata layout for local TV episodes imported into the macOS TV app, rather than streamed or purchased content. Any guidance on the expected container structure or the TV app's indexing behavior would be greatly appreciated.
Replies
0
Boosts
0
Views
328
Activity
Sep ’26
AVPlayer reports unexpected ~26-hour timeline
Hello, We are seeing an issue where AVPlayer sometimes reports time values close to 95,443 seconds (~26.5 hours), even though the actual content duration is much shorter. For example, in normal cases we see: currentTime: ~355 seconds duration: 6958 seconds seekableTimeRange: 0...6958 seconds However, in some cases AVPlayer reports values around: currentTime: ~95443 seconds duration: ~95449 seconds We checked the underlying MPEG-TS segments with ffprobe. The video and audio PTS values are around 6426 seconds and are consistent with the HLS EXTINF durations, so we do not see a 95,443-second timestamp directly in the media segments. Could you please help us understand: 1- Why can AVPlayer expose values close to 95,443 seconds? 2- Could this be related to MPEG-TS PTS wrap-around or AVFoundation timeline mapping? 3- How does AVPlayer calculate currentTime and AVPlayerItem.duration for HLS MPEG-TS content? Interestingly, Audio Only / Now Playing mode shows the expected timeline, while video playback can expose the incorrect values. Thank you.
Replies
0
Boosts
0
Views
356
Activity
Sep ’26
Synchronization between displays (M6/M5 Pro)
Hello, Apple recently announced that M6 and M5 Pro are featuring genlock support through USB-C to enables precise synchronization between a display and camera. Regarding displays : would this be possible to frame-lock several displays with this feature ? I mean outputting through several Thunderbolt and synchronizing every output so they works exactly at the same frequency and show exactly the same frame at the same time. If so, how would this work ? Is there some documentation about genlocking displays ? Thank you for your help.
Replies
1
Boosts
1
Views
256
Activity
Sep ’26
Screen capture of other apps without the broadcast picker — any public API?
Two quick questions about ReplayKit on iOS: Is there any public API or entitlement that lets an app capture the screen while another app is in the foreground without presenting RPSystemBroadcastPickerView each session? Is custom-content Picture-in-Picture (AVSampleBufferDisplayLayer with AVPictureInPictureController) considered acceptable for a non-video status overlay, or is Live Activities the intended surface for that? Thanks.
Replies
0
Boosts
0
Views
432
Activity
Sep ’26
AVAssetDownloadConfiguration does not download supplemental tracks on iOS 16
We're downloading HLS content for offline playback using AVAssetDownloadConfiguration. It's critical for our app that all subtitle tracks are downloaded together with the video, so users can watch any subtitle and audio option while offline. On iOS 16, not all tracks are downloaded, and when the user goes offline, playback fails with NSURLErrorNotConnectedToInternet (error -1009) because AVFoundation tries to fetch the missing subtitle segments over the network. On iOS 17 and later, everything works as expected — all subtitle and audio tracks are downloaded. However, on iOS 16 we observe two issues: First issue: Supplemental tracks are not downloaded on iOS 16 (device) Only the default audio track is downloaded. All non-default supplemental tracks — both subtitles and additional audio tracks — exhibit the same problem: Some supplemental tracks listed in the manifest don't even get a directory created in the .movpkg package Some supplemental tracks do get a directory created, but it contains only StreamInfoBoot.xml and StreamInfoRoot.xml — no actual segment data (.frag files) is downloaded. The StreamInfoBoot.xml for these directories shows NO, 0, and 0, with Supplemental Additionally, a Manifest.m3u8 stream directory is created with Complete=NO and no segment data. On iOS 17+ devices, all supplemental track directories are created and all contain segment data. Second issue: Downloads fail entirely on iOS 16 Simulator We receive the following error: Error Domain=NSURLErrorDomain Code=-16090 UserInfo={ NSLocalizedDescription=The operation could not be completed, _NSURLErrorFailingURLSessionTaskErrorKey=BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=( "BackgroundAVAssetDownloadTask <DEC0D555-6C05-416C-91B8-F490F5A29046>.<3>" ), NSLocalizedFailureReason=An unknown error occurred (-16090) } On iOS 17+ Simulators, downloads work without issues. Our setup: let urlAsset = AVURLAsset(url: manifestURL) let preferredMediaSelection = try await urlAsset.load(.allMediaSelections) let config = AVAssetDownloadConfiguration(asset: urlAsset, title: title) config.primaryContentConfiguration.mediaSelections = preferredMediaSelection let task = assetDownloadURLSession.makeAssetDownloadTask(downloadConfiguration: config) task.resume() Example HLS manifest: #EXTM3U #EXT-X-VERSION:7 #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French",LANGUAGE="fr",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_25/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="French (forced)",LANGUAGE="fr-x-cc",DEFAULT=NO,AUTOSELECT=YES,FORCED=YES,URI="s_31/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",LANGUAGE="en",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_37/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Kazakh",LANGUAGE="kk",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_43/stream.m3u8" #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Uzbek",LANGUAGE="uz",DEFAULT=NO,AUTOSELECT=YES,FORCED=NO,URI="s_49/stream.m3u8" #EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aud_ec3",NAME="French 5.1",LANGUAGE="fr",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="6",URI="a_18/stream.m3u8" #EXT-X-STREAM-INF:AUDIO="aud_ec3",SUBTITLES="subs",BANDWIDTH=1931408,VIDEO-RANGE=SDR,CODECS="hvc1.1.6.L90.90,ec-3",RESOLUTION=1024x428,FRAME-RATE=25,HDCP-LEVEL=NONE v_6/stream.m3u8 #EXT-X-SESSION-KEY:METHOD=SAMPLE-AES,URI="skd://XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",KEYFORMAT="com.apple.streamingkeydelivery",KEYFORMATVERSIONS="1" Questions: What is the correct way to make sure all supplemental tracks (subtitles, non-default audio) are actually downloaded on an iOS 16 real device? We pass asset.load(.allMediaSelections) to primaryContentConfiguration.mediaSelections — is there a different configuration, or should we use auxiliaryContentConfigurations, or something else entirely? Is there any way to make asset downloads work on the iOS 16 Simulator at all? We always get NSURLErrorDomain error -16090, while iOS 17+ Simulators work fine. If the supplemental tracks download bug on iOS 16 cannot be fixed or worked around from the app side, do you have any recommended approach for handling incomplete .movpkg packages at offline playback time?
Replies
1
Boosts
0
Views
548
Activity
Sep ’26
Is there a supported way to read on-screen text with the user's consent? (worker-safety / income-transparency app)
Hello, I am an independent developer in Argentina building an app for rideshare drivers, and I would like to know what the supported approach is on iOS before I build the wrong thing. The problem When a trip offer appears, the driver sees a fare, a distance and a duration. What they do not see is what is left after the cost of running their own car — fuel, maintenance, and very often the weekly rent they pay for the vehicle. Here those costs are high and change constantly, and drivers routinely accept trips that lose them money without realising it. They have about three seconds to decide, while driving. Our app reads the numbers already visible on the driver's own screen, subtracts that particular driver's cost per kilometre, and shows one figure: what this trip actually leaves them per hour. In our own field measurements with two drivers over several weeks, being able to see that number raised their hourly earnings and — just as important — kept their eyes off mental arithmetic at the wheel. What we will not do, on purpose I want to be explicit about this, because I know other tools in this category cut corners: We never ask for the driver's Uber or DiDi username or password, and we never connect to or automate their account. Some tools do, and that puts the driver's account — their livelihood — at risk of deactivation. We will not build that. We never accept or decline a trip for the driver. The decision and the tap are always theirs. We never store passenger data: no name, no rating, no photo, no exact address. Text recognition runs entirely on device, offline. The captured image is discarded immediately and never leaves the phone or reaches any server. No advertising, no data sales, no third-party trackers. The app does not read anything the driver is not already looking at, it only reads while the driver has explicitly turned it on for their shift, and it exists for one purpose: so that someone working long hours knows what an hour of their work is actually worth. My question On Android this is done with MediaProjection, with explicit user consent each session and a persistent system indicator. On iOS I understand a Broadcast Upload Extension is not intended for background processing, and that there is no overlay across apps. Is there any supported path on iOS for an app to read, with the user's explicit and repeated consent, text the user is already looking at on their own screen, in order to give that same user an immediate assessment of their own work? Accessibility APIs, a system extension point, anything. If there is no supported path, I would rather know now and design the iOS version around what is allowed, instead of shipping something that gets rejected or that quietly breaks the rules. Thank you.
Replies
1
Boosts
0
Views
466
Activity
Sep ’26