Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

All subtopics
Posts under Media Technologies topic

Post

Replies

Boosts

Views

Activity

ARKit AR View on both screens
Hello everyone! Does the ARKit ARSession qualify for an active camera session? Reading through the documentation, I understand that showing content on both iPhone Duo screens at the same time is only possible by using CameraCaptureAccessory. ARSession drives the camera but doesn't vend an AVCaptureSession that the app owns, so it isn't clear whether the availability check keys off app-owned AVCaptureSession state specifically, or off the camera actively streaming for the app. If it is currently not possible, do you see any way or workaround to show the camera stream on the outside display while the inner display is displaying an AR view? Best regards, André N.
4
1
237
Sep ’26
Shutter button
Any tips on positioning a shutter button on camera screen (fullscreen preview) when device is in half-open state? As well as other centred controls (camera mode, etc)? How to handle hinge area updates? Thanks
2
0
157
Sep ’26
CameraCaptureAccessory: orientation of the accessory scene in the partially folded (laptop) pose
We present custom content on the outer display through CameraCaptureAccessory while a capture session runs (iOS 27.1). Our app is used in the partially folded "laptop" pose: the inner display is portrait with a horizontal fold, controls below the fold, content above it. The person in front of the outer camera sits opposite the device, so for them the outer display's long edge is horizontal. In Simulator (iPhone Duo, iOS 27.1) the accessory scene is always laid out as 466×678 pt — portrait — regardless of the pose, and regardless of the app's UISupportedInterfaceOrientations. Our landscape content is therefore letterboxed into a narrow band instead of filling the display. On device, what interface orientation does the accessory scene get in the partially folded pose? Does the system rotate it to match the physical orientation of the outer display, or is it always the outer display's portrait orientation? Is Simulator's behaviour here representative, or is accessory-scene orientation one of the things that only behaves correctly on device? If we have to compensate ourselves, is there a supported way to learn the outer display's physical orientation from inside the accessory scene — reserved regions, the scene's interfaceOrientation, something else? We would rather not hardcode a rotation. Does UIRequiresFullScreen (deprecated in iOS 26) or the app's supported interface orientations constrain the accessory scene in any way?
1
0
197
Sep ’26
ARKit behaviour across the fold
Hi, Can you talk about what happens with ARKit by default when the unfolding occurs in a normal ARWorldTrackingConfiguration - does the ARSession pause at all or is tracking completely uninterrupted? Does it swap cameras when using ARFaceTrackingConfiguration! Thanks.
1
1
183
Sep ’26
Is the inner under-display camera a supported source for real-time 2D body-pose analysis?
Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a few meters away, about 30 fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera. Two things are unclear from the camera tech talk and the specs: Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews? Thanks
1
0
190
Sep ’26
All cameras being used
Can all the cameras on iPhone Duo be active while an app tries to translate sign language using the front facing camera and back camera to monitor hand movements and show the translation with action in the expanded display with its camera on?
1
0
168
Sep ’26
Constituent Photo Delivery
When iPhone Duo is fully open, so the outer front camera and the rear cameras face the same direction, is there a virtual AVCaptureDevice whose constituentDevices include both the outer front camera and the rear 48 MP wide camera, so that virtual device constituent photo delivery that could produce one photo from each of those cameras simultaneously, as is possible on existing iPhones with two rear cameras, using isVirtualDeviceConstituentPhotoDeliveryEnabled / virtualDeviceConstituentPhotoDeliveryEnabledDevices?
1
0
164
Sep ’26
Camera positioning
In our app we assume that user positioned the smartphone the way when the front camera (120fps) is on the left and the screen position is landscape (for watching video). We use it in our ML model. Can we adopt it somehow on iPhone Duo?
4
0
252
Sep ’26
Fixed Interface Orientation
Camera apps commonly use portrait-only orientation specifically to prevent the interface from rotating. Otherwise, toolbars, the preview aspect ratio, and camera frames all need to be reorganized just to undo the rotation. On the iPhone Duo inner display, portrait-only no longer prevents rotation. Please provide a UIRequiresFixedInterfaceOrientation key! :-)
2
1
361
Sep ’26
Simultaneous capture and fold transitions in a live-streaming app on iPhone Duo
I build Lumecast, a live-streaming app (custom AVCapture pipeline feeding WebRTC). Three questions:   1. Does iPhone Duo support AVCaptureMultiCamSession — specifically front + rear simultaneously? Our co-casting feature would let a solo host fill two panes with their face and what they're looking at. How should a continuous capture session (live broadcast, can't pause) respond to camera direction changes as the device opens and closes? Is there a recommended pattern for swapping to the Virtual Front Camera mid-session without a visible gap for viewers?  3. Which cameras are available to third-party apps in each posture (folded / half-open / fully open), and does availability change at a specific hinge state? Goal: a host folds or unfolds mid-broadcast and their audience never sees a dropped frame.
3
0
383
Sep ’26
Is AVCaptureMultiCamSession supported on iPhone Duo
is AVCaptureMultiCamSession supported on iPhone Duo — specifically streaming two cameras that face opposite directions simultaneously (e.g. .builtInInnerUltraWideCamera + .builtInOuterUltraWideCamera, or the Virtual Front Camera plus a backward-facing camera)? I noticed the article recommends single-input reconfiguration over multi-cam; is that a performance recommendation, or is opposite-direction multi-cam unavailable on this device? Our use case is a live co-cast where one host fills two video panes: their face and what they're looking at.
0
0
219
Sep ’26
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
3
3
1.4k
Sep ’26
Apple Music data not syncing via MusicKit / Apple Music API
Hi, experiencing an issue with Apple Music syncing in our app. Apple Music data hasn’t been syncing for a while. We’ve investigated the issue on our side and haven’t found anything that could explain it, so we’d like to check whether there are any known issues or recent changes affecting Apple Music API data. Could someone from the Apple Music/MusicKit team help us investigate this? We can provide affected user examples, timestamps, API responses, and any other information needed.
0
0
117
Sep ’26
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
3
2
1.8k
Sep ’26
ARKit AR View on both screens
Hello everyone! Does the ARKit ARSession qualify for an active camera session? Reading through the documentation, I understand that showing content on both iPhone Duo screens at the same time is only possible by using CameraCaptureAccessory. ARSession drives the camera but doesn't vend an AVCaptureSession that the app owns, so it isn't clear whether the availability check keys off app-owned AVCaptureSession state specifically, or off the camera actively streaming for the app. If it is currently not possible, do you see any way or workaround to show the camera stream on the outside display while the inner display is displaying an AR view? Best regards, André N.
Replies
4
Boosts
1
Views
237
Activity
Sep ’26
Shutter button
Any tips on positioning a shutter button on camera screen (fullscreen preview) when device is in half-open state? As well as other centred controls (camera mode, etc)? How to handle hinge area updates? Thanks
Replies
2
Boosts
0
Views
157
Activity
Sep ’26
CameraCaptureAccessory: orientation of the accessory scene in the partially folded (laptop) pose
We present custom content on the outer display through CameraCaptureAccessory while a capture session runs (iOS 27.1). Our app is used in the partially folded "laptop" pose: the inner display is portrait with a horizontal fold, controls below the fold, content above it. The person in front of the outer camera sits opposite the device, so for them the outer display's long edge is horizontal. In Simulator (iPhone Duo, iOS 27.1) the accessory scene is always laid out as 466×678 pt — portrait — regardless of the pose, and regardless of the app's UISupportedInterfaceOrientations. Our landscape content is therefore letterboxed into a narrow band instead of filling the display. On device, what interface orientation does the accessory scene get in the partially folded pose? Does the system rotate it to match the physical orientation of the outer display, or is it always the outer display's portrait orientation? Is Simulator's behaviour here representative, or is accessory-scene orientation one of the things that only behaves correctly on device? If we have to compensate ourselves, is there a supported way to learn the outer display's physical orientation from inside the accessory scene — reserved regions, the scene's interfaceOrientation, something else? We would rather not hardcode a rotation. Does UIRequiresFullScreen (deprecated in iOS 26) or the app's supported interface orientations constrain the accessory scene in any way?
Replies
1
Boosts
0
Views
197
Activity
Sep ’26
ARKit behaviour across the fold
Hi, Can you talk about what happens with ARKit by default when the unfolding occurs in a normal ARWorldTrackingConfiguration - does the ARSession pause at all or is tracking completely uninterrupted? Does it swap cameras when using ARFaceTrackingConfiguration! Thanks.
Replies
1
Boosts
1
Views
183
Activity
Sep ’26
Is the inner under-display camera a supported source for real-time 2D body-pose analysis?
Hi! We run continuous 2D human body-pose estimation on a front-facing camera feed: one person, full body in frame, standing a few meters away, about 30 fps. With iPhone Duo open, the camera that faces the person looking at the inner display is the under-display inner camera (builtInInnerUltraWideCamera). Published specs cap it at 1080p at 30 or 60 fps, and it is presented as the FaceTime camera. Two things are unclear from the camera tech talk and the specs: Is that 1080p format a native readout we can feed to Vision body pose, or a preview-oriented stream whose effective detail is too low for full-body pose at that distance? Is third-party use of this camera supported for a long-running analysis session, or is it intended only for video calls and live previews? Thanks
Replies
1
Boosts
0
Views
190
Activity
Sep ’26
All cameras being used
Can all the cameras on iPhone Duo be active while an app tries to translate sign language using the front facing camera and back camera to monitor hand movements and show the translation with action in the expanded display with its camera on?
Replies
1
Boosts
0
Views
168
Activity
Sep ’26
Constituent Photo Delivery
When iPhone Duo is fully open, so the outer front camera and the rear cameras face the same direction, is there a virtual AVCaptureDevice whose constituentDevices include both the outer front camera and the rear 48 MP wide camera, so that virtual device constituent photo delivery that could produce one photo from each of those cameras simultaneously, as is possible on existing iPhones with two rear cameras, using isVirtualDeviceConstituentPhotoDeliveryEnabled / virtualDeviceConstituentPhotoDeliveryEnabledDevices?
Replies
1
Boosts
0
Views
164
Activity
Sep ’26
Camera positioning
In our app we assume that user positioned the smartphone the way when the front camera (120fps) is on the left and the screen position is landscape (for watching video). We use it in our ML model. Can we adopt it somehow on iPhone Duo?
Replies
4
Boosts
0
Views
252
Activity
Sep ’26
Fixed Interface Orientation
Camera apps commonly use portrait-only orientation specifically to prevent the interface from rotating. Otherwise, toolbars, the preview aspect ratio, and camera frames all need to be reorganized just to undo the rotation. On the iPhone Duo inner display, portrait-only no longer prevents rotation. Please provide a UIRequiresFixedInterfaceOrientation key! :-)
Replies
2
Boosts
1
Views
361
Activity
Sep ’26
Duo Camera
When iphone duo opens and closes, how does the camera automatically decide which camera to use, and what new possibilities does this create for users?
Replies
1
Boosts
0
Views
73
Activity
Sep ’26
Simultaneous capture and fold transitions in a live-streaming app on iPhone Duo
I build Lumecast, a live-streaming app (custom AVCapture pipeline feeding WebRTC). Three questions:   1. Does iPhone Duo support AVCaptureMultiCamSession — specifically front + rear simultaneously? Our co-casting feature would let a solo host fill two panes with their face and what they're looking at. How should a continuous capture session (live broadcast, can't pause) respond to camera direction changes as the device opens and closes? Is there a recommended pattern for swapping to the Virtual Front Camera mid-session without a visible gap for viewers?  3. Which cameras are available to third-party apps in each posture (folded / half-open / fully open), and does availability change at a specific hinge state? Goal: a host folds or unfolds mid-broadcast and their audience never sees a dropped frame.
Replies
3
Boosts
0
Views
383
Activity
Sep ’26
Is AVCaptureMultiCamSession supported on iPhone Duo
is AVCaptureMultiCamSession supported on iPhone Duo — specifically streaming two cameras that face opposite directions simultaneously (e.g. .builtInInnerUltraWideCamera + .builtInOuterUltraWideCamera, or the Virtual Front Camera plus a backward-facing camera)? I noticed the article recommends single-input reconfiguration over multi-cam; is that a performance recommendation, or is opposite-direction multi-cam unavailable on this device? Our use case is a live co-cast where one host fills two video panes: their face and what they're looking at.
Replies
0
Boosts
0
Views
219
Activity
Sep ’26
Where is the API for variable aperture of the iPhone 18 Pro?
According to Apple's press release (https://www.apple.com/pt/newsroom/2026/09/apple-debuts-iphone-18-pro-and-iphone-18-pro-max/): Variable aperture also provides more control for creative pros by giving them the ability to manually adjust any of the four aperture settings in the Camera app, and an API is available to developers for even more control across the aperture range in their apps. Can a DTS engineer specify the quoted API please? I can't see it anywhere in the documentation yet. What does it mean "even more control across the aperture range"? Will it be possible to set the aperture in a 0.0-1.0 range?
Replies
3
Boosts
3
Views
1.4k
Activity
Sep ’26
Apple Music data not syncing via MusicKit / Apple Music API
Hi, experiencing an issue with Apple Music syncing in our app. Apple Music data hasn’t been syncing for a while. We’ve investigated the issue on our side and haven’t found anything that could explain it, so we’d like to check whether there are any known issues or recent changes affecting Apple Music API data. Could someone from the Apple Music/MusicKit team help us investigate this? We can provide affected user examples, timestamps, API responses, and any other information needed.
Replies
0
Boosts
0
Views
117
Activity
Sep ’26
Picture-In-Picture not starting with Custom Player
Team, I am trying to start PictureInPicture for my video from custom player as mentioned in the following link, but I am getting an error Steps: Created a player using AVPlayer. Initiated AVPictureInPictureController with AVPlayerLayer Added observer on PIP button enable status When PIP button is tapped executed pipController.startPictureInPicture() statement. PIP is not getting started & getting following error Project created in Xcode11 & currently using Xcode13.2 Error Domain=AVKitErrorDomain Code=-1001 "Failed to start picture in picture." UserInfo={NSLocalizedDescription=Failed to start picture in picture., NSLocalizedFailureReason=The UIScene for the content source has an activation state other than UISceneActivationStateForegroundActive, which is not allowed.} I didn't get any solutions online, plz help me on this issue. FYI: PIP button/mode is working if I use AVPlayerViewController(with default playerItem buttons) instead of AVPlayer & AVPictureInPictureController with custom controls Thanks in advance. Adopting Picture in Picture in a Custom Player
Replies
3
Boosts
2
Views
1.8k
Activity
Sep ’26