Post

Replies

Boosts

Views

Activity

Reply to iPhone 18 Pro: First Core ML load on Neural Engine up to 65× slower than on iPhone 17 Pro
I'm actually the author of the post you linked! I've also contacted DTS about this, using one of our company's two annual code-level support requests lol. From my limited testing, models with large activation maps seem to be particularly affected, or at least those were the ones I had the most trouble with. I also had an agent help me dig into the Instruments call tree for ANECompilerService. On the iPhone 18 Pro, the compiler consumed around 68.6 CPU-seconds, compared to roughly 1.2–1.6 on the iPhone 17e. Some of the major hotspots were: EvaluateComputeCostForTileHeightCandidates (~20.4 CPU-s) TileWithGlobalRefinement (~15.6 CPU-s) ZinCpBasedAllocator::Execute (~15.6 CPU-s) DetermineClusteringFromCostModelDynamicProgramming (~12.8 CPU-s) These are call-tree measurements, so some of the timings may overlap. From what I can tell, a significant amount of time is being spent exploring and evaluating different optimization strategies during ANE compilation. My suspicion is that the compiler might be exploring a much larger optimization space on the newer hardware, though I haven't established why. Unfortunately, the only workaround I've found so far is to avoid the ANE entirely lol. I'm currently running the affected models on the GPU instead, which obviously isn't ideal. I'd also be interested to know if anyone has found a better temporary workaround. Hope this helps narrow things down a bit!
Topic: Machine Learning & AI SubTopic: Core ML Tags:
3w
Reply to iPhone 18 Pro: First Core ML load on Neural Engine up to 65× slower than on iPhone 17 Pro
I'm actually the author of the post you linked! I've also contacted DTS about this, using one of our company's two annual code-level support requests lol. From my limited testing, models with large activation maps seem to be particularly affected, or at least those were the ones I had the most trouble with. I also had an agent help me dig into the Instruments call tree for ANECompilerService. On the iPhone 18 Pro, the compiler consumed around 68.6 CPU-seconds, compared to roughly 1.2–1.6 on the iPhone 17e. Some of the major hotspots were: EvaluateComputeCostForTileHeightCandidates (~20.4 CPU-s) TileWithGlobalRefinement (~15.6 CPU-s) ZinCpBasedAllocator::Execute (~15.6 CPU-s) DetermineClusteringFromCostModelDynamicProgramming (~12.8 CPU-s) These are call-tree measurements, so some of the timings may overlap. From what I can tell, a significant amount of time is being spent exploring and evaluating different optimization strategies during ANE compilation. My suspicion is that the compiler might be exploring a much larger optimization space on the newer hardware, though I haven't established why. Unfortunately, the only workaround I've found so far is to avoid the ANE entirely lol. I'm currently running the affected models on the GPU instead, which obviously isn't ideal. I'd also be interested to know if anyone has found a better temporary workaround. Hope this helps narrow things down a bit!
Topic: Machine Learning & AI SubTopic: Core ML Tags:
Replies
Boosts
Views
Activity
3w