Communication Safety

RSS for tag

Communication Safety helps families to keep children safe, by warning children and providing helpful resources if they receive, or attempt to share photos that likely contain nudity.

Posts under Communication Safety tag

3 Posts

Post

Replies

Boosts

Views

Activity

Adopting Trust Insights in a global app: what works, what I measured, and what I could not find
I have been working through what it would take for a real app, say a bank's, to adopt Trust Insights in a way that works for customers around the world, using only what has been published so far: session 379, the framework documentation and the support article. While the client side works well, there is information a production app needs that I could not find. So I am posting this for two reasons. Some of what I learned may save you time, and since the rest needs an answer from Apple, I would like to have those questions answered in one place. If you would like to see working code, the sample project I used keeps all of the API in one file. What I measured Adding the Trust Insights capability in Xcode was all I needed to get the whole flow running on a device: the permission alert, the evaluation and the report. The simulator returns .unavailable, so you will need a device to test any of this. There are two switches to keep in mind, the system one at Settings > Privacy & Security > Impersonation Risk Detection, and one for each app underneath it. This is what I got from authorizationStatus(for:) in a development build: With the setting turned off, for the app or for everything, it returned .denied right away, even though the support article says changes might take up to 24 hours. It was still .denied 24 hours later. When I turned the setting back on, it returned .authorized right away. After deleting and reinstalling the app it returned .denied, and did not go back to .notDetermined. So while the documentation describes .denied as the person refusing permission, in practice it also covers the setting being off, and you should plan for customers who will never see your permission alert. Since I could not find an API that opens that page in Settings, all you can do is tell them where it is, and you should call it Impersonation Risk Detection because that is the name they will see there. What I would do today Treat having no evaluation as a normal path. Between permissions, availability, the network and server errors, some of your customers will not get a result, and in all of those cases your existing checks should decide and the transaction should carry on as it did before. Start the evaluation early. The session says it takes a couple of seconds and the documentation says several, so I start it when the screen appears, and a production app should also stop waiting after a deadline of its own. An evaluation that comes back too late still has to be reported, as .notUsedError. Call reportConsumption once for every evaluation, on every path, including a transaction the person cancels (.notUsedNotNeeded). The documentation says that skipping it may result in rate limiting or losing access to the service, so this is not something to leave for later. Until the server side details are available, only use the result to add friction. A decision made in the app can be changed by anyone who can change the app, which does not matter for a warning, but it does matter if unknown lets someone skip a check. Keep the eventID with your transaction records, even if you have no way of using it yet, because that is what the documentation says identifies the evaluation when fraud is discovered later. If your fraud team wants to see numbers before anything changes for customers, .usedEvaluationOnly is the documented way to report an evaluation that had no effect on a decision. For UI work, put your decision behind a type of your own and drive the warning screens from a debug switch, because there is no way to create the conditions for a medium or a high on purpose. What I could not find: the information we need from Apple These are the questions I would like answered, along with what each one means for you until they are. The session says values and errors can be overridden with launch arguments, and refers to the documentation for the list. Where can we find that list? Until then, the medium, high and error paths can only be tested by reading the code, which is not a comfortable place to be for a flow that holds someone's money. Which devices and regions are supported, and what determines the region that applies to a device? The documentation for .unavailable mentions both without listing them, so if your app has customers around the world, you cannot yet tell your support team who should be getting this, and .unavailable on its own will not tell you why. What status should an app expect on a phone where the setting has never been changed, and can it show the permission alert in that state? If the answer is .denied, your app cannot ask until the customer has found the setting on their own, which would change how you plan for adoption. What does the 24 hour delay apply to, and do evaluations behave differently during it? Because the status changes immediately, at least in development, I cannot tell you what to expect in production. Which environment do TestFlight, Ad Hoc and Enterprise builds use? The session only separates development (sandbox) from the App Store (production), so you cannot be sure how much your beta testing tells you about production. What are the rate limits, and is there guidance for rolling this out to a large installed base, as there is for App Attest? The documentation says a limit can apply to the app on one device or across all devices. For now I would ramp up slowly, and treat rateLimitExceeded as having no evaluation. What period of activity does an evaluation reflect, and how long does the result stay meaningful? Because I have to start it early, I do not know how much a result from when the screen appeared says about what is happening when the person taps Confirm a few minutes later. How does a server verify signedPayload? I could not find the keys or the payload schema. Until that is available, using a result to reduce friction (.usedReducedFriction) cannot be done safely. Where is the API for reporting fraud after the fact documented, and can it be used without a verified business on Apple Business Register? The session refers to the documentation for this, and also to Partner Data Services, which I could not find described anywhere public. I have filed all of this as a documentation request, FB25453150, which also covers a few smaller items like language support and privacy declarations. I will update this thread as I learn more, and if you have measured something different from what I describe here, please post it, because that would help everyone working on this. S.BECK - 7strikes.dev
0
0
35
5h
Crash Detection / Emergency SOS: desafios reais de segurança pessoal em escala
Estou compartilhando algumas observações técnicas sobre Crash Detection / Emergency SOS no ecossistema Apple, com base em eventos amplamente observados em 2022 e 2024, quando houve chamadas automáticas em massa para serviços de emergência. A ideia aqui não é discutir UX superficial ou “edge cases isolados”, mas sim comportamento sistêmico em escala, algo que acredito ser relevante para qualquer time que trabalhe com sistemas críticos orientados a eventos físicos. Contexto resumido A partir do iPhone 14, a Detecção de Acidente passou a correlacionar múltiplos sensores (acelerômetros de alta faixa, giroscópio, GPS, microfones) para inferir eventos de impacto severo e acionar automaticamente chamadas de emergência. Em 2022, isso resultou em um volume significativo de falsos positivos, especialmente em atividades com alta aceleração (esqui, snowboard, parques de diversão). Em 2024, apesar de ajustes, houve recorrência localizada do mesmo padrão. Ponto técnico central O problema não parece ser hardware, nem um “bug pontual”, mas sim o estado intermediário de decisão: Aceleração ≠ acidente Ruído ≠ impacto real Movimento extremo ≠ incapacidade humana Quando o classificador entra em estado ambíguo, o sistema depende de uma janela curta de confirmação humana (toque/voz). Em ambientes ruidosos, com o usuário em movimento ou fisicamente ativo, essa confirmação frequentemente falha. O sistema então assume incapacidade e executa a ação fail-safe: chamada automática. Do ponto de vista de engenharia de segurança, isso é compreensível. Do ponto de vista de escala, é explosivo. Papel da Siri A Siri não “decide” o acidente, mas é um elo sensível na cadeia humano–máquina. Falhas de compreensão por ruído, idioma, respiração ofegante ou ausência de resposta acabam sendo interpretadas como sinal de emergência real. Isso é funcionalmente equivalente ao que vemos em sistemas automotivos como o eCall europeu, quando a confirmação humana é inexistente ou degradada. O dilema estrutural Há um trade-off claro e inevitável: Reduzir falsos negativos (não perder um acidente real) Aumentar falsos positivos (chamadas indevidas) Para o usuário individual, errar “para mais” faz sentido. Para serviços públicos de emergência, milhões de dispositivos errando “para mais” criam ruído operacional real. Por que isso importa para developers A Apple hoje opera, na prática, um dos maiores sistemas privados de segurança pessoal automatizada do mundo, interagindo diretamente com infraestrutura pública crítica. Isso coloca Crash Detection / SOS na mesma categoria de sistemas safety-critical, onde: UX é parte da segurança Algoritmos precisam ser auditáveis “Human-in-the-loop” não pode ser apenas nominal Reflexões abertas Alguns pontos que, como developer, acho que merecem discussão: Janelas de confirmação humana adaptativas ao contexto (atividade física, ruído). Cancelamento visual mais agressivo em cenários de alto movimento. Perfis de sensibilidade por tipo de atividade, claramente comunicados. Critérios adicionais antes da chamada automática quando o risco de falso positivo é estatisticamente alto. Não é um problema simples, nem exclusivo da Apple. É um problema de software crítico em contato direto com o mundo físico, operando em escala planetária. Justamente por isso, acho que vale uma discussão técnica aberta, sem ruído emocional. Curioso para ouvir perspectivas de quem trabalha com sistemas similares (automotivo, wearables, safety-critical, ML embarcado). — Rafa
0
0
838
Jan ’26
Questions about user-generated content rules
Hello, My app was rejected by Apple for violations Guideline 1.2 - Safety - User-Generated Content We found in our review that your app includes user-generated content but does not have all the required precautions. Apps with user-generated content must take specific steps to moderate content and prevent abusive behavior. Next Steps To resolve this issue, please revise your app to implement the following precautions: A method for filtering objectionable content A mechanism for users to flag objectionable content A mechanism for users to block abusive users I received the message mentioned above; however, I am uncertain about how to address the specific error it references. Do I need to implement both an automatic content review mechanism and a manual review mechanism? Could you please provide detailed guidance on what actions need to be taken? Thank you in advance for your assistance. sontung.tran
1
0
1.5k
Apr ’24
Adopting Trust Insights in a global app: what works, what I measured, and what I could not find
I have been working through what it would take for a real app, say a bank's, to adopt Trust Insights in a way that works for customers around the world, using only what has been published so far: session 379, the framework documentation and the support article. While the client side works well, there is information a production app needs that I could not find. So I am posting this for two reasons. Some of what I learned may save you time, and since the rest needs an answer from Apple, I would like to have those questions answered in one place. If you would like to see working code, the sample project I used keeps all of the API in one file. What I measured Adding the Trust Insights capability in Xcode was all I needed to get the whole flow running on a device: the permission alert, the evaluation and the report. The simulator returns .unavailable, so you will need a device to test any of this. There are two switches to keep in mind, the system one at Settings > Privacy & Security > Impersonation Risk Detection, and one for each app underneath it. This is what I got from authorizationStatus(for:) in a development build: With the setting turned off, for the app or for everything, it returned .denied right away, even though the support article says changes might take up to 24 hours. It was still .denied 24 hours later. When I turned the setting back on, it returned .authorized right away. After deleting and reinstalling the app it returned .denied, and did not go back to .notDetermined. So while the documentation describes .denied as the person refusing permission, in practice it also covers the setting being off, and you should plan for customers who will never see your permission alert. Since I could not find an API that opens that page in Settings, all you can do is tell them where it is, and you should call it Impersonation Risk Detection because that is the name they will see there. What I would do today Treat having no evaluation as a normal path. Between permissions, availability, the network and server errors, some of your customers will not get a result, and in all of those cases your existing checks should decide and the transaction should carry on as it did before. Start the evaluation early. The session says it takes a couple of seconds and the documentation says several, so I start it when the screen appears, and a production app should also stop waiting after a deadline of its own. An evaluation that comes back too late still has to be reported, as .notUsedError. Call reportConsumption once for every evaluation, on every path, including a transaction the person cancels (.notUsedNotNeeded). The documentation says that skipping it may result in rate limiting or losing access to the service, so this is not something to leave for later. Until the server side details are available, only use the result to add friction. A decision made in the app can be changed by anyone who can change the app, which does not matter for a warning, but it does matter if unknown lets someone skip a check. Keep the eventID with your transaction records, even if you have no way of using it yet, because that is what the documentation says identifies the evaluation when fraud is discovered later. If your fraud team wants to see numbers before anything changes for customers, .usedEvaluationOnly is the documented way to report an evaluation that had no effect on a decision. For UI work, put your decision behind a type of your own and drive the warning screens from a debug switch, because there is no way to create the conditions for a medium or a high on purpose. What I could not find: the information we need from Apple These are the questions I would like answered, along with what each one means for you until they are. The session says values and errors can be overridden with launch arguments, and refers to the documentation for the list. Where can we find that list? Until then, the medium, high and error paths can only be tested by reading the code, which is not a comfortable place to be for a flow that holds someone's money. Which devices and regions are supported, and what determines the region that applies to a device? The documentation for .unavailable mentions both without listing them, so if your app has customers around the world, you cannot yet tell your support team who should be getting this, and .unavailable on its own will not tell you why. What status should an app expect on a phone where the setting has never been changed, and can it show the permission alert in that state? If the answer is .denied, your app cannot ask until the customer has found the setting on their own, which would change how you plan for adoption. What does the 24 hour delay apply to, and do evaluations behave differently during it? Because the status changes immediately, at least in development, I cannot tell you what to expect in production. Which environment do TestFlight, Ad Hoc and Enterprise builds use? The session only separates development (sandbox) from the App Store (production), so you cannot be sure how much your beta testing tells you about production. What are the rate limits, and is there guidance for rolling this out to a large installed base, as there is for App Attest? The documentation says a limit can apply to the app on one device or across all devices. For now I would ramp up slowly, and treat rateLimitExceeded as having no evaluation. What period of activity does an evaluation reflect, and how long does the result stay meaningful? Because I have to start it early, I do not know how much a result from when the screen appeared says about what is happening when the person taps Confirm a few minutes later. How does a server verify signedPayload? I could not find the keys or the payload schema. Until that is available, using a result to reduce friction (.usedReducedFriction) cannot be done safely. Where is the API for reporting fraud after the fact documented, and can it be used without a verified business on Apple Business Register? The session refers to the documentation for this, and also to Partner Data Services, which I could not find described anywhere public. I have filed all of this as a documentation request, FB25453150, which also covers a few smaller items like language support and privacy declarations. I will update this thread as I learn more, and if you have measured something different from what I describe here, please post it, because that would help everyone working on this. S.BECK - 7strikes.dev
Replies
0
Boosts
0
Views
35
Activity
5h
Crash Detection / Emergency SOS: desafios reais de segurança pessoal em escala
Estou compartilhando algumas observações técnicas sobre Crash Detection / Emergency SOS no ecossistema Apple, com base em eventos amplamente observados em 2022 e 2024, quando houve chamadas automáticas em massa para serviços de emergência. A ideia aqui não é discutir UX superficial ou “edge cases isolados”, mas sim comportamento sistêmico em escala, algo que acredito ser relevante para qualquer time que trabalhe com sistemas críticos orientados a eventos físicos. Contexto resumido A partir do iPhone 14, a Detecção de Acidente passou a correlacionar múltiplos sensores (acelerômetros de alta faixa, giroscópio, GPS, microfones) para inferir eventos de impacto severo e acionar automaticamente chamadas de emergência. Em 2022, isso resultou em um volume significativo de falsos positivos, especialmente em atividades com alta aceleração (esqui, snowboard, parques de diversão). Em 2024, apesar de ajustes, houve recorrência localizada do mesmo padrão. Ponto técnico central O problema não parece ser hardware, nem um “bug pontual”, mas sim o estado intermediário de decisão: Aceleração ≠ acidente Ruído ≠ impacto real Movimento extremo ≠ incapacidade humana Quando o classificador entra em estado ambíguo, o sistema depende de uma janela curta de confirmação humana (toque/voz). Em ambientes ruidosos, com o usuário em movimento ou fisicamente ativo, essa confirmação frequentemente falha. O sistema então assume incapacidade e executa a ação fail-safe: chamada automática. Do ponto de vista de engenharia de segurança, isso é compreensível. Do ponto de vista de escala, é explosivo. Papel da Siri A Siri não “decide” o acidente, mas é um elo sensível na cadeia humano–máquina. Falhas de compreensão por ruído, idioma, respiração ofegante ou ausência de resposta acabam sendo interpretadas como sinal de emergência real. Isso é funcionalmente equivalente ao que vemos em sistemas automotivos como o eCall europeu, quando a confirmação humana é inexistente ou degradada. O dilema estrutural Há um trade-off claro e inevitável: Reduzir falsos negativos (não perder um acidente real) Aumentar falsos positivos (chamadas indevidas) Para o usuário individual, errar “para mais” faz sentido. Para serviços públicos de emergência, milhões de dispositivos errando “para mais” criam ruído operacional real. Por que isso importa para developers A Apple hoje opera, na prática, um dos maiores sistemas privados de segurança pessoal automatizada do mundo, interagindo diretamente com infraestrutura pública crítica. Isso coloca Crash Detection / SOS na mesma categoria de sistemas safety-critical, onde: UX é parte da segurança Algoritmos precisam ser auditáveis “Human-in-the-loop” não pode ser apenas nominal Reflexões abertas Alguns pontos que, como developer, acho que merecem discussão: Janelas de confirmação humana adaptativas ao contexto (atividade física, ruído). Cancelamento visual mais agressivo em cenários de alto movimento. Perfis de sensibilidade por tipo de atividade, claramente comunicados. Critérios adicionais antes da chamada automática quando o risco de falso positivo é estatisticamente alto. Não é um problema simples, nem exclusivo da Apple. É um problema de software crítico em contato direto com o mundo físico, operando em escala planetária. Justamente por isso, acho que vale uma discussão técnica aberta, sem ruído emocional. Curioso para ouvir perspectivas de quem trabalha com sistemas similares (automotivo, wearables, safety-critical, ML embarcado). — Rafa
Replies
0
Boosts
0
Views
838
Activity
Jan ’26
Questions about user-generated content rules
Hello, My app was rejected by Apple for violations Guideline 1.2 - Safety - User-Generated Content We found in our review that your app includes user-generated content but does not have all the required precautions. Apps with user-generated content must take specific steps to moderate content and prevent abusive behavior. Next Steps To resolve this issue, please revise your app to implement the following precautions: A method for filtering objectionable content A mechanism for users to flag objectionable content A mechanism for users to block abusive users I received the message mentioned above; however, I am uncertain about how to address the specific error it references. Do I need to implement both an automatic content review mechanism and a manual review mechanism? Could you please provide detailed guidance on what actions need to be taken? Thank you in advance for your assistance. sontung.tran
Replies
1
Boosts
0
Views
1.5k
Activity
Apr ’24