Alpay Calalli

How to protect your users from Scam Coaching in your iOS App with Trust Insights

2026-08-31T13:00:00+00:00

“The Trust Insights framework enables your app to request an evaluation, or insight, to help detect and respond to social engineering threats people may face.” as the official TrustInsight documentation says.

So what does it really mean and how does it work? These were the questions that popped out to my mind, when I saw about this in WWDC26.

Social threats don’t always happen when someone on the other side of the world steals your password and takes control of your account. It also commonly happens with manipulation, the person you speak to on the phone or on the street misidentifies himself and pushes you to do actions such as loging-in to your account, sending money to an account, etc. Since, YOU are the one doing the operation, no security protections like 2FA will be help in these scenarios. From the moment our imaginary victim “Bill” also gives the 2FA code to the other person who is guiding him, done, there is nothing to do about it.

Well, what can you do as an Apple developer to prevent these on the client side? This is what TrustInsight is for. It “magically”(of course not magically, will come to this later in the article) finds these potential anomalies, coercive activies based on the context you give and returns the result to you to take an appropriate action, such as adding friction, requiring additional verification, or notifying the user about it. BUT, Apple does not recommend completely blocking the operation based solely on a trust insight.

Note: Note that this evalutation process may take a few seconds and requires an internet connection, so when you use it is also important for user experience.

In this article, we will look at a banking app use-case example.

Note: Trust Insights is an iOS 27 API and is currently marked as beta. You need Xcode 27 or later, and the API may change before its final release.

Create Entitlement

First, creating an entitlement is required. You can do so by going to your app target, press “+ Capability” and select “Trust Insights”.

Setup

To request a trust insight evaluation, we need to provide 2 elements: the request and the context.

request - The type of insight we want the framework to evaluate. Currently, the framework supports one type of request, IsLikelyBeingCoachedInsight. As we can understand from the name, it is for scenarios where the user may be coached by someone to perform certain actions. Potentially, we may see more types of requests in the future.

context - The kind of action the user is trying to perform.

Here are all the available contexts we can choose:

.payment An action that indicates some form of payment or purchase.

.account An account operation including registration, login, or the modification of account details.

.resourceUse Usage of some resource, online service such as purchasing API credits, cloud rendering, etc.

.communication An action that indicates communication operation, such as sending bulk messages or making connections to other people.

.other Fallback when none of the above categories fit. Use cases: Exporting sensitive data, granting remote access, or any custom sensitive action that doesn’t naturally belong to the other categories.

Apple recommends filing a Feedback report with the details relating to the category of your interest, when you are left to use .other.

Implementation

As a basic use-case scenario for Trust Insight, let’s look at a banking app example. We have a person named Bill. One day, he gets a call, and the caller introduces himself as a bank employee and tells him that money was accidentally transferred to Bill’s account and asks him to return it to another account.

Bill says ok, opens the app, eventually comes to the final screen to confirm the transaction, and presses the confirm button. Before processing it or sending an authentication code, we can create and call Trust Insight request, wait for the result, and decide the next action based on it.

Here is a look at our implementation in a view model.

import Foundation
#if canImport(TrustInsights)
import TrustInsights
#endif

final class OurViewModel {

    // some code

    #if canImport(TrustInsights)
    @available(iOS 27.0, *)
    func evaluateCoachingRiskWithTrustInsights() async {
        let request = IsLikelyBeingCoachedInsight.request(
            schema: .version1,
            modelVersion: .current
        )

        let context = InsightEvaluator.InsightContext(
            operationCategory: .payment,
            requestedEvaluations: request
        )

        let evaluator = InsightEvaluator()

        do {
            guard try await evaluator.requestAuthorization(for: context) == .authorized else { return }

            let assessment = try await evaluator.requestEvaluation(context: context)
            let outcome = try assessment.insight.outcome.get()

            switch outcome {
            case .unknown:
                assessment.reportConsumption(.usedUnchangedFriction)
            case .medium:
                addExtraVerificationStep()
                
                assessment.reportConsumption(.usedIncreasedFriction)
            case .high:
                reportToCallService()
                
                assessment.reportConsumption(.usedIncreasedFriction)
            @unknown default:
                assessment.reportConsumption(.notUsedError)
            }
        } catch {}
    }
    #endif
}

A few things to look out for in the code:

Don’t we feed it any data? The answer is no, at least not from our app. We only provide the operation context, such as .payment, and request the evaluation. We don’t send the person’s messages, call audio, transaction details, or any other custom payload to the framework.

So how does Trust Insights really work under the hood? Apple gives us a high-level explanation in the Meet Trust Insights session, but does not reveal the model architecture or exactly how each signal is weighted. The framework uses a machine-learning model and combines on-device processing with Apple’s cloud infrastructure. On the device, it looks at signals such as interaction patterns, timing, context, and basic sensor data. Apple specifically says that it does not inspect content from Photos, Messages, or Mail. Device-sourced data is processed locally and discarded immediately after the evaluation; only a single result value leaves the device. Apple’s service may then combine that result with Apple Account signals and velocity checks—patterns such as unusual activity happening repeatedly or too quickly—to add more context before returning .unknown, .medium, or .high to the app.
For unknown outcome case, we are not actually doing anything in the code example. But unknown does not mean there is no risk, so taking some precautionary actions, such as adding extra friction is still recommended.
What is assessment.reportConsumption(:)? - calling this is actually mandatory by Apple by after each evalutation request. Rate limits may apply if you decide not to call it with the correct consumption value related to your taken action. Here are 6 types available:
1. .usedReducedFriction — insight made operation easier, caused your app to remove or reduce security friction for the user
2. .usedUnchangedFriction — insight evaluated but didn’t change experience for user
3. .usedIncreasedFriction — insight led to additional checks (as I mentioned earlier, blocking the whole operation based solely on this is not recommended.)
4. .notUsedNotNeeded — user cancelled the operation
5. .notUsedError — technical failure (e.g. result arrived too late)
6. .usedEvaluationOnly — insight used for internal benchmarking only, no UX impact
There is also a thing called Offline label submissions, which is for cases in which the evaluation ultimately results in a confirmed fraud after days, weeks, or months. It’s not mandatory, but again, recommended by Apple to understand the model’s real-world performance and strengthen the ecosystem. Learn more about it by watching the Meet Trust Insights.

Well, back to Bill. After we run this insight evalution, it told us there is a high-scam risk. So in our banking app example, we can pause the operation, show an alert to Bill to wait for a few minutes due to a possibly suspicious operation, and notify the call-service members to get in contact with Bill to make sure it’s all intentional and confirm the operation.

Note: Sandbox environment is used for Trust Insight during development. Production models/servers are used after App Store distribution.

Conclusion

Trust Insights is not something that can completely stop social engineering scams, and it should not be treated as the final decision-maker. But still it gives us something that the usual security protections don’t: a signal that the person performing a legitimate action may actually be doing it under someone else’s pressure.

As we saw, requesting an evaluation is the easy part. The more important part is deciding when to request it and what to do with the result. Depending on your app, this may mean showing a warning, delaying the operation, asking for additional verification, or sending it for manual review by a human. The response should be proportional to the risk, and a trust insight should always work together with your existing fraud and security logic.

That’s it for now, Thanks for reading!

Building an On-Device Video Editor app with Swift/SwiftUI and SpeechAnalyzer — Part 3

2026-08-24T13:00:00+00:00

Here we are with Part 3!

If you are new here, head back to Part 1 and Part 2 to see the progress so far. In this part, we will look at how to split the full transcript into subtitle groups and show them on top of the video player.

We will show subtitles in three ways, and the user can choose the one that suits them:

One word at a time
A group of words
A group of words with the currently spoken word highlighted

Let’s define these methods as an enum.

import SwiftUI

enum SubtitleShowMethod: String, CaseIterable, Identifiable {
    case oneWord
    case wholeSentence
    case wholeSentenceWithHighlightedWord

    var id: Self { self }

    var description: LocalizedStringResource {
        switch self {
        case .oneWord:
            "Word"
        case .wholeSentence:
            "Group"
        case .wholeSentenceWithHighlightedWord:
            "Highlight"
        }
    }
}

Now, let’s move on to look at how to make our subtitles from the full transcript

struct SubtitleGrouper {
    private let maximumCharacterCount = 40

    func apply(to words: [TranscriptWord]) -> [SubtitleGroup] {
        let timedWords = words.filter { $0.audioTimeRange != nil }
        guard !timedWords.isEmpty else { return [] }

        var groups: [SubtitleGroup] = []
        var currentWords: [TranscriptWord] = []
        var currentCharacterCount = 0

        for word in timedWords {
            let separatorCount = currentWords.isEmpty ? 0 : 1
            let proposedCount = currentCharacterCount + separatorCount + word.word.characters.count

            if currentWords.isEmpty || proposedCount < maximumCharacterCount {
                currentWords.append(word)
                currentCharacterCount = proposedCount
            } else {
                groups.append(makeGroup(from: currentWords, endTime: word.audioTimeRange?.start))
                currentWords = [word]
                currentCharacterCount = word.word.characters.count
            }
        }

        if let finalEndTime = currentWords.last?.audioTimeRange?.end {
            groups.append(makeGroup(from: currentWords, endTime: finalEndTime))
        }

        return groups
    }

    private func makeGroup(from words: [TranscriptWord], endTime: CMTime?) -> SubtitleGroup {
        let startTime = words.first?.audioTimeRange?.start ?? .zero
        let endTime = endTime ?? words.last?.audioTimeRange?.end ?? startTime
        let text = AttributedString(
            words
                .map { String($0.word.characters) }
                .joined(separator: " ")
        )

        return SubtitleGroup(
            text: text,
            words: words,
            startTime: startTime,
            endTime: endTime
        )
    }

}

import AVFoundation

/// A short group of timed words displayed together over the video.
struct SubtitleGroup: Identifiable {
    let id = UUID()
    let text: AttributedString
    let words: [TranscriptWord]
    let startTime: CMTime
    let endTime: CMTime
}

To keep things simple, the only rule I use to separate subtitles is an exclusive character-count limit. With maximumCharacterCount set to 40, each group stays below 40 characters. You can extend this with more detailed grouping rules if needed. Storing both text and words properties helps us to have more control over each word like we did the same for the transcript model.

Well. since we are going to show each word as a separate UI element, again, we need to define a custom Layout for the subtitles.

import SwiftUI

/// Wraps subtitle words into centered rows.
struct SubtitleLayout: Layout {
    var alignment: Alignment = .center
    var spacing: CGFloat = 4

    func sizeThatFits(
        proposal: ProposedViewSize,
        subviews: Subviews,
        cache: inout ()
    ) -> CGSize {
        let maximumWidth = proposal.width ?? .infinity
        var x: CGFloat = 0
        var y: CGFloat = 0
        var rowHeight: CGFloat = 0

        for subview in subviews {
            let size = subview.sizeThatFits(.unspecified)

            if x > 0, x + size.width > maximumWidth {
                x = 0
                y += rowHeight
                rowHeight = 0
            }

            x += size.width + spacing
            rowHeight = max(rowHeight, size.height)
        }

        return CGSize(width: maximumWidth, height: y + rowHeight)
    }

    func placeSubviews(
        in bounds: CGRect,
        proposal: ProposedViewSize,
        subviews: Subviews,
        cache: inout ()
    ) {
        let rows = rows(for: subviews, maximumWidth: bounds.width)
        var y = bounds.minY

        for row in rows {
            let rowWidth = row.reduce(0) {
                $0 + $1.sizeThatFits(.unspecified).width
            } + CGFloat(max(row.count - 1, 0)) * spacing

            var x = startingX(for: rowWidth, in: bounds)
            var rowHeight: CGFloat = 0

            for subview in row {
                let size = subview.sizeThatFits(.unspecified)
                subview.place(
                    at: CGPoint(x: x, y: y),
                    proposal: ProposedViewSize(size)
                )
                x += size.width + spacing
                rowHeight = max(rowHeight, size.height)
            }

            y += rowHeight
        }
    }

    private func rows(for subviews: Subviews, maximumWidth: CGFloat) -> [[LayoutSubview]] {
        var rows: [[LayoutSubview]] = []
        var currentRow: [LayoutSubview] = []
        var currentWidth: CGFloat = 0

        for subview in subviews {
            let size = subview.sizeThatFits(.unspecified)

            if !currentRow.isEmpty, currentWidth + size.width > maximumWidth {
                rows.append(currentRow)
                currentRow = []
                currentWidth = 0
            }

            currentRow.append(subview)
            currentWidth += size.width + spacing
        }

        if !currentRow.isEmpty {
            rows.append(currentRow)
        }

        return rows
    }

    private func startingX(for rowWidth: CGFloat, in bounds: CGRect) -> CGFloat {
        switch alignment {
        case .leading:
            bounds.minX
        case .trailing:
            bounds.maxX - rowWidth
        default:
            bounds.minX + (bounds.width - rowWidth) / 2
        }
    }
}

Well, I didn’t write all these mathematical calculations by hand, AI did almost the whole job, but basically what’s happening here is that the layout measures each word, moves it to a new row when it no longer fits within the available width, and then centers each row.

SubtitleOverlayView

import SwiftUI

struct SubtitleOverlayView: View {
    let group: SubtitleGroup?
    let currentWord: TranscriptWord?
    let showMethod: SubtitleShowMethod

    var body: some View {
        Group {
            if let group {
                switch showMethod {
                case .oneWord:
                    if let currentWord {
                        Text(String(currentWord.word.characters))
                    }
                case .wholeSentence:
                    Text(group.text)
                        .multilineTextAlignment(.center)
                case .wholeSentenceWithHighlightedWord:
                    SubtitleLayout(alignment: .center) {
                        ForEach(group.words) { word in
                            Text(String(word.word.characters))
                                .foregroundStyle(
                                    word.id == currentWord?.id ? .yellow : .white.opacity(0.95)
                                )
                        }
                    }
                }
            }
        }
        .foregroundStyle(.white)
        .padding(.horizontal, 24)
        .padding(.bottom, 24)
        .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .bottom)
        .font(.headline.weight(.semibold))
        .fontDesign(.rounded)
        .shadow(color: .black.opacity(0.85), radius: 6)
        .allowsHitTesting(false)
        .accessibilityHidden(true)
    }
}

This is our view for displaying the subtitle depending on the selected style of user. As you can see, SubtitleLayout is used here for putting all words inside ForEach and highlighting the specific Text view by comparing ids’(word.id == currentWord?.id). For all the other cases, putting single Text view is enough

ViewModel setup

Now we have implemented all the pieces, it’s time to put them in places. Here are the new properties we want to introduce in our ViewModel

final class TranscriptDemoViewModel {
    /* ... */

    private(set) var subtitleGroups: [SubtitleGroup] = [] // +
    /// User's selected style to show subtitles. // +
    var subtitleShowMethod: SubtitleShowMethod = .wholeSentenceWithHighlightedWord // +

    var currentSubtitleGroup: SubtitleGroup? { // +
        subtitleGroups.first { group in // +
            group.startTime.seconds <= playbackTime && playbackTime < group.endTime.seconds // +
        } // +
    } // +

    var currentSubtitleWord: TranscriptWord? { // +
        currentSubtitleGroup?.words.first { word in // +
            guard let timeRange = word.audioTimeRange else { return false } // +
            return timeRange.start.seconds <= playbackTime && playbackTime < timeRange.end.seconds // +
        } // +
    } // +

    /* ... */
}

And update processVideo(at:) method

private func processVideo(at videoURL: URL) async throws {
    // Playback can be prepared immediately; transcription does not need to
    // finish before the user can inspect the selected video.
    preparePlayer(for: videoURL)
    transcript = nil
    transcriptWords = []
    subtitleGroups = [] // +
    playbackTime = 0
    state = .extractingAudio

    let audioURL = try await audioExtractor.extractAudio(from: videoURL)
    defer {
        // The extracted track is an intermediate input, not user data.
        try? FileManager.default.removeItem(at: audioURL)
    }

    state = .transcribing
    let transcript = try await transcriber.transcribe(audioAt: audioURL)
    self.transcript = transcript
    transcriptWords = transcript.transcriptWords()
    subtitleGroups = subtitleGrouper.apply(to: transcriptWords) // +
    state = .ready
}

View setup

VStack(spacing: 24) {
    if let player = model.player {
        VideoPlayer(player: player) // -
        VideoPlayer(player: player) { // +
            SubtitleOverlayView( // +
                group: model.currentSubtitleGroup, // +
                currentWord: model.currentSubtitleWord, // +
                showMethod: model.subtitleShowMethod // +
            ) // +
        } // +
        .aspectRatio(16 / 9, contentMode: .fit)
        .clipShape(.rect(cornerRadius: 12))
    }

    SubtitleShowMethodPicker(selection: $model.subtitleShowMethod) // +

    TranscriptContainerView(
        /* ... */
    )
}

import SwiftUI

struct SubtitleShowMethodPicker: View {
    @Binding var selection: SubtitleShowMethod

    var body: some View {
        Picker("Subtitle Display", selection: $selection) {
            ForEach(SubtitleShowMethod.allCases) { method in
                Text(method.description)
                    .tag(method)
            }
        }
        .pickerStyle(.segmented)
    }
}

Result

We now have a nicely working subtitles appearing over the video in 3 different versions with seamless transitions while switching between them.

Download project after changes here in part 3.

Thanks for reading, see you in the part 4!

Your browser does not support embedded videos.

Final Result.

Building an On-Device Video Editor app with Swift/SwiftUI and SpeechAnalyzer — Part 2

2026-08-20T13:00:00+00:00

In Part 1, we imported a video, transcribed its audio fully on-device, and used the timing attributes returned by SpeechTranscriber to highlight the currently spoken word.

This time, we are going to make the transcript interactive. The user will be able to tap any word and jump directly to the moment where it is spoken in the video.

In this part, we will look at how to:

Split an AttributedString into individual words without losing Speech’s attributes that we are gonna need.
Display every word as a separate SwiftUI button.
Jump to the exact starting time of the selected word.

If you want to get the current version of the project that was built in part 1, here you can download.

Why we cannot keep using one full Text view for the whole transcription

In Part 1, the entire transcript was displayed with one Text view.

That worked nicely for highlighting because we could create a copy of the attributed transcript, find the range for the current word, and change the styling of that range.

However, now we want each word to have its own tap action. A single Text view does not give us a separate button for every attributed range, so we need to split the transcript into individual values and render them separately.

There is one important detail here: we should not convert the transcript to a regular String before splitting it.

The audioTimeRange that tells us when a word was spoken is stored as an attribute on the AttributedString. If we first convert it to a String, split it, and create new attributed strings afterward, that timing information will be gone.

Instead, we split the attributed string through its characters view and keep each result as an AttributedSubstring.

Creating a model for each word

We begin with a small model that stores one attributed word.

import AVFoundation
import Speech

struct TranscriptWord: Identifiable {
    let id = UUID()
    let word: AttributedSubstring

    var audioTimeRange: CMTimeRange? {
        word.audioTimeRange
    }
}

The word property is an AttributedSubstring, not a String. Because it is still a slice of the original attributed value, it keeps the attributes attached by SpeechTranscriber.

We also expose its timing as a computed property:

var audioTimeRange: CMTimeRange? {
    word.audioTimeRange
}

The range is optional because an attributed string can contain parts without an audio timing attribute. Later, words without a valid range will simply have their buttons disabled.(hopefully we won’t face a situation like this in our example.)

Splitting the attributed transcript

Next, we add an extension that turns the complete transcript into an array of TranscriptWord values.

extension AttributedString {
    func transcriptWords() -> [TranscriptWord] {
        characters
            .split { character in
                character.isWhitespace || character.isNewline
            }
            .map { characterRange in
                TranscriptWord(
                    word: self[
                        characterRange.startIndex..<characterRange.endIndex
                    ]
                )
            }
    }
}

We split the characters collection wherever we find whitespace or a new line. Each result gives us indices that still belong to the original attributed string, so we can use those indices to create an AttributedSubstring.

self[characterRange.startIndex..<characterRange.endIndex]

That line is the important part. We are taking a slice of the existing value rather than rebuilding the word from plain characters. In this way, we keep the Speech attributes, including the word’s audioTimeRange.

Once transcription finishes, we store both versions in the view model:

let transcript = try await transcriber.transcribe(audioAt: audioURL)
self.transcript = transcript
transcriptWords = transcript.transcriptWords()

We still keep the original transcript because it’s gonna be useful in the future. The new array exists for the interactive UI, where every word needs its own identity and action.

Displaying words as buttons

We can now replace the single transcript Text with a ForEach that creates one button for every word.

struct DynamicTranscriptTextView: View {
    let words: [TranscriptWord]
    let playbackTime: TimeInterval
    let onSelectWord: (CMTime) -> Void

    var body: some View {
        TranscriptLayout {
            ForEach(words) { word in
                let isHighlighted = isHighlighted(word)

                Button {
                    guard let startTime = word.audioTimeRange?.start else {
                        return
                    }

                    onSelectWord(startTime)
                } label: {
                    Text(String(word.word.characters))
                        .font(.body)
                        .lineLimit(1)
                        .foregroundStyle(isHighlighted ? .primary : .secondary)
                        .underline(isHighlighted)
                }
                .buttonStyle(TranscriptWordButtonStyle())
                .disabled(word.audioTimeRange == nil)
            }
        }
        .frame(maxWidth: .infinity, alignment: .leading)
    }
}

We use .lineLimit(1) because a single word should never wrap internally. The custom layout will decide when the complete word needs to move to the next line.

The current word is still highlighted by comparing its time range with the player’s time:

private func isHighlighted(_ word: TranscriptWord) -> Bool {
    guard let timeRange = word.audioTimeRange else {
        return false
    }

    let start = timeRange.start.seconds
    let end = timeRange.end.seconds

    return start <= playbackTime && playbackTime < end
}

Notice that the end comparison uses < rather than <=. Two consecutive ranges may meet at the same boundary. Treating the end as exclusive prevents both words from being considered active at that exact time.

I also added a small pressed state and selection feedback through a custom ButtonStyle:

struct TranscriptWordButtonStyle: ButtonStyle {
    func makeBody(configuration: Configuration) -> some View {
        configuration.label
            .background {
                if configuration.isPressed {
                    RoundedRectangle(cornerRadius: 8)
                        .inset(by: -4)
                        .fill(.thinMaterial)
                }
            }
            .sensoryFeedback(
                .selection,
                trigger: configuration.isPressed
            )
    }
}

Making separate buttons flow like text

If we put the buttons in an HStack, they will stay on one line and eventually go outside the available width. A LazyVGrid can wrap them, but its column-based structure does not look like a natural paragraph because every item is placed into a fixed column.

What we need is a flow layout: place views from leading to trailing, then move the next view to a new line whenever it no longer fits.

SwiftUI’s Layout protocol gives us control over exactly that.

struct TranscriptLayout: Layout {
    var spacing: CGFloat = 4

    func sizeThatFits(
        proposal: ProposedViewSize,
        subviews: Subviews,
        cache: inout ()
    ) -> CGSize {
        let maxWidth = proposal.width ?? .infinity
        var x: CGFloat = 0
        var y: CGFloat = 0
        var lineHeight: CGFloat = 0

        for subview in subviews {
            let size = subview.sizeThatFits(.unspecified)

            if x > 0, x + size.width > maxWidth {
                x = 0
                y += lineHeight + spacing
                lineHeight = 0
            }

            x += size.width + spacing
            lineHeight = max(lineHeight, size.height)
        }

        return CGSize(
            width: maxWidth,
            height: y + lineHeight
        )
    }
}

sizeThatFits answers the first question SwiftUI asks our layout: how much space do you need for these subviews?

We measure each word with an unspecified proposal so it can report its natural size. We then keep track of the current horizontal position and the tallest item on the current line.

if x > 0, x + size.width > maxWidth {
    x = 0
    y += lineHeight + spacing
    lineHeight = 0
}

When the next word would exceed the proposed width, we reset the horizontal position and increase the vertical position by the previous line’s height.

The x > 0 check prevents us from wrapping before the first word. It also means that a word wider than the complete proposed width stays on its current line instead of causing an empty first line.

Measuring the required size is only half of a custom layout. We also need to place the subviews.

func placeSubviews(
    in bounds: CGRect,
    proposal: ProposedViewSize,
    subviews: Subviews,
    cache: inout ()
) {
    var x = bounds.minX
    var y = bounds.minY
    var lineHeight: CGFloat = 0

    for subview in subviews {
        let size = subview.sizeThatFits(.unspecified)

        if x > bounds.minX, x + size.width > bounds.maxX {
            x = bounds.minX
            y += lineHeight + spacing
            lineHeight = 0
        }

        subview.place(
            at: CGPoint(x: x, y: y),
            proposal: ProposedViewSize(size)
        )

        x += size.width + spacing
        lineHeight = max(lineHeight, size.height)
    }
}

The logic intentionally matches sizeThatFits. If measuring and placement use different wrapping rules, SwiftUI may reserve one height and then place the words as if it had another, which can produce clipping or unexpected empty space.

Here, bounds.minX and bounds.minY are also important. The placement coordinates belong to the bounds given to our layout, so we should not assume that its origin is always (0, 0).

Now we can use TranscriptLayout almost like a standard SwiftUI stack:

TranscriptLayout(spacing: 4) {
    ForEach(words) { word in
        // Word button
    }
}

The result looks like one wrapping paragraph, but every word remains a real, independently accessible button.

Seeking to the selected word

The final step is small because the timing information has already done most of the work for us.

The transcript view sends the selected word’s start time upward:

guard let startTime = word.audioTimeRange?.start else {
    return
}

onSelectWord(startTime)

The parent passes the view model’s seek method as that closure:

TranscriptContainerView(
    transcript: model.transcript,
    words: model.transcriptWords,
    playbackTime: model.playbackTime,
    onSelectWord: model.seek
)

And the view model asks the player to seek to the supplied CMTime:

func seek(to time: CMTime) {
    player?.seek(
        to: time,
        toleranceBefore: .zero,
        toleranceAfter: .zero
    )
}

By default, media seeking can use a tolerance around the requested position. That is useful for efficient playback because landing on a nearby frame can be faster than decoding forward to an exact time.

For word selection, however, jumping noticeably before or after the word would make the transcript feel inaccurate. We therefore pass zero tolerance on both sides and ask AVPlayer to seek as precisely as the media allows.

There is still a tradeoff here. Exact seeking may take more work than approximate seeking, especially with videos that have widely spaced keyframes. For this interaction, I prefer accuracy because the user selected a specific spoken word. In another interface, such as quickly scrubbing through thumbnails, a nonzero tolerance might be the better choice.

The periodic time observer from Part 1 continues to update playbackTime after the seek. That means we do not need to set the highlighted word manually. The player moves, its observer reports the new time, and SwiftUI highlights whichever word contains that time.

Accessibility details

Because every word is now a button, VoiceOver should also understand what selecting it will do.

.accessibilityLabel(String(word.word.characters))
.accessibilityHint("Seeks the video to this word")
.accessibilityValue(
    isHighlighted ? "Currently speaking" : ""
)

The label provides the spoken word, the hint explains the action, and the value identifies the word that currently matches playback.

This is another advantage of using actual Button views instead of adding a tap gesture to plain text. We get button semantics and keyboard or accessibility activation behavior without recreating them ourselves.

Putting everything together

The complete flow for this part looks like this:

SpeechTranscriber returns an AttributedString with an audioTimeRange attached to each timed word.
We split its characters into AttributedSubstring values so those attributes are preserved.
A custom SwiftUI Layout displays the word buttons as a wrapping paragraph.
Tapping a word sends its starting CMTime to the view model.
AVPlayer seeks to that time, and the existing time observer updates the highlighted word.

We now have a transcript that works in both directions. Playing or seeking the video updates the transcript, and selecting the transcript updates the video.

Download project after changes here in part 2.

Your browser does not support embedded videos.

Final Result.(Well, screen touches are not visible due to recording on a real device, `SpeechAnalyzer` does not work on simulator. So assume that I press on the word that video jumps to.)

In the next part, we will use these timed words to create subtitle groups and place them over the video. And after that, maybe will look at censoring some selected words on the video, I’m not sure yet:)

Thanks for reading, see you in the part 3!

Building an On-Device Video Editor app with Swift/SwiftUI and SpeechAnalyzer — Part 1

2026-08-17T12:00:00+00:00

Hi everyone, this is going to be the first part of a few-part series where I’m going to share my experience of building kind of a video editor app where user can see the full transcription of the video, see auto-generated subtitles on it, and even select words to censor(add beep or duck sound). It will work fully on-device, thanks to Apple’s SpeechAnalyzer API for speech-to-text conversion.

Actually, I built the final version of this app as my submission for the Swift Student Challenge this year, which didn’t turn out to be selected among the winners. Still, after some time passed, I thought it was a cool app where I tried and learned different technologies, and it would be nice to share my experiences and how I solved some technical barriers.

Your browser does not support embedded videos.

A preview of the final app, including transcription, subtitles, and word censoring.

There are several interesting parts we will take a look at, such as muting selected words with high precision to not cause any odd behaviour or adding layers at specific positions in an export video which is unfortunately, is not as straightforward as adding an .overlay modifier in SwiftUI.

We will get to those in the following parts of the series.

In this first part, we will look at how to:

Import a video from Files or the Photo Library.
Transcribe the audio fully on-device.
Display the video using SwiftUI’s native VideoPlayer.
Highlight the currently spoken word as the video plays or the user changes its playback position.

I haven’t included all the code in this post to keep the code snippets minimal. You can download the complete Xcode project for Part 1 here.

Creating the transcriber

Here is our Transcriber class, which accepts the URL of an audio file, transcribes it, and returns the final result as an AttributedString.

import Speech

protocol Transcribable {
    func transcribe(audioAt url: URL) async throws -> AttributedString
}

final class Transcriber: Transcribable {
    private let locale = Locale(identifier: "en-US")

    func transcribe(audioAt url: URL) async throws -> AttributedString {
        guard SpeechTranscriber.isAvailable else {
            throw TranscriptError.notAvailable
        }

        let transcriber = SpeechTranscriber(
            locale: locale,
            transcriptionOptions: [],
            reportingOptions: [],
            attributeOptions: [.audioTimeRange]
        )

        let analyzer = SpeechAnalyzer(modules: [transcriber])
        let audioFile = try AVAudioFile(forReading: url)

        if let lastSample = try await analyzer.analyzeSequence(from: audioFile) {
            try await analyzer.finalizeAndFinish(through: lastSample)
        } else {
            await analyzer.cancelAndFinishNow()
        }

        var transcription = AttributedString()

        for try await result in transcriber.results where result.isFinal {
            transcription.append(result.text)
        }

        guard !transcription.characters.isEmpty else {
            throw TranscriptError.emptyTranscript
        }

        return transcription
    }
}

Let’s go through some of the code here.

We check the availability of SpeechTranscriber for the user’s device by putting a guard statement with SpeechTranscriber.isAvailable at the very beginning.

This property tells us whether the transcriber module is available based on the device’s hardware and capabilities.

guard SpeechTranscriber.isAvailable else {
    throw TranscriptError.notAvailable
}

If it is unavailable, you can disable this feature or fall back to DictationTranscriber. In this article, however, we are going to use the new SpeechTranscriber API.

This demo assumes that the on-device speech model for en-US is already installed. In a production app, you should also handle cases where the required model is unavailable.

Configuring SpeechTranscriber

After checking its availability, we create a SpeechTranscriber for the locale we want to support.

let transcriber = SpeechTranscriber(
    locale: locale,
    transcriptionOptions: [],
    reportingOptions: [],
    attributeOptions: [.audioTimeRange]
)

The most important option for our use case is:

attributeOptions: [.audioTimeRange]

Without this option, we would still receive the transcribed text, but we would not know when each word was spoken.

With .audioTimeRange, the returned AttributedString contains an audioTimeRange attribute for its timed ranges. That timing information is what will later allow us to compare every word with the current playback time of the video.

This is also why we return an AttributedString instead of converting the result into a regular String. A regular String would preserve the characters, but it would lose the Speech framework’s timing attributes.

Analyzing the file

SpeechAnalyzer manages the analysis session, while SpeechTranscriber is the module that performs speech-to-text processing.

let analyzer = SpeechAnalyzer(modules: [transcriber])
let audioFile = try AVAudioFile(forReading: url)

We then give the audio file to the analyzer:

if let lastSample = try await analyzer.analyzeSequence(from: audioFile) {
    try await analyzer.finalizeAndFinish(through: lastSample)
} else {
    await analyzer.cancelAndFinishNow()
}

There is a small but important detail here.

analyzeSequence(from:) returns after the file has been read, but the last audio samples may still be undergoing analysis. Its return value is the time of the last sample that was read.

Passing that value to finalizeAndFinish(through:) tells the analyzer to finish processing everything through that point before we continue.

Apple explains this behavior in the documentation for analyzeSequence(from:).

Finally, transcription results are delivered asynchronously through transcriber.results.

var transcription = AttributedString()

for try await result in transcriber.results where result.isFinal {
    transcription.append(result.text)
}

For a prerecorded video, we only need finalized results. Volatile results are more useful for live transcription, where you want to display SpeechTranscriber’s latest guess while the person is still speaking.

View setup

For this demo, the user can select a video from either Files or the Photo Library.

import AVKit
import PhotosUI
import SwiftUI
import UniformTypeIdentifiers

struct TranscriptDemoView: View {
    @State private var model = TranscriptDemoViewModel()
    @State private var isFileImporterPresented = false
    @State private var isPhotoPickerPresented = false
    @State private var photoPickerItem: PhotosPickerItem?

    var body: some View {
        ScrollView {
            VStack(spacing: 24) {
                if let player = model.player {
                    VideoPlayer(player: player)
                        .aspectRatio(16 / 9, contentMode: .fit)
                        .clipShape(.rect(cornerRadius: 12))

                    TranscriptContainerView(
                        transcript: model.transcript,
                        playbackTime: model.playbackTime
                    )
                } else {
                    contentUnavailableView
                }
            }
            .padding()
        }
        .safeAreaInset(edge: .bottom) {
            if model.player != nil {
                videoImporterView
            }
        }
        .fileImporter(
            isPresented: $isFileImporterPresented,
            allowedContentTypes: [.movie],
            allowsMultipleSelection: false,
            onCompletion: handleFileImport
        )
        .photosPicker(
            isPresented: $isPhotoPickerPresented,
            selection: $photoPickerItem,
            matching: .videos
        )
        .onChange(of: photoPickerItem, handlePhotoSelection)
        .onDisappear(perform: model.stopPlayback)
    }

    private func handleFileImport(_ result: Result<[URL], any Error>) {
        switch result {
        case .success(let urls):
            guard let url = urls.first else { return }
            Task {
                await model.importVideo(from: url)
            }
        case .failure(let error):
            model.reportFileImportFailure(error)
        }
    }

    private func handlePhotoSelection(_ oldItem: PhotosPickerItem?, _ newItem: PhotosPickerItem?) {
        guard let newItem else { return }

        Task {
            await model.importVideo(from: newItem)
            photoPickerItem = nil
        }
    }
}

extension TranscriptDemoView {
    private var contentUnavailableView: some View {
        ContentUnavailableView {
            Label("Import a Video", systemImage: "video.badge.plus")
        } description: {
            Text("Choose a video from Files or your Photo Library to create a word-timed transcript.")
        } actions: {
            VideoImportMenu(
                isFileImporterPresented: $isFileImporterPresented,
                isPhotoPickerPresented: $isPhotoPickerPresented,
                isDisabled: model.isProcessing
            )
        }
        .containerRelativeFrame(.vertical) { height, _ in
            height * 0.65
        }
    }
    
    private var videoImporterView: some View {
        VideoImportMenu(
            isFileImporterPresented: $isFileImporterPresented,
            isPhotoPickerPresented: $isPhotoPickerPresented,
            isDisabled: model.isProcessing
        )
        .padding()
    }
}

ViewModel setup

import AVFoundation
import Observation
import PhotosUI
import SwiftUI

/// Coordinates importing, audio extraction, transcription, and video playback.
///
/// The view owns this model, while the model exposes only UI-facing state. Media
/// processing details stay in the focused helper types below it.
@MainActor
@Observable
final class TranscriptDemoViewModel {
    private(set) var player: AVPlayer?
    /// Speech's attributed result, including an `audioTimeRange` on each word.
    private(set) var transcript: AttributedString?
    private(set) var playbackTime: TimeInterval = 0
    private(set) var state = TranscriptState.empty

    @ObservationIgnored private let audioExtractor = VideoAudioExtractor()
    @ObservationIgnored private let transcriber: Transcribable = Transcriber()
    @ObservationIgnored private let videoStore = ImportedVideoStore()
    @ObservationIgnored private var timeObserver: Any?

    init() {
        AVPlayer.isObservationEnabled = true
    }

    var isProcessing: Bool {
        state.isProcessing
    }

    func importVideo(from fileURL: URL) async {
        state = .importing

        // File importer URLs require explicit access. Copy the video while access
        // is active, then release the external file immediately afterward.
        guard fileURL.startAccessingSecurityScopedResource() else {
            state = .failed("The selected file could not be accessed.")
            return
        }

        defer {
            fileURL.stopAccessingSecurityScopedResource()
        }

        do {
            let localURL = try await videoStore.copyVideo(from: fileURL)
            try await processVideo(at: localURL)
        } catch {
            state = .failed(error.localizedDescription)
        }
    }

    func importVideo(from item: PhotosPickerItem) async {
        state = .importing

        do {
            guard let video = try await item.loadTransferable(type: TransferableVideo.self) else {
                throw TranscriptError.photoImportFailed
            }

            try await processVideo(at: video.url)
        } catch {
            state = .failed(error.localizedDescription)
        }
    }

    func reportFileImportFailure(_ error: any Error) {
        state = .failed(error.localizedDescription)
    }

    func stopPlayback() {
        player?.pause()
    }

    private func processVideo(at videoURL: URL) async throws {
        // Playback can be prepared immediately; transcription does not need to
        // finish before the user can inspect the selected video.
        preparePlayer(for: videoURL)
        transcript = nil
        playbackTime = 0
        state = .extractingAudio

        // somehow, i remember that feeding video url to transcriber used to cause
        // some issue/throw error, but now it seems fine, so keeping both.
        // let audioURL = try await audioExtractor.extractAudio(from: videoURL)
        let audioURL = videoURL
        defer {
            // The extracted track is an intermediate input, not user data.
            try? FileManager.default.removeItem(at: audioURL)
        }

        state = .transcribing
        transcript = try await transcriber.transcribe(audioAt: audioURL)
        state = .ready
    }

    private func preparePlayer(for videoURL: URL) {
        // An observer belongs to the player that created it, so remove the old
        // token before replacing the player with one for a newly imported video.
        removeTimeObserver()
        player?.pause()

        let player = AVPlayer(url: videoURL)
        self.player = player

        // VideoPlayer owns play, pause, and seeking controls. This observer
        // mirrors its clock into observable state so the transcript follows them.
        let interval = CMTime(seconds: 0.05, preferredTimescale: 600)
        timeObserver = player.addPeriodicTimeObserver(forInterval: interval, queue: .main) { [weak self] time in
            Task { @MainActor [weak self] in
                self?.playbackTime = time.seconds
            }
        }
    }

    private func removeTimeObserver() {
        guard let player, let timeObserver else { return }
        player.removeTimeObserver(timeObserver)
        self.timeObserver = nil
    }
}

There is one thing we should keep in mind here: the URLs received from these importers may not remain available forever.

Files returns a security-scoped URL. We start accessing it, copy the video into our app’s temporary directory, and then stop accessing the original URL.

guard fileURL.startAccessingSecurityScopedResource() else {
    state = .failed("The selected file could not be accessed.")
    return
}

defer {
    fileURL.stopAccessingSecurityScopedResource()
}

let localURL = try await videoStore.copyVideo(from: fileURL)

The file delivered by PhotosPicker is also temporary. Our TransferableVideo implementation copies it before the transfer closure returns.

import CoreTransferable
import Foundation
import UniformTypeIdentifiers

/// A durable local copy of a video selected through `PhotosPicker`.
///
/// The URL supplied to a transfer closure is temporary. Copying the file before
/// the closure returns ensures AVPlayer and the audio extractor can use it later.
struct TransferableVideo: Transferable {
    let url: URL

    nonisolated static var transferRepresentation: some TransferRepresentation {
        FileRepresentation(importedContentType: .movie) { receivedVideo in
            let sourceURL = receivedVideo.file
            let destinationURL = FileManager.default.temporaryDirectory
                .appending(path: UUID().uuidString)
                .appendingPathExtension(sourceURL.pathExtension)

            try FileManager.default.copyItem(at: sourceURL, to: destinationURL)
            return TransferableVideo(url: destinationURL)
        }
    }
}

By doing this, AVPlayer and the transcriber both receive a local URL that remains valid while the demo is running.

Using VideoPlayer means we do not need to build our own play, pause, scrubbing, or fullscreen controls.

However, the transcript still needs to know the current playback time. For that, we added an observer to the player.

let interval = CMTime(
    seconds: 0.05,
    preferredTimescale: 600
)

timeObserver = player.addPeriodicTimeObserver(
    forInterval: interval,
    queue: .main
) { [weak self] time in
    Task { @MainActor [weak self] in
        self?.playbackTime = time.seconds
    }
}

Every time the observer runs, we copy the player’s time into observable state.

This works not only while the video is playing. If the user pauses the video or changes its position using the native controls, the observer receives the updated time and the transcript follows it.

Before replacing the player with another imported video, we also need to remove the previous observer:

private func removeTimeObserver() {
    guard let player, let timeObserver else {
        return
    }

    player.removeTimeObserver(timeObserver)
    self.timeObserver = nil
}

A periodic time observer belongs to the player that created it. Keeping an observer after replacing its player can leave us with callbacks from an object we no longer use.

Highlighting the current word

At this point, we have two pieces of information:

An AttributedString whose words contain audioTimeRange attributes.
The current playback time of the video.

Now we need to connect and synchronize them.

import Speech
import SwiftUI

struct DynamicTranscriptTextView: View {
    let transcript: AttributedString
    let playbackTime: TimeInterval

    var body: some View {
        Text(highlightedTranscript)
            .font(.body)
            .multilineTextAlignment(.leading)
            .frame(maxWidth: .infinity, alignment: .leading)
            .accessibilityLabel(String(transcript.characters))
    }

    private var highlightedTranscript: AttributedString {
        var result = transcript
        result.foregroundColor = .secondary

        let currentTime = CMTime(
            seconds: playbackTime,
            preferredTimescale: 600
        )

        let currentInstant = CMTimeRange(
            start: currentTime,
            duration: CMTime(value: 1, timescale: 600)
        )

        guard let currentWord = result
            .rangeOfAudioTimeRangeAttributes(
                intersecting: currentInstant
            ) else {
            return result
        }

        result[currentWord].foregroundColor = .primary
        result[currentWord].backgroundColor = .mint.opacity(0.35)
        result[currentWord].underlineStyle = .single

        return result
    }
}

We begin by creating a copy of the transcript and applying the default secondary color.

Then we convert the player’s current time into CMTime and create a very small CMTimeRange starting at that position.

let currentInstant = CMTimeRange(
    start: currentTime,
    duration: CMTime(value: 1, timescale: 600)
)

This range represents one tick on a 600-timescale timeline. In other words, it gives us a tiny interval around the current playback position.

We then ask the attributed transcript for the range whose audioTimeRange intersects that interval:

guard let currentWord = result
    .rangeOfAudioTimeRangeAttributes(
        intersecting: currentInstant
    ) else {
    return result
}

If a matching word exists, we update only that attributed range.

result[currentWord].foregroundColor = .primary
result[currentWord].backgroundColor = .mint.opacity(0.35)
result[currentWord].underlineStyle = .single

Because this view receives playbackTime from observable state, SwiftUI recalculates the attributed text whenever the player time changes. The highlighted range therefore moves through the transcript together with the video.

The nice part is that we do not need to create a separate array containing every word and its timestamp. SpeechTranscriber has already attached the timing information to the returned AttributedString.

Apple demonstrates the same general technique—comparing playback time with the audioTimeRange attributes—in its WWDC25 SpeechAnalyzer session.

Putting everything together

Your browser does not support embedded videos.

The complete flow now looks like this:

The user imports the video.
SpeechAnalyzer and SpeechTranscriber process its audio on-device.
We keep the returned AttributedString, including its word-timing attributes.
We compare those audio ranges with the current playback time and highlight the matching word.

And that gives us a fully on-device video transcript that follows the native video player, including when the user pauses or seeks to another position.

In the next part, we will look at generating subtitle groups and placing them over the video. After that, we will look at the more complicated part: rendering those layers into the exported video itself.

Thanks for reading, see you in the next part!

How I managed requsting reviews in Treaty

2025-06-21T12:00:00+00:00

Requesting reviews and gathering feedback from users is an important part of all apps.

Apple allows you to show this alert to users only up to 3 times within a 365-day period, so it’s important to use it wisely.

I’ll show how I managed requesting reviews in Treaty - Dog Health by building a similar structure to TipKit.

Before starting, let’s set the rules:

Don’t show the alert more than once

in the same app version
within a certain period (I will set it to 7 days)

actor RequestReviewManager {
   private let userDefaults: UserDefaults
   private let lastRequestReviewKey = "lastRequestReviewDate"
   private let lastVersionPromptedKey = "lastVersionPromptedForReview"
   
   private var lastRequestDate: Date? { userDefaults.value(forKey: lastRequestReviewKey) as? Date }
   private var lastVersionPrompted: String? { userDefaults.string(forKey: lastVersionPromptedKey) }
   
   private var currentAppVersion: String = Bundle.main.releaseVersionNumber

   init(userDefaults: UserDefaults = .group) { self.userDefaults = userDefaults }
}

extension Bundle {
    var releaseVersionNumber: String {
       return infoDictionary?["CFBundleShortVersionString"] as? String ?? "1.0.0"
    }
}

Here are all the properties we need.

Now, obviously, I want to show the alert to users at the most appropriate times. Enum is a nice choice to define our showing cases.

enum RequestReviewReason: String {
   case loggedEnoughFeedActivities
   case loggedEnoughHygieneActivities
   case endedWalk
   
   var triggerValue: Int {
      switch self {
         case .loggedEnoughFeedActivities, .loggedEnoughHygieneActivities: 3
         case .endedWalk: 1
      }
   }
}

import StoreKit

actor RequestReviewManager {
   /*...*/
   func donate(to reason: RequestReviewReason) async {
      // Don't show the alert, if it's already shown within 7 days.
      if let lastRequestDate, lastRequestDate.numberOfDays(from: .now) < 7 { return }
      
      // Don't show the alert, if it's already shown in current app version.
      if let lastVersionPrompted, lastVersionPrompted == currentAppVersion { return }
      
      let count = userDefaults.integer(forKey: reason.rawValue)
      if count + 1 == reason.triggerValue {
         guard let scene = await UIApplication.shared.foregroundActiveScene else { return }
         await AppStore.requestReview(in: scene)
         
         userDefaults.set(Date.now, forKey: lastRequestReviewKey)
         userDefaults.set(currentAppVersion, forKey: lastVersionPromptedKey)
      }
      userDefaults.set(count + 1, forKey: reason.rawValue)
   }
}

extension UIApplication {
    var foregroundActiveScene: UIWindowScene? {
        connectedScenes
            .first(where: { $0.activationState == .foregroundActive }) as? UIWindowScene
    }
}

extension Date {
   func numberOfDays(from date: Date) -> Int {
      let calendar = Calendar.current
      let fromDate = calendar.startOfDay(for: self)
      let toDate = calendar.startOfDay(for: date)
      let numberOfDays = calendar.dateComponents([.day], from: fromDate, to: toDate)
      
      return abs(numberOfDays.day!)
   }
}

Ignoring donations and not incrementing the counter if the 7-day period hasn’t passed yet or this version has already been prompted for review.

You can change the waiting period for the next trigger, apply your own rules, etc.

Now, all we need to do is call the donate() method in the right places, and the alert will trigger if all conditions are met.

struct FeedView: View {
   @Environment(\.requestReviewManager) var requestReviewManager

   var body: some View { /*...*/ }
   func addButtonTapped() async {
      // save action...
      
      await requestReviewManager.donate(to: .loggedEnoughFeedActivities)
   }
}

Your browser does not support the video tag.

Thanks for reading!

How to create a pull-to-trigger action for ScrollView in SwiftUI

2024-11-06T12:00:00+00:00

While I was working on making a performant calendar for my SwiftUI project, I was looking for a solution to load new dates as users scroll. Finally, decided to put only a single year’s data in a lazy container, and when you reach the end, pull to load more data, similar to “pull to refresh”. The solutions I found for pull-to-trigger action in SwiftUI were using drag gestures to track changes, but encountered bugs and weird behaviors while using ScrollView + drag gestures, so looked for another solution. With iOS 17, ScrollView has become more powerful with the addition of the missing APIs. The main modifier we’re going to use is scrollPosition(id:anchor:) to track the current position.

struct CalendarView: View {
    @State private var scrollId: Int? = 3
    var body: some View {
        ScrollView(.horizontal) {
            LazyHStack(spacing: 10) {
                ForEach(0..<30) { item in
                    ItemView(item)
                        .containerRelativeFrame(.horizontal, count: 7, spacing: 10)
                        .background(scrollId == item ? .pink : .gray.opacity(0.3), in: .circle)
                        .id(item)
                }
            }
            .scrollTargetLayout()
        }
        .scrollPosition(id: $scrollId, anchor: .center)
        .scrollTargetBehavior(.viewAligned)
        .frame(height: 90)
    }
}

Here is our setup, now, we can trigger an event depending on the current scroll position.

ScrollView(.horizontal) {
    LazyHStack(spacing: 10) { /*...*/ }
    .scrollTargetLayout()
    .overlay(alignment: .leading) {
        Text("Text")
            .padding()
            .background([0, 1].contains(scrollId) ? .green : .gray, in: .capsule)
            .offset(x: -90)
    }
}
.onChange(of: scrollId, { oldValue, newValue in
    // trigger action if scrollId goes from trigger zone to non trigger zone
    if [0, 1].contains(oldValue),
       ![0, 1].contains(newValue) {
        // do something
    }
})

Obviously, added a label for users to see what is going on and used a different color when the scroll position comes to the “trigger zone.”

It seems done, but let’s say we don’t want to trigger the action if we change our mind midway by scrolling back. In the current version, the action will be triggered no matter what. We can track the scroll phase change to determine whether the user is interacting with the ScrollView or not. Fortunately, from iOS 18, there is a onScrollPhaseChange(_:) modifier.

/* ... */

@State private var isInteracting = false
var body: some View {
ScrollView(.horizontal) { /*...*/ }
.onChange(of: scrollId, { oldValue, newValue in
    if [0, 1].contains(oldValue),
       ![0, 1].contains(newValue),
       !isInteracting {
        // do something
        print("triggered")
    }
})
.onScrollPhaseChange({ _, phase in
    isInteracting = phase == .interacting
})

Finally, now we only trigger the action if the scrollId goes from the trigger zone to the non-trigger zone “naturally”. Obviously, this approach is not suitable for all pull-to-trigger scenarios since it relies on the scroll position. You can use onScrollGeometryChange(for:of:action:) to get the current offset to handle more complex scenarios. I made a Calendar View for SwiftUI using this approach — take a look if you’re interested. Thanks for reading!