A detection box is not an avatar

The detector has finished. It has returned four coordinates and a score. From a model’s perspective, the job looks complete.

The person adding a pet profile has a different definition of done. They want a recognisable avatar, with the face framed naturally and an easy way to change it. A rectangle is an input to that experience, not the experience itself.

In the first article, I described the small head detector behind Pet Analysis Studio. This article follows what happens next: translating coordinates, composing a crop, handling the image boundary and giving the person control.

First, agree on which image the box belongs to

The pipeline first detects a whole animal. It then runs the head detector on that animal’s crop. The resulting head coordinates describe a position inside that smaller image.

Drawing them straight onto the source photograph would place the rectangle in the wrong location. Before any avatar composition can happen, the system needs to return the geometry to a shared coordinate space.

This is an easy boundary to miss because each component can look correct in isolation. The model has located a head. The interface can draw a rectangle. Their agreement about the origin and scale determines whether those two facts connect.

source_x = animal.x1 + head.x1
source_y = animal.y1 + head.y1
head_width = head.x2 - head.x1
head_height = head.y2 - head.y1

Compose for the frame people will see

A head box is usually a tight rectangle. The avatar is displayed as a circle. Using the rectangle unchanged can crowd the ears or chin, and converting it to a square without a composition rule leaves too much to accident.

The crop service starts with the larger of the head box’s width and height, then expands it by a configurable margin. The default is 27.5% in total, not 27.5% on each side.

The head’s focus point anchors the square. Horizontally it is centred; vertically the crop puts 46% of its height above that point and 54% below. It is a small, explicit composition choice that can be inspected and changed independently of the model.

This is where a numeric output becomes a design decision. The detector estimates where the head is. The crop rule decides how that estimate should be presented.

size = min(
    max(head_width, head_height) * (1 + margin_percent / 100),
    image_width,
    image_height,
)
x = min(max(0, centre_x - size / 2), image_width - size)
y = min(max(0, centre_y - size * 0.46), image_height - size)

The edge of the photograph is a real constraint

A pet near the edge of an image makes the composition problem more interesting. There may not be enough original photograph to place an expanded square exactly where the rule would prefer.

The backend caps the square to the source dimensions and clamps its position inside the image. This keeps the crop valid, but it can move the head away from the intended position in the frame. The constraint has not disappeared just because the output has the right shape.

It is tempting to describe an automatic crop as a finished answer. In practice, it is a starting point shaped by the photograph. Tight framing, partial heads and unusual poses deserve explicit inspection, especially through the final circular mask.

I keep the geometry separate from inference partly for this reason. A composition rule can be changed or tested without attributing its successes and failures to the detector.

Let the person adjust the result cheaply

The Studio exposes a margin control so the person can adjust framing. Its preview can use the already-returned geometry; changing the crop does not need another model call.

That separation matters beyond latency or compute. A correction should feel like ordinary editing. If every adjustment launches another uncertain inference, the person is negotiating with the model instead of simply positioning a picture.

The API returns the head box and focus point as well as the proposed avatar crop. Preserving those useful intermediate results gives the interface enough information to support adjustment while keeping the model’s role bounded.

The design intent is to remove the first piece of manual work without making the automatic result compulsory. I have not measured how often customers accept the proposed crop unchanged; that needs a product study.

A fallback needs a truthful name

Sometimes the head detector has no credible result. The rest of the pet workflow may still have useful information, including the whole-animal box.

The fallback uses an upper-animal heuristic to propose an adjustable crop. Crucially, the response labels it heuristic-upper-animal, while a crop derived from a detected head is labelled detected-head. The pipeline also returns a warning when the head is not found.

That provenance lets the product distinguish a detected result from a best-effort starting point. It also helps an engineer investigating a poor crop: was there no head detection, a misplaced head box, or an unsuitable composition rule?

Those are different problems with different remedies. Combining them into a single vague success state would make both the interface and the diagnostics less useful.

Evaluate the whole decision chain

The first article’s detector metrics are useful evidence about head localisation on a particular dataset. To judge this feature, I also need to assess the crop people actually see.

A structured evaluation would ask whether the right animal was selected, whether the head was found, whether the final frame is usable and whether correction is straightforward. It should include tight crops, small subjects, occlusion and multiple pets—not just portraits that already look like avatars.

Each stage deserves its own explanation when something goes wrong. A better head model cannot create missing pixels outside the photograph. A higher confidence score cannot decide someone’s preferred composition. A well-designed adjustment can make a technically imperfect starting point entirely usable.

For me, this is an essential part of applied AI engineering. The component after the model may be ordinary software, but its behaviour determines much of the value people receive from the model.

The finished feature is the entire chain: inference, geometry, provenance, presentation and correction. Designing those pieces together is how four coordinates become an experience.

Sources and scope

This article describes the retained Pet Analysis Studio geometry service, pipeline, API contract and Studio preview. The default margin, source labels and crop formula were checked against that implementation. The numeric examples and diagram are explanatory schematics, not reported evaluation outputs.

Customer crop acceptance, task-time savings and satisfaction remain unmeasured here. They are proposed evaluation targets, not claimed outcomes.