Model choice is the fun meeting. The meeting that saves you is the one about IAM, logs, and which subnet is allowed to call an external API. AI on AWS is still AWS: identities, environments, and bills that need an owner.
Decide where the tokens go
OpenAI, Anthropic, Bedrock, or a GPU box you babysit. Each has a data-handling story. If you sell to US and UK customers, write down residency and subprocessors before marketing ships a screenshot. A private VPC endpoint and a key that is not in a frontend repo are not optional extras.
Reuse the platform you already have
Queues, secrets, staging, and alarms should look like the rest of the product. Do not stand up a parallel “ML platform” for one copilot. Our AWS checklist for product teams still applies; AI adds token metrics and a red-button disable. See also the broader AWS practice.
Cost is an architecture decision
Embeddings you recompute on every request, 128k context “just in case,” and no cache will show up as a surprise invoice. Budget per feature, not per company. Alert on spend the same way you alert on 5xx.
If the feature itself is still fuzzy, start with the job and the data, then pick the model. The reverse order is how teams buy GPUs they never fill.