You already have users, roles, and a database. The AI feature should look like every other feature: an endpoint, a queue job, a UI state, and a way to turn it off. Wrapping ChatGPT in a new microservice that ignores your session cookie is how projects stall.
Put the model behind your auth
The browser should talk to your app. Your app should talk to the model. Prompts must include only the data that user is allowed to see. Stream the answer if you must, but log the request id, the model, the tokens, and who asked. That is ordinary web application work with a new upstream.
Keep cost and latency on the board
Token spend is a product decision. Cache embeddings. Cap context. Prefer a small model for classification and a larger one for generation. Put a budget alarm next to the other AWS alarms. A spike in “helpful” completions should not be how you learn about a prompt injection or a loop.
Ship a slice, then evaluate
Pick one screen. Write twenty golden questions. Measure groundedness and refusal. Put a human in the loop where the answer can move money or reputation. Then expand. WisdomSol’s AI team will not start with a six-month platform if a single endpoint would prove the job.
If the surrounding product is still a pile of spreadsheets, fix that first. AI does not replace a missing domain model. It amplifies whatever you already have — including the mess.