The question usually arrives as a hiring plan or a vendor shortlist, but underneath it sits a different one: which parts of your SI work are your product, and which are plumbing any competent team could build? Answer that well and most of the decision follows. Answer it badly and you either spend a long stretch recruiting for work a partner could have shipped, or hand your core advantage to people who will leave with the knowledge.
This guide is vendor-neutral. It sets out the factors that decide the question, the hybrid models between the two extremes, what you should own whichever way you go, and the signs you chose wrong. If you are not yet sure the project needs custom SI at all, test that first with our build-vs-buy framework for when SI is the wrong answer.
Six factors that decide it
1. How central SI is to the product
If the SI behaviour is what customers pay for, the people who understand it deeply should eventually work for you. If it supports an internal process, such as triaging tickets or summarising documents, a partner can build and run it without putting your advantage at risk.
A useful test: if a competitor had access to the same models, what would make your version better? If the answer is your data, your evaluation cases and your domain judgement, those must sit in-house even if a partner writes much of the code.
2. The hiring market for experienced LLM engineers
Engineers who have run LLM systems in production, not just built prototypes, remain a small and contested pool. Many candidates show impressive demos and little experience of what happens after launch: evaluation, monitoring, provider model changes, incidents. Hiring well needs someone on your side who can tell the difference. If nobody can yet, your first hire is the riskiest one, and a bad one is expensive to unwind.
Be honest about your own recruiting record for senior technical roles. If those searches usually drag on, assume this one will too.
3. How fast you need the first production release
If the release is tied to a customer commitment, a funding milestone or a regulatory deadline, building a team first adds a stage before the real work starts: recruiting, onboarding and learning to work together. A partner with an existing team skips that stage, although it still has to learn your domain. Without a fixed date, building in-house starts slower but compounds, because every lesson stays with you.
4. The knowledge you must own long-term
Code can be handed over. Judgement is harder to transfer: why a prompt is phrased the way it is, which evaluation cases caught real failures, which data sources are unreliable and why. If that knowledge lives only in a partner's heads, you depend on them whatever the contract says. Decide which knowledge must sit with your own staff, and plan for it from the first stage rather than at handover.
5. Regulatory and data constraints
Some constraints rule options in or out, and most of them shape how you oversee a partner rather than forbidding one.
- Data location: customer contracts or residency rules may limit where data is processed and by whom. Check these before you shortlist anyone.
- EU personal data: if a partner processes it on your behalf, Article 28 of the GDPR requires a binding contract that covers, among other things, acting only on your documented instructions, confidentiality, security, and deleting or returning the data at the end.
- US banking: the agencies' revised model risk management guidance (OCC Bulletin 2026-13, April 2026) replaced the 2011 guidance and includes considerations for vendor and third-party models. It leaves generative and agentic models out of scope for now, with a request for information planned, and the 2023 interagency guidance on third-party relationships still sets expectations for managing vendors. Either way, the oversight stays with the bank.
- Audit evidence: in regulated work you need records of what the system did and why, whoever built it. Our guide to audit trails for LLM features in fintech covers what to keep.
6. Management capacity
An in-house team needs a technical lead who sets standards, reviews work and makes trade-offs. An outsourced team still needs a capable owner on your side who writes clear briefs, reviews releases and makes decisions without delay. If neither person exists, neither model will work well, and your first appointment should be that person rather than more engineers.
Should I build an in-house SI team or use an outsourced partner?
Weigh how central SI is to your product, how hard experienced LLM engineers are to hire, how soon you need a production release, what knowledge you must own, your regulatory and data limits, and whether someone can lead the work. SI that customers pay for favours in-house; supporting workflows suit a partner. Many teams use a hybrid, with a partner building while they hire and a planned handover. In every model, own the accounts, evaluation sets, prompts, data pipelines and runbooks.
The hybrid models
Most companies that handle this well end up between the extremes. Two patterns come up most often.
A partner builds while you hire
The partner delivers the first production release while you recruit. Knowledge transfer is designed in from the start, not left to the last stage: your new hires join code reviews, pair on features, take ownership of parts of the evaluation set, and eventually run a release with the partner watching rather than doing.
- Agree which roles you are hiring for, and whether the partner helps you assess candidates.
- Agree which areas transfer first, usually the evaluation suite and operations, then features.
- Write the handover criteria down, for example your team shipping a change and handling an incident without help.
- Decide the partner's role after handover: on-call support, periodic reviews, or nothing at all.
An embedded dedicated team
Engineers from the partner work inside your processes: your tools, your stand-ups, your code review rules, with a lead on your side setting priorities. It behaves like an in-house team without the recruiting, and it suits companies that have the management capacity but not the hiring pipeline. The risk is that it becomes permanent by default. Decide whether the goal is a long-term extension or an eventual handover, and revisit that at each stage.
What to own in every model
Whichever model you choose, some things should belong to you from the first day. They are what let you change partners, bring work in-house or simply understand your own system.
- Accounts: model provider, cloud, repositories, monitoring and secrets, all in your organisation's name, with outside engineers as invited users.
- Evaluation sets and results: the cases, the labels, the scoring code and the history of scores for each release. This is your working definition of "good enough", and our note on eval-first delivery explains why it matters so much.
- Prompts and system instructions, in version control, with the reasons for significant changes recorded.
- Data pipelines: ingestion, cleaning, chunking and indexing code, plus documentation of each source and its owner.
- Runbooks: what to do when quality drops, a provider retires a model, usage spikes or a connected system fails.
- A decision log: the important trade-offs and why they were made.
If you could not hand these to a new team and have them run the system, you do not own it yet. Our checklist for outsourcing SI development to Pakistan covers the contract terms that secure this, and most of it applies to any offshore partner.
Signs you chose wrong
In-house was the wrong call if
- Open roles stay unfilled and the roadmap waits on recruiting.
- The team rebuilds infrastructure that a managed service or partner already provides, instead of working on the product.
- One or two people hold all the knowledge, and one resignation would stall the work.
- Releases are judged on demos because nobody has had time to build the evaluation suite.
Outsourcing was the wrong call if
- Your team cannot explain why the system behaves as it does without asking the partner.
- Every small change needs a new statement of work.
- Accounts, keys or evaluation data sit with the partner.
- SI has become your main differentiator, but nobody on your payroll understands it deeply.
- The handover keeps slipping because its criteria were never written down.
Either situation can be fixed. Changing course mid-project is disruptive, but it does less damage than staying with the wrong model, and it is far easier if you already own the artefacts listed above. If you want an outside view on which way to go, an independent SI consulting review can map the six factors against your situation before you commit to hiring or signing.
Where Octalcode fits
Octalcode is an outsourced partner, so read this section with that in mind. We are a studio in Lahore, with a US presence in Austin, and our senior engineers build LLM applications, SI agents and RAG systems, either as a defined build or as a team embedded in a client's own processes.
If you have not decided yet, our SI consulting work helps you sort what to own, what to hand to a partner and how a handover would be judged. Sometimes the honest answer is to hire rather than outsource, and we will say so.