If you are looking at Pakistani teams to build an LLM application, an SI agent or a RAG system, you will not run short of options, and most proposals will look alike: a confident deck, a polished demo and a long list of model names. The differences that decide whether your project works sit underneath that, in how the team communicates, who owns what when the work ends, and how they prove a system is ready.
This is the checklist we would use if we were on the buying side. It does not compare rates or argue that one country is cheaper than another. Those comparisons go stale quickly and say little about whether a team can ship. It covers the questions that separate a vendor you can rely on from one you end up supervising more closely than the work itself.
First, should this be outsourced at all?
Before you choose a vendor, check that the project needs custom SI work. Plenty of teams commission an agent when a configured SaaS tool or a plain workflow would do the job for less effort and less risk. Our build-vs-buy framework for when SI is the wrong answer is a quick way to test that. A good vendor will raise the same question in the first call, and whether they do is a useful signal in itself.
If custom work is the answer, be clear about what you are buying: a defined build with a launch date, engineers who extend your own team, or advice before you commit to anything. Those are different contracts with different risks. If the scope is still fuzzy, a short discovery phase that ends in a written plan, like the steps in how to build an SI agent for your business, is cheaper to get wrong than a full build.
What should I check before outsourcing LLM development to Pakistan?
Check how the vendor proves quality, who owns the results, and how the team works across time zones, before you compare price. Pakistan keeps UTC+5 all year, so Lahore sits nine or ten hours ahead of US Eastern time and four or five ahead of the UK; agree the overlap hours and on-call cover in writing. Make sure a signed assignment covers code, prompts, evaluation data, indexes and any fine-tuned weights, and that cloud and model accounts are in your name. For EU personal data, Pakistan has no adequacy decision, so you need standard contractual clauses and a data processing agreement. Ask for an anonymised evaluation report rather than a demo, and how they test for prompt injection and data leakage. Call references from similar projects, request written security policies, and agree monitoring, model changes, handover and exit terms before work starts.
Time-zone overlap and how the team communicates
Pakistan runs on Pakistan Standard Time, UTC+5, all year, with no daylight saving change. That puts Lahore nine or ten hours ahead of US Eastern time depending on the US season, and four or five hours ahead of the UK. A normal Lahore working day overlaps with the European and UK morning and early afternoon. Overlap with the US is thin unless the team shifts its hours, which many teams serving US clients do. Either way, get it in writing rather than assuming it.
- Which hours will the engineers doing the work be reachable, not only the account manager?
- Who joins your calls: the people who write the code, or a project manager relaying messages?
- Where are decisions recorded: tickets, a shared decision log, recorded walkthroughs of each release?
- What happens when production breaks outside the overlap window, and who is on call?
- How do they report progress between calls? A team that writes clear asynchronous updates needs far less overlap than one that relies on meetings.
Ask to speak with the lead engineer before you sign. How clearly they explain trade-offs, and whether they push back on parts of your brief, tells you more than any sales call.
Who owns the code, the data and the model artefacts?
In an SI project the deliverable is more than source code. It includes prompts and system instructions, evaluation datasets, retrieval indexes and the pipelines that build them, fine-tuned weights or adapters if there are any, infrastructure code, and the accounts with model providers and cloud hosts. Each of these needs a named owner in the contract.
- Ownership: under US copyright law, paying a contractor does not on its own make the code yours; ownership normally moves through a signed, written assignment. Have your lawyer confirm how the governing law in your contract handles this, and make sure the assignment lists everything above, not just source code.
- Accounts: model provider, cloud and repository accounts should sit in your organisation's name from the start, with the vendor's engineers as invited users. Untangling them later is slow and sometimes not possible at all.
- Pre-existing tools: vendors often reuse their own libraries and templates. That is fine, as long as you receive a licence that survives the end of the engagement.
- Your data: where it is stored, who can see it, a written promise that it is used for nothing else, and how it is deleted when the work ends.
- Cross-border transfers: Pakistan is not on the European Commission's list of adequacy decisions, so sending EU personal data there needs another safeguard, usually standard contractual clauses. Ask the vendor to sign a data processing agreement and have your privacy counsel review it.
- The legal entity: confirm which company signs, and check it on the national registry, which in Pakistan is the Securities and Exchange Commission of Pakistan.
Proof: evals and references, not demos
How they test
A demo is the easiest thing to get wrong by accident. Someone picks questions that work, the answers look fluent, and everyone relaxes. LLM systems fail on the questions nobody picked, so the real question is how the vendor knows a system is good enough to launch.
A strong answer describes an evaluation set built from your real questions, scored automatically on every change, with clear rules about what blocks a release. For retrieval systems, retrieval and answer quality should be scored separately. Our list of questions to ask a RAG development company goes deeper on this, and most of it applies to agents and copilots too.
- Ask to see an anonymised evaluation report from a past project, not a slide about their process.
- Ask who builds and labels the test set. Your domain experts should be involved, because the vendor does not know your edge cases.
- Ask what stops a release: a score that must hold, or a judgement call made under deadline pressure.
- Ask how they test for misuse, including the risks in the OWASP Top 10 for LLM Applications such as prompt injection, sensitive information disclosure and excessive agency. A team that has done this will have a prompt injection testing checklist or something like it.
- Ask how quality will be measured once real users arrive, since their questions will not match the test set exactly.
References you can actually call
Case studies on a vendor's website are marketing, and short online reviews say little about a long build. Ask for references from projects similar to yours and speak to them on a call. Then ask them things a case study will never cover.
- What went wrong during the project, and how did the vendor handle it?
- Did the people who started the work stay on it, or did the team change halfway?
- Is the system still running, and who maintains it now?
- Would they hire the same team again for the next project?
Also ask the vendor whether the engineers on the reference project will work on yours. A strong reference is worth less if the team behind it is busy elsewhere. A vendor that cannot produce a single client willing to talk is telling you something, even when the reasons sound plausible.
Security practice, in writing
Verbal reassurance costs nothing. Ask for the written policies and for answers you can check.
- Access: named accounts, multi-factor authentication, least privilege, and access removed the day someone leaves the project.
- Secrets: no API keys in code, tickets or chat messages; a secrets manager instead.
- Devices: managed laptops with disk encryption, and a rule about personal machines.
- Data in development: whether production data is ever copied to test environments, and how it is masked if so.
- Incidents: who tells you, how quickly that is promised in the contract, and what is logged so the cause can be traced.
- Certifications: if a vendor claims SOC 2 or ISO 27001, ask for the report or certificate and check its scope. A team that says it is aligned with those practices without an audit can be perfectly sound, as long as it says so plainly.
What happens after launch?
SI systems change after launch even when nobody touches the code. Providers update and retire models, your documents change, and the questions users ask drift away from the ones you tested. The contract should say who notices and who acts.
- Monitoring: who watches quality, cost and errors, and which dashboards your team can see without asking.
- Model changes: how a provider retiring or changing a model is handled, and whether the evaluation suite runs before any switch.
- Support: what is covered, how requests are raised, and the response expectations written into the agreement.
- Handover: documentation, runbooks, and at least one release your own team deploys without the vendor in the room.
- Exit: what you receive if the contract ends, and how the vendor's access is removed.
The best sign in a vendor is one that plans for the day you no longer need them.
Where Octalcode fits
Octalcode is one such vendor: a studio in Lahore, with a US presence in Austin, that builds LLM applications, SI agents and RAG systems. We would expect you to put us through this list like anyone else, and we would rather you did.
If you want a second opinion before you speak to any vendor, including us, our SI consulting work starts with the questions above: whether to build at all, what to own, and how to measure it. You leave with a written view either way.