13 — Zoho Leverage Plan¶

Every previous document in this Phase points here. The roadmap is not a self-improvement exercise; it is a career campaign. Zoho is your leverage. The company already has cloud + on-prem infrastructure, a real agentic product surface, and a customer base that structurally requires on-prem AI. Almost no other engineer in the world has that combination of situational advantages. This document lists the specific plays that convert Phase 7 study into promotion / equity / autonomy / market position.

The frame: your unfair advantage is that you can move up-stack faster than a hyperscaler infra engineer, and down-stack faster than a Zoho application engineer. Every play below exploits that.

1. The four flagship plays (do all four, in this order)¶

Play 1: Prefix-cache-aware serving for agent loops¶

Status: highest-ROI, immediate, publishable internally in 30 days.

The agentic products you already work on have a specific traffic profile: long, stable system prompts (5-10K tokens including tool descriptions and role definitions), plus turn-by-turn user messages. Every tool call restarts the conversation with the full history. This is the exact workload SGLang’s RadixAttention was designed to eat.

What you do:

Instrument the current production LLM traffic (whether via OpenAI API, Zoho’s internal endpoints, or a self-hosted vLLM) to measure [system_prompt_prefix, per-turn tokens_added, cache-hit-rate-if-caching-were-enabled]. Two week measurement window.
Compute the theoretical prefill savings from prefix caching. Expect 60-90% reduction in prefill tokens processed on multi-turn sessions. Expect 30-50% reduction in total $/token given prefill/decode mix.
Prove it on a bench: same workload, same model, --enable-prefix-caching vs off. Publish the delta internally with a one-slide summary.
If Zoho serves via external API: this becomes a cost-per-session-halved memo. If Zoho serves via internal vLLM: this becomes a config change + rollout. Either way you own the story.

What makes this leverage: the ROI is expressible in dollars per month per product, in exactly the format finance teams and VPs recognize. It also positions you as the inference person internally in a way that no other work does. Every product team with an agentic feature now needs to talk to you.

Deliverable: an internal memo titled something like “Prefix cache measurement across Zoho agentic products: expected 40-60% inference cost reduction, one-config-flag rollout plan.” Timeline: 30 days from start.

Play 2: Internal LLM benchmark harness¶

Status: political + technical play. Do this second, in months 2-4.

Zoho serves multiple LLM-using products across cloud and on-prem. Each team currently makes model + engine + quantization decisions informally. There is no shared benchmark. You build the harness that becomes the authority on those decisions.

What you build:

A repo (zoho-llm-bench or similar) that runs a fixed evaluation battery against any {model, engine, quantization, hardware} tuple.
Batteries: (a) latency profile (TTFT, ITL at target concurrencies), (b) quality battery (task evals: extraction, summarization, tool-use, plus a Zoho-domain suite you construct from real product traffic — carefully de-PIIed), (c) cost extraction ($/1M token from measured goodput ÷ GPU-hour cost).
Output: a report card per tuple, published in an internal dashboard.
CI integration: any team can wire their target tuple to a nightly bench run.

What makes this leverage: whoever owns the benchmark owns the vocabulary for the decisions. When Zoho debates “should we move from GPT-4.1 to Llama 3.3 70B for product X”, the debate is settled by your benchmark. This is a Chesterton’s-fence position — it takes time to build, but once built it is nearly impossible to displace, and it makes you an ex-officio participant in every model-selection meeting.

Deliverable: the benchmark tool + a first published report card comparing 3 models × 2 engines × 2 quantizations on a Zoho-representative workload. Timeline: 3 months from start.

Play 3: On-prem model registry + license compliance system¶

Status: institutional play. Months 4-8.

Zoho’s on-prem customers need models that can be shipped legally, reproducibly, and safely (per 11_security_supply_chain.md). There is currently, almost certainly, no formal system for this. You build one.

What you build:

Internal model registry (MinIO or S3-compatible + Postgres for metadata, per 07_on_prem_enterprise.md §2).
Signed manifests per model artifact.
License compliance database: for each model, the license, the terms, the restrictions, the tenant classes it can be shipped to. This is the document Zoho legal will actually consume.
Automated ingestion pipeline: pull model from HF (or vendor), verify hashes, sign, tag with license class, register.
Distribution mechanism: air-gapped customer sites pull from a customer-specific mirror of the registry.

What makes this leverage: two-layered. Technically, you become the only person who understands the model provenance chain at Zoho. Politically, you have solved a problem that Zoho’s legal team has probably been sweating quietly about for two years. Legal + security + on-prem product all now depend on your system.

Deliverable: working registry + license doc + at least one on-prem customer shipping models through it. Timeline: 4-6 months. Get product exec sponsorship early; this play needs air cover.

Play 4: Fine-tuned small model champion¶

Status: economic + product play. Months 6-12.

For any Zoho product where the LLM is used in a bounded task (extract fields from a ticket, classify a lead, draft a follow-up email in a fixed style), a fine-tuned 7-14B model beats prompted-frontier-model on cost by 10-100x, and often matches on quality. You pick one such workload, do the fine-tune, own the deployment.

Sequence:

Identify the workload. Best candidates: high-volume + bounded-output + latency-sensitive (extraction, classification, structured summarization, template generation).
Curate a fine-tuning dataset from existing product traffic. This is where you spend the time; the actual fine-tune is 4-8 hours of compute.
Fine-tune Qwen 2.5 7B or 14B (Apache-2.0, on-prem safe) with LoRA. Evaluate against the prompted-GPT-4.1-class baseline on your benchmark harness (Play 2).
If quality holds: propose replacing the prompted-frontier deployment with the fine-tuned small model. Attach the cost delta ($X/month → $X/10/month) and quality delta (equal or better on task evals).
Ship it. Own it for a quarter, tune it, document the wins.

What makes this leverage: you produce a measured cost reduction attributable to your work. Not “faster response times” or “more scalable” — an actual line-item change in the AWS bill or the API-usage bill or the GPU-hour bill. This is the most legible-to-leadership form of engineering output.

Deliverable: one shipped fine-tuned model + a quarterly cost impact report. Timeline: 6-8 months from start including data curation, training, eval, deployment, and one quarter of production observation.

2. What to say yes to, what to say no to¶

The prerequisite for all four plays is scope. You will get asked to do many things at Zoho over the next 12 months. Optimize your yes / no responses to feed the campaign:

Say YES to:

Any on-prem LLM deployment project. First choice, always.
Anything involving a customer with a data-residency requirement.
Cost-optimization requests for LLM-heavy products.
Model-selection discussions.
Cross-team meetings about agentic behavior in production.
Any escalation involving LLM latency or availability.
Kubernetes / infrastructure / GPU capacity discussions.

Say NO (politely, with an alternative) to:

Pure application-layer work that another engineer could do.
Being the only agent-loop implementer on a specific product feature. (Contribute the serving layer; let others own the harness UI.)
Non-LLM ML work that pulls you away from the inference discipline you’re building. (Legacy recommender bug fixes, generic classifier improvements — unless they’re the fine-tuned-small workload.)
Anything that has “AI” in the name but is actually “traditional ML” or “rules engine”. Redirect to a colleague.

The heuristic: does this task move me one step closer to being the inference authority at Zoho? If yes, take it. If no, delegate.

3. Visibility calendar¶

You need three artefacts per quarter that are legible externally (inside Zoho and outside). Rough schedule:

Quarter	Internal artefact	External artefact
Q1	Prefix-cache measurement memo (Play 1)	Blog post: “Measuring prefix cache hit rate on a real agentic workload” (r/LocalLLaMA and Zoho’s engineering blog)
Q2	First benchmark harness report card (Play 2)	Talk proposal to a local meetup or MLSys workshop
Q3	Model registry + license doc (Play 3)	Blog post: “The safetensors-only enterprise: our model supply chain”
Q4	Fine-tuned small model production report (Play 4)	Case study (with Zoho PR approval): “Fine-tuning a 7B model to replace API calls, $X/month saved”

External artefacts do two things: they establish market position (recruiters at Anyscale, Together, Fireworks, Baseten start reading your name), and they create a public ratchet against Zoho — you become someone whose loss would be embarrassing.

4. Skills-to-role mapping¶

The Phase 7 skills that map to specific Zoho promotions or role changes:

Skill from Phase 7	Zoho role opportunity
Full-stack inference (Phases 0-7 combined)	Staff/Principal engineer track, LLM infrastructure
Kubernetes + GPU + observability	Platform/SRE lead for AI infrastructure
On-prem deployment expertise	Field engineer for enterprise on-prem AI (customer-facing, high-leverage)
Model + license selection	Technical PM adjacent to AI products
Fine-tuning + evaluation	Applied ML lead for a specific product line
Benchmark harness ownership	Cross-org AI infrastructure architect

Notice that these are not competing roles — they are the same person seen through different org lenses. The unifying identity is “person who owns LLM serving at Zoho.” Frame your campaigns and reviews with that identity.

5. Escape-hatch options (should you leave Zoho)¶

The plays above build the Zoho position, but they also build the resume that opens external doors. In descending order of specialization:

vLLM / SGLang / FlashInfer team hire. Open-source contributions (from Phases 2-6) plus enterprise deployment experience (Phase 7) is the ideal profile. Anyscale, LMSYS, Nvidia teams behind these projects hire directly from OSS contributors.
Enterprise inference startups. Baseten, Fireworks, Together, Modal, Anyscale — all hire people with your exact profile. Baseten in particular targets the enterprise on-prem-adjacent market where your Zoho experience is legible.
Hyperscaler LLM infra teams. AWS Bedrock, GCP Vertex AI, Azure AI Foundry — senior IC roles. Your gap is “worked at massive scale” which the Zoho experience partially fills.
Hardware vendor solutions architect. Nvidia, AMD, Intel. Field roles work with customers on inference deployments. Zoho + on-prem + benchmark expertise maps directly.
Independent consulting on enterprise LLM deployments. The four plays above form a productizable methodology. Zoho would be your first case study.

You should not need to use any of these in the next 24 months if the Zoho plays land. But the fact that you could is what gives you leverage in the current job.

6. Anti-patterns (the ways this plan fails)¶

Drifting into application work. Every hour spent building a UI or wiring an OAuth flow is an hour lost from the inference discipline. Guard this ruthlessly.
Skipping the measurement / publishing loop. Doing the work but not publishing it internally means someone else claims the win or the win is invisible to leadership. Publish everything.
Being “helpful” instead of strategic. Zoho will absorb infinite scope; you have to be the one who says “the value here is in the serving layer, and I’ll own that.”
Chasing the frontier at the expense of production. Trying to run DeepSeek-V4 in production is exciting but not what pays Zoho’s bill. The 7B fine-tune that saves $X/month per product is worth ten frontier-model experiments.
Not building the benchmark harness first. Without Play 2, you can’t defend the numbers in Plays 1, 3, and 4. Prioritize it.
Getting siloed in one product. The plays only compound if you can bring wins from Product A to Product B. Insist on cross-product scope from the start.

7. The one-year outcome¶

If Plays 1-4 land, at the 12-month mark:

You are the named inference-serving owner at Zoho for at least 2 product lines.
There is a documented, measurable cost reduction (aim for $X00k/year across product lines) attributable to your work.
You have shipped at least one on-prem enterprise deployment with your name on it.
You have 2-3 external publications (blog posts, talks, or benchmark reports) that are cited when people search “on-prem LLM serving.”
The internal benchmark harness is in daily use by at least 3 teams.
You have made at least one merged OSS contribution to vLLM, SGLang, or FlashInfer (the Phase 6 discipline continues).
You have a defensible answer to “what should we do next quarter” that spans architecture, cost, and capability.

That is a staff-engineer profile in specific, legible artefacts. What promotion committees look for.

8. Reading list¶

Camille Fournier, The Manager’s Path — for the political and career-visibility mechanics.
Will Larson, Staff Engineer — for the archetypes and how to slot into one.
The Character.AI engineering blog — evidence of what “owning inference at a company” looks like when done publicly.
Existing Zoho engineering blog posts on infrastructure work — read them all, understand the internal writing style and audience, match it.

9. Exit test¶

You have executed this plan when:

Play 1 shipped: prefix caching enabled on a Zoho agentic product, measured cost delta reported.
Play 2 shipped: benchmark harness in use by at least three product teams.
Play 3 shipped: on-prem model registry live, at least one enterprise customer using it.
Play 4 shipped: at least one fine-tuned small model in production replacing a prompted-frontier deployment.
Your title, scope, or compensation reflects the work — or you have accepted an external role that does.

If you complete Phases 0-6 of the roadmap and Plays 1-4 of this document, you are, by construction, one of the most valuable inference engineers in enterprise AI in 2026-2027. That is the destination. The 13 preceding documents in this folder are the map.