mb.
Resume
All experience

Microsoft / Aug 2025 – Apr 2026

Grounding a field-sales agent in the right evidence

Software Engineer (Contract)

The problem

Field-sales teams needed answers from contracts, CRM records and internal databases that were spread across Salesforce, HubSpot, OneDrive and SQL systems. Retrieving the right passage was only one part of the problem: the agent also needed to maintain context, execute tools reliably and return an answer grounded in the evidence.

My role

I joined an existing team system as a contract software engineer. I implemented new LangGraph use cases and extended existing workflows, with direct work on retrieval, guardrails, failure controls and RLHF. The overall agent served 75,000 daily active users; the component/request category I worked on processed 1.5 million requests over six months.

Architecture / Microsoft

From a question to a grounded answer

Stateful orchestration, evidence retrieval and bounded failure handling.

Request / data flowEvent / ingestion flowControl / fallbackSwipe to explore →
From a question to a grounded answerStateful orchestration, evidence retrieval and bounded failure handling. Sales user: Natural-language; question. Input guardrails: Scope + input validation; Prompt-injection checks. LangGraph workflow: Plan → select tools → act; Shared state across nodes. Grounded response: Rerank supporting context; Citations → output checks. Failure controls: Planning timeout · capped retries · backoff; Fallback / dead-letter handling. Tool dispatch: Tool manifests + routing; Execute the selected plan. Output policy gate: Validate response before delivery; Return an explicit failure if needed. CRM & documents: Salesforce · HubSpot; OneDrive / contract sources. Internal SQL: Structured sales records; Query-specific tool access. Evidence retrieval: Document vectors + table metadata; Relevant child + parent context. Source documents: Long contracts; Tables + diagrams. Structure-aware chunks: Heading-scoped parent sections; Child passages for retrieval. Embeddings: BGE / Qwen representations; Incremental vector updates. PostgreSQL metadata: Table / diagram descriptions; Evidence and citation context. LangSmith traces → investigate divergent plans and tool failures. Human feedback → reward-model training → reinforcement learning → production model01 / REQUEST & ORCHESTRATIONA bounded execution path for field-sales questions02 / TOOL & RETRIEVAL LAYERSelect sources according to the query and current graph state03 / KNOWLEDGE PREPARATIONIncremental document processing preserves section context04 / OBSERVABILITY & LEARNINGProduction traces and a separate human-feedback training loopvector indexmetadataSales userNatural-languagequestionInput guardrailsScope + input validationPrompt-injection checksLangGraph workflowPlan → select tools → actShared state across nodesSTATEFULGrounded responseRerank supporting contextCitations → output checksRETURN TO USERFailure controlsPlanning timeout · capped retries · backoffFallback / dead-letter handlingTool dispatchTool manifests + routingExecute the selected planOutput policy gateValidate response before deliveryReturn an explicit failure if neededCRM & documentsSalesforce · HubSpotOneDrive / contract sourcesInternal SQLStructured sales recordsQuery-specific tool accessEvidence retrievalDocument vectors + table metadataRelevant child + parent contextSource documentsLong contractsTables + diagramsStructure-aware chunksHeading-scoped parent sectionsChild passages for retrievalEmbeddingsBGE / Qwen representationsIncremental vector updatesPostgreSQL metadataTable / diagram descriptionsEvidence and citation contextLangSmith traces → investigate divergent plans and tool failuresHuman feedback → reward-model training → reinforcement learning → production model
Conceptual view of the workflows I contributed to. Retrieval combines document vectors with table and diagram metadata; the failure controls apply across planning and tool execution.
Read the architecture as text

Stateful orchestration, evidence retrieval and bounded failure handling.

  • Sales user: Natural-language. question.
  • Input guardrails: Scope + input validation. Prompt-injection checks.
  • LangGraph workflow: Plan → select tools → act. Shared state across nodes.
  • Grounded response: Rerank supporting context. Citations → output checks.
  • Failure controls: Planning timeout · capped retries · backoff. Fallback / dead-letter handling.
  • Tool dispatch: Tool manifests + routing. Execute the selected plan.
  • Output policy gate: Validate response before delivery. Return an explicit failure if needed.
  • CRM & documents: Salesforce · HubSpot. OneDrive / contract sources.
  • Internal SQL: Structured sales records. Query-specific tool access.
  • Evidence retrieval: Document vectors + table metadata. Relevant child + parent context.
  • Source documents: Long contracts. Tables + diagrams.
  • Structure-aware chunks: Heading-scoped parent sections. Child passages for retrieval.
  • Embeddings: BGE / Qwen representations. Incremental vector updates.
  • PostgreSQL metadata: Table / diagram descriptions. Evidence and citation context.

LangSmith traces → investigate divergent plans and tool failures

Human feedback → reward-model training → reinforcement learning → production model

Keep document structure in the retrieval context

Long contracts contained headings spanning multiple pages, alongside tables and diagrams. Fixed-size chunks could separate an answer from the context that explained it. I introduced heading-based parent/child chunks so retrieval could find a relevant passage and retain its surrounding section.

The change improved retrieval-context relevance by approximately 30–40%. For tables and diagrams, I added descriptive metadata in PostgreSQL to complement vector retrieval and connect the answer to supporting evidence.

Make planning and tool execution bounded

I implemented graph nodes, edges, state and routing for new use cases, and extended existing graph paths. Shared state carried the query and retrieved context through planning, tool use and answer preparation.

Planning timeouts, capped retries, exponential backoff and fallback paths bounded failures. Stalled or unsuccessful runs could be handed off through dead-letter handling instead of continuing indefinitely. LangSmith traces supported investigation of divergent plans.

Validate both sides of the response

Input validation, scope checks and prompt-injection checks ran before agent execution; output checks ran before returning a response. Retrieval evaluation used relevance, precision, recall and ranking measures to assess quality.

I also implemented reward-model training and the reinforcement-learning stage of an RLHF pipeline whose resulting model reached production. The final satisfaction score was approximately 75%, assessed against the first three months of operation; this is a final score, not a 75% increase.

Outcomes

  • Delivered retrieval and orchestration components processing 1.5 million requests over six months within a field-sales agent serving 75,000 daily active users by implementing and extending workflows across CRM, document and SQL data sources on Azure.
  • Improved retrieval-context relevance by approximately 30–40% by replacing fixed-size chunks with heading-based parent/child chunks that preserved context across long contract sections.
  • Supported new sales-agent use cases while retaining context across tool calls by implementing LangGraph nodes, edges, shared state and routing, and extending the existing orchestration graph.
  • Bounded agent planning and tool-execution failure paths with timeouts, capped retries, exponential backoff and fallbacks by implementing explicit controls and dead-letter handling for stalled or unsuccessful runs.
  • Delivered an RLHF-trained model into production by implementing reward-model training and the reinforcement-learning stage using human feedback.

Tradeoffs

Larger parent sections preserve context but can increase the amount of material passed downstream. Reranking and selective context assembly help keep the answer focused. Bounded retries improve predictability, while explicit failure responses are still necessary when a source or tool remains unavailable.

Tools and technologies

PythonAzureLangGraphLangChainLangSmithPostgreSQLSQLPineconeBGE/Qwen embeddingsRAGrerankingRLHFCRM integrations

Related work

Role CollectorColdstart