LLM.co delivers private, secure large language model (LLM) solutions tailored for enterprises operating in highly regulated industries such as law, healthcare, finance, and government. We build and deploy domain-specific AI tools that help our clients gain insight and efficiency while maintaining full control over their data and compliance requirements.

Position Overview

We are looking for a Senior Software Engineer with a strong background in backend systems, distributed infrastructure, and AI product integration. You’ll be at the core of building robust, scalable software systems that enable the secure deployment, customization, and orchestration of large language models for enterprise use.

As a key engineering leader, you'll work closely with AI researchers, DevOps engineers, and product teams to bring cutting-edge LLM capabilities into real-world enterprise environments—on-prem, cloud, or hybrid.

Key Responsibilities

• Architect, develop, and maintain high-performance backend systems for LLM deployments
• Design APIs, services, and tooling to support model fine-tuning, inference, and RAG (retrieval-augmented generation) pipelines
• Collaborate with AI and ML engineers to integrate models into scalable and secure enterprise software workflows
• Build modular, reusable components for BYOD (Bring Your Own Data) ingestion, vector indexing, and embedding pipelines
• Optimize systems for speed, security, scalability, and maintainability across varied deployment environments (cloud/on-prem)
• Contribute to CI/CD pipelines, system observability, and automated testing infrastructure
• Lead code reviews, mentor junior engineers, and contribute to architectural decisions
• Stay current with the latest advancements in LLM orchestration, security best practices, and distributed system design

Required Qualifications

• Bachelor's or Master’s degree in Computer Science, Engineering, or a related field
• 5+ years of software engineering experience, with a focus on backend or systems engineering
• Strong proficiency in Python and at least one other language (Go, Rust, TypeScript, etc.)
• Experience building APIs (RESTful or GraphQL) and backend services
• Familiarity with containerization, orchestration, and cloud infrastructure (Docker, Kubernetes, AWS/GCP/Azure)
• Strong understanding of data structures, concurrency, and distributed systems
• Comfort working with AI/ML teams and integrating AI models into productized environments
• Experience with SQL and NoSQL databases, vector stores (e.g., Pinecone, Qdrant, Weaviate), and RAG frameworks

Preferred Qualifications

• Experience integrating LLMs and NLP systems into production workflows
• Exposure to LangChain, LlamaIndex, HuggingFace Transformers, or similar tools
• Security-first mindset with experience in enterprise authentication, authorization, and compliance
• Background in DevOps, observability, or cloud automation
• Previous work in legaltech, healthtech, or fintech sectors

What We Offer

• Competitive salary and stock options
• Remote-first flexibility with collaborative team culture
• Opportunity to work on frontier AI technologies for high-impact industries
• Access to GPU compute and state-of-the-art LLM infrastructure
• Influence over architectural and product decisions at an early stage

‍