Contact
Contact

CAREER OPPORTUNITIES > MALAYSIA

Lead AI Infrastructure & Inference Engineer – AI Engineering & Operations

The Lead AI Infrastructure & Inference Engineer is responsible for the hosting, deployment, and optimization of open-source model weights — including the latest DeepSeek series models — on Apexone's private infrastructure. You will ensure that Enterprise AI running inside Apexone's data centers operates with minimal server overhead, fast response speeds, and absolute data sovereignty. This role is critical to Apexone's sovereign AI strategy. Unlike vendors that rely on public cloud APIs, we deploy and serve open-weight models entirely within client-controlled infrastructure, ensuring zero data leakage and full compliance with Malaysia PDPA, Oman PDPL, and GDPR.

You will architect private model server setups that automatically balance workloads during high-traffic trading windows on our Apexium Centralized Exchange (CEX), usage spikes on our Universal Loyalty Ecosystem, and Enterprise AI workloads across Digital Banking, Smart Manufacturing, and Smart City platforms. Your work will directly enable Apexone's Human-in-the-Loop AI philosophy — delivering AI that augments human potential while maintaining strict performance, privacy, and sovereignty guarantees.

Model Deployment & Optimization

LOCATION

DEPARTMENT

REPORTS TO

One City, Malaysia (with regional travel as required)

AI Engineering & Operations

Chief Technology Officer (CTO)

Core Responsibilities

Key Role Objectives:
A. Model Deployment & Inference Optimization

  • Open-Source Model Hosting: Download, host, and serve open-source model checkpoints — including the latest DeepSeek series (e.g., DeepSeek-V4, DeepSeek-R1) and their distilled variants — on local private hardware using high-speed inference engines.

  • Inference Engine Selection & Tuning: Deploy and optimize models using production-grade inference frameworks:

    • vLLM — high-throughput serving with dynamic batching and support for the latest model architectures.

    • TensorRT-LLM — latency optimization for NVIDIA GPUs with 40%+ latency reduction.

    • SGLang — recommended for full-capability MoE model deployment and structured generation.

    • Ollama — for distilled model variants, rapid prototyping, and lightweight deployments.

  • Distributed Inference Architecture: Design and deploy multi-node, multi-GPU inference clusters for full-capability models. DeepSeek full models require 16+ H100 GPUs deployed across multiple machines, with distributed serving configurations across Kubernetes clusters and Ray-based orchestration.

  • Model Registry & Versioning: Establish private model registry for tracking model versions, quantization formats, performance benchmarks, and deployment configurations across all instances.

B. Model Quantization & Weight Compression

  • Advanced Quantization Techniques: Compress massive open models using state-of-the-art quantization to cut hardware costs while maintaining performance:

    • AWQ (Activation-aware Weight Quantization) — INT4 quantization with minimal accuracy loss.

    • GPTQ (Generative Pre-trained Transformer Quantization) — INT4/INT8 weight-only quantization.

    • FP8 Quantization — natively supported by latest DeepSeek models for efficient inference.

    • GGUF — for Ollama-based deployments and CPU/GPU hybrid inference.

  • Quantization Impact Management: Achieve ~5% quality loss with INT4 quantization while reducing memory footprint by 50-75%. For a 330B parameter model: ~66 GB (BF16) → ~16-17 GB (GPTQ-4bit).

  • Hardware-Specific Optimization: Tailor quantization strategies to target hardware — NVIDIA H100/A100/H200 (TF32/FP8), AMD GPUs via SGLang, or Huawei Ascend NPU for sovereign deployments.

C. Private DevOps & Scale Architecture

  • Private Model Server Architecture: Architect local, private model server setups that automatically balance workloads during high-traffic trading windows on Apexium Centralized Exchange (CEX), active usage spikes on Universal Loyalty Ecosystem, and Enterprise AI workloads across all platforms.

  • Kubernetes-Native Deployment: Deploy and manage inference workloads using Kubernetes — with KubeRay for distributed Ray clusters, Kubernetes Operators for model lifecycle management, and GPU-aware scheduling with node affinity and taints.

  • Infrastructure as Code: Implement Terraform-based infrastructure provisioning for reproducible, auditable, and version-controlled AI infrastructure.

  • CI/CD for AI: Establish GitHub CI/CD pipelines with aggressive caching, model validation gates, cost controls, and automated rollback capabilities for model deployment and updates.

  • Storage & Data Management: Architect high-performance storage solutions — NVMe PCIe 4.0 with read speeds ≥7000 MB/s, with data disk capacity at 1.5x model size (DeepSeek series: ~1.3 TiB → 2 TiB).

D. GPU Infrastructure & Hardware Management

  • GPU Cluster Management: Deep knowledge of NVIDIA GPU hardware management — H100, A100, H200, B200 — including multi-node clustering, NVLink/NVSwitch topology optimization, GPU memory allocation, and NCCL tuning.

  • Performance Monitoring & Tuning: Implement GPU utilization monitoring, latency tracking, throughput optimization, and model performance benchmarking across inference clusters.

  • Capacity Planning: Forecast GPU capacity requirements based on model growth, new model releases, traffic patterns, and business demand across all platforms.

  • Driver & CUDA Management: Manage NVIDIA driver versions (550+ recommended) and CUDA environments (12.4+) across the GPU fleet with rolling updates and compatibility testing.

E. Sovereign AI & Security Compliance

  • Private/On-Premises Deployment: Configure private containers (Docker/Kubernetes) and localized GPU runtimes (vLLM/Ollama) directly inside client infrastructure, ensuring zero data leakage and compliance with regional data sovereignty regulations.

  • Security Hardening: Implement model access controls, encrypted model weights at rest, secure API endpoints for inference services, and audit logging for all model interactions.

  • Audit & Compliance: Ensure all AI infrastructure deployments meet ISO/IEC 27001, PDPA, PDPL, and GDPR requirements for data protection and sovereign AI operations.

  • Forward Deployed Engineering (FDE) Enablement: Build standardized deployment templates, runbooks, and tooling that enable FDE teams to deploy private AI models on client infrastructure independently.

F. Continuous Innovation & Technology Radar

  • Model & Framework Evaluation: Continuously track, evaluate, and benchmark the latest open-source models (e.g., DeepSeek, Llama, Qwen, Mistral) and inference frameworks to ensure Apexone's AI infrastructure remains at the forefront of performance, cost-efficiency, and capability.

  • Rapid Prototyping: Establish a framework for rapidly prototyping and stress-testing new model releases in a sandboxed environment to assess their viability for production deployment.

  • Performance Benchmarking: Maintain up-to-date performance benchmarks across model variants, quantization levels, hardware configurations, and inference engines — enabling data-driven decisions for model selection.

Required Qualifications & Experience:

  • 3+ years of experience in AI/ML infrastructure, MLOps, or production-grade model deployment

  • Proven experience deploying and serving large language models (100B+ parameters) in production environments — preferably including the latest DeepSeek series (e.g., V4, R1) or similar open-weight models (Llama, Qwen, Mistral)

  • Deep expertise in at least two production inference frameworks:

    • vLLM — high-throughput serving

    • TensorRT-LLM — NVIDIA-optimized inference

    • SGLang — structured generation for MoE models

    • Ollama — lightweight deployment

  • Hands-on experience with model quantization techniques: AWQ, GPTQ, FP8, or GGUF

  • Production Kubernetes experience — including GPU scheduling, KubeRay, Kubernetes Operators for AI workloads, and Helm charts

  • Infrastructure as Code — Terraform, CI/CD pipelines (GitHub Actions/GitLab CI), and model deployment automation

  • Deep knowledge of NVIDIA GPU hardware (H100/A100/H200/B200) management, CUDA, NCCL, and performance optimization

  • Strong programming skills — Python, PyTorch, HuggingFace Transformers

  • Linux system expertise — Docker, kernel tuning, storage optimization, network configuration for distributed training/inference

  • Preferred Qualifications:

    • Experience with sovereign AI deployments in regulated industries (Enterprise AI, FinTech, Banking, Government, Healthcare)

    • Knowledge of alternative hardware platforms — AMD GPUs, Huawei Ascend NPU, or AWS Trainium/Inferentia

    • Experience with DeepSeek Orchestrator or similar open-source infrastructure frameworks

    • Contributions to open-source inference frameworks or quantization libraries

    • Experience with vector databases (Qdrant, Milvus, pgvector) and RAG pipeline infrastructure

    • Experience with model distillation and knowledge transfer techniques

    • Experience with Federated Learning or Privacy-Preserving Machine Learning frameworks

Success Criteria in First 12 Months:

  • Successfully deploy and serve at least two production-grade open-source models (including the latest DeepSeek series) on private infrastructure with <100ms latency and 99.9% uptime

  • Establish quantization pipeline reducing model memory footprint by 50%+ while maintaining <5% quality degradation

  • Architect and implement auto-scaling inference cluster capable of handling 3x peak traffic spikes on Apexium Centralized Exchange (CEX), Universal Loyalty Ecosystem, and other platforms

  • Achieve 40% reduction in inference cost through quantization, workload optimization, and efficient GPU utilization

  • Build private model registry with versioning, performance benchmarks, and automated deployment pipelines

  • Establish GPU capacity planning framework forecasting 12-month infrastructure needs based on model growth and traffic patterns

  • Enable Forward Deployed Engineers (FDE) to deploy private AI models on client infrastructure through standardized deployment templates, runbooks, and tooling

  • Establish model benchmarking framework enabling data-driven model selection for new use cases within 2 weeks of model release

  • Successfully evaluate and prototype at least 4 new open-source model releases for production viability

EMPLOYMENT TYPE: FULL-TIME, PERMANENT

APPLICATION PROCESS

Submit Your Application

Ready to lead enterprise transformation in Oman?

Our recruitment process: CV Review → Technical Assessment → Leadership Interview → Offer

Apexone is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

FUTURE OPPORTUNITIES

Ready to Transform Your Career?

Don't see the perfect role? We are always looking for exceptional talent in Green Tech, AI, Big Data, IoT, Blockchain, Cybersecurity, and Enterprise Architecture. Submit your CV to our talent pool or contact us at